Skip to content

Add preliminary instructions for energy measurement #38

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add preliminary instructions for energy measurement #38

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,33 @@ For each test, you should see output like
Latency 9.58ms 6.03ms 94.90ms 85.57%
Req/Sec 9936.90 2222.61 10593.00 95.24
```

#### Measuring energy consumption

You can also measure energy consumption, rather than throughput.
This tooling is still evolving, so the current process is a bit manual and needs several windows. The current tooling requires MacOS or Linux.

1. Clone the `power-server` utilities, https://github.com/metacosm/power-server. These provide an hardware-independent abstraction layer over hardware power metric tooling.
2. Build with `mvn verify`
3. Change directory to the cli: `cd cli`
4. In the first window, launch the Spring or Quarkus application, wrapped in the power-server application:
```shell
<your-code-dir>/spring-quarkus-perf-comparison/scripts/infra.sh -s
java -jar target/quarkus-app/quarkus-run.jar -c "java -XX:ActiveProcessorCount=4 -Xms512m -Xmx512m -jar <your-code-dir>/spring-quarkus-perf-comparison/quarkus3/target/quarkus-app/quarkus-run.jar"
```
or
```shell
<your-code-dir>/spring-quarkus-perf-comparison/scripts/infra.sh -s
java -jar target/quarkus-app/quarkus-run.jar -c "java -XX:ActiveProcessorCount=4 -Xms512m -Xmx512m -jar <your-code-dir>/spring-quarkus-perf-comparison/springboot3/target/springboot3.jar"
```
5. Meanwhile, open a second window and start the load test. When it's finished it will stop the application, which will terminate the energy measurement:
```
<your-code-dir>/spring-quarkus-perf-comparison/scripts/energy-stress-and-stop.sh
```

The energy measurement is subject to the same qualifications as normal performance measurement; the processor's thermal regulation may affect results, without tuning the test paramers, on some systems the bottleneck may not be the application, and the timing between starting the application and starting the load test will also affect results.
For more reliable, reproducible, results, these experiments should be done using the more advanced scripts, in a controlled environment.

### Acceptable: Run on a single machine, with solid automation and detailed output

These scripts are being developed.
Expand Down
8 changes: 8 additions & 0 deletions scripts/energy-stress-and-stop.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash
# This script is useful for measuring energy consumption, in combination with https://github.com/metacosm/power-server

thisdir=`dirname "$0"`

jbang wrk2@hyperfoil -t2 -c100 -d20s --rate 2000 --timeout 1s http://localhost:8080/fruits
Copy link

@franz1981 franz1981 Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this is ok with any laptop?

If the disk is slow it won't be able to sustain such load, so would be preferrable to make it parametric

Copy link
Collaborator Author

@holly-cummins holly-cummins Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm quite sure it's not ok with any laptop. :)

So yes, it could use the same parameterising logic as the other scripts. We should probably also add some instructions to say something like "do the throughput test and then set a load which is x% of the lowest max when you do the energy test."

Copy link
Collaborator Author

@holly-cummins holly-cummins Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I had load which was too low, the test was boring, because everything was almost the same. And you're right that load which is too high will be an incorrect test.

Copy link

@franz1981 franz1981 Nov 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, the "educational" part is related:

  1. if you want to measure energy consumption and compare 2 applications both should have the same capacity and the work performed should be the same e.g. there are many car analogies which would work the same for this
  2. the work performed should be "enough" to use the given capacity: an external bottleneck (e.g. database) could make the application to perform little work, meaning that its cost will be dominated by going idle/awaking - making more complex to draw conclusions on its efficiency

Both are concept which can be challenging to grok...
The latter, in particular, indeed many industrial benchmarks refer to this as modelling power efficiency at different "operational ranges".

Copy link
Collaborator

@edeandrea edeandrea Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to defer my review here for the others. They know more about this stuff than me.

kill $(lsof -t -i:8080) &>/dev/null
${thisdir}/infra.sh -d