Repository navigation
system, check: warn when a benchmark ran under a cgroup CPU quota - #277
Open
manunicholasjacob wants to merge 1 commit into
Open
manunicholasjacob wants to merge 1 commit into
manunicholasjacob wants to merge 1 commit into
Conversation
pyperf system show registers ten operations and pyperf check warns on four statistical conditions, but nothing looks at whether the run was throttled. A cgroup CPU quota (docker run --cpus, a Kubernetes CPU limit, systemd CPUQuota=) throttles the benchmark: it moves the mean and leaves the relative spread alone, so a capped run comes back looking statistically stable while being wrong, and the advice offered on an unstable result (more runs, system tune) does not help because the machine was not jittery, it was capped. Add a "CPU quota" operation next to "Power supply" that reports a binding quota and, from cgroup v2 cpu.stat, whether the benchmark cgroup was throttled. Reuse get_cgroup_cpu_quota() from the cpu_quota metadata (psfgh-265) and add get_cgroup_cpu_throttling() beside it. pyperf check emits the same warning from the cpu_quota metadata of a saved result, so an already stored result can be judged after the fact. Signed-off-by: Manu Nicholas Jacob <manunicholasjacob@gmail.com>
vstinner
reviewed
Nov 10, 2026
| return stat if found else None | ||
|
|
||
|
|
||
| def get_cgroup_cpu_throttling(): |
Member
There was a problem hiding this comment.
Please don't duplicate code, there is already get_cgroup_cpu_quota(). Why not reusing it?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
pyperf system showruns ten operations andpyperf checkwarns on four statistical conditions, but nothing in either looks at whether the benchmark was throttled while it ran.A cgroup CPU quota (
docker run --cpus, a Kubernetes CPU limit, systemdCPUQuota=) throttles the benchmark, and throttling does not look like noise. A quota that binds for the whole run slows every value by about the same factor, so the mean shifts but the relative spread does not, and the result comes back looking statistically stable while being wrong. The advicecheckoffers on an unstable result (more runs and values,pyperf system tune) does not help, because the machine was not jittery, it was capped. This is the common case on CI runners and in containers. Follows up #274 and reuses thecpu_quotadetection from #265.What this does
system show: a new "CPU quota" operation, next to "Power supply", that reports a binding quota (viaget_cgroup_cpu_quota()) and, from cgroup v2cpu.stat, whether the benchmark cgroup was throttled. It reports and advises only, the same shape as the power supply check, because nothing pyperf can tune fixes an externally imposed quota.check: warns when a saved result carriescpu_quotametadata, so a stored result is judged after the fact instead of reading as stable._collect_metadata.py: addsget_cgroup_cpu_throttling()besideget_cgroup_cpu_quota(). It walks the same cgroup hierarchy and readsnr_throttled/throttled_usec(cgroup v2) orthrottled_time(cgroup v1), taking the largest counters across the hierarchy because the throttling is recorded where the limit binds while the root stays at zero. It returnsNonewhen the counters cannot be read, so "could not be checked" stays distinct from zero.Scope
This is the quota-detection piece. The two design questions in #274 are left as follow-ups: capturing the per-run
cpu.statdelta around the benchmark (and whether to store throttle counters in metadata), and thermal throttling (core_throttle_counton x86,vcgencmd get_throttledon the Pi), where "could not be checked" has to stay distinct from zero.Tests
test_metadata.py: cgroup v2 throttled, zero-is-not-None, cgroup v1 nanosecond conversion, and no-cgroup returningNone.test_cli.py:checkwarns oncpu_quotametadata and does not warn without it.test_system.py: the CPU quota operation is silent with no quota, reports a set quota, and reports throttling.Checked on a Raspberry Pi 5
Under
systemd-run --user --scope -p CPUQuota=25%, released pyperf reports "The benchmark seems to be stable" on a run that was throttled for hundreds of milliseconds, andsystem showsays nothing about the quota. With this changecheckwarns that the benchmark ran under a CPU quota, andsystem showreports the quota and the throttling.