test(hud): stop measuring the builder CPU jitter (ENG-11404)

[andrewgazelka]

Fixes the enforced smash-hud-e2e failure. ENG-11404.

What was failing

FAIL a new CPU reading that moves only the label costs one packet carrying only the label: ['add']

['add'] was the entire packet history for that bar: the Add, then twenty seconds of nothing. Nothing sent a wrong action; nothing was sent at all, and the message could not tell those two apart. The replacement distinguishes them in its own text: the same break now reads 0 of 0 retitle(s).

The mechanism, which is the inverse of the obvious one

The builder was not too busy. It was too quiet and too large.

The CPU label is a whole percent of ONE core (crates/hyperion/src/egress/server_load.rs, percent() rounds cores * 100.0 into a u32). The game server idles at about 0.01 cores and vc1-nix has 128 of them, so the label sat at CPU 1% of 12800% and consecutive one-second windows rounded to the same integer. operation() correctly sent nothing, and the check timed out waiting for a packet that was right not to exist:

33.59s [H1 ] <- boss bar add 'CPU 1% of 12800%' 0.000 full, blue
53.67s       TIMED OUT waiting for three CPU readings
53.67s FAIL a new CPU reading that moves only the label ... : ['add']
54.49s [H1 ] <- boss bar update_name 'CPU 2% of 12800%' 0.000 full, blue

It moved 0.8 s after the deadline. Hunting for contention would have found nothing and read as the hypothesis being wrong.

The defect was in the check, not the game.

There were two coin flips, not one

load_check's disconnect liveness claim waited on the same count("update_name") >= 2 and had been passing on the same jitter. Removing the first assertion shifted the phase and turned the second red on the very next run:

54.64s FAIL a viewer disconnecting with a bar on their screen leaves the server drawing everybody else's: 0 further packet(s)

Fixing only the reported one would have left a landmine that fires after the fix lands.

What replaces them

• The "an update carries only the update" claim now reads off the lobby countdown bar, whose label moves on the server's own tick clock (whole_seconds is a ceil) rather than the host's load.
• The liveness claim causes the change it observes: the surviving viewer toggles /serverload, and the Remove then Add have to come back.
• The CPU and memory packet counts are logged, not asserted, for the reason the memory bar already carried.
• egress::boss_bar gains a truth-table test over operation(), all 16 subsets of the four fields.

Why it should stay green: the margin, not the run count

Six green runs would say little. The assertion needs 3 label-only retitles out of the ~9 second-flips in a countdown, and measured 7, 7, 7, 7, 7, 8, 8 across seven runs. It is nowhere near its threshold, which is the thing that predicts tomorrow. Do not tighten the threshold without re-measuring that spread.

Residual: the countdown timer advances by wall-clock dt, so a server crawling below roughly 7 tps for a whole countdown would collide every second-flip with a fill step and turn them all into Adds. Several other assertions here fail first at that tick rate, but it is not zero.

Negative controls

Breaking operation()'s title-only branch to send a whole-bar Add turns it red for the right reason:

30.07s FAIL a second passing that moves only the countdown's label ... : 0 of 0 retitle(s) carried the title and nothing else
34.76s FAIL and a whole bar was only ever resent where two or more fields moved at once: 48 of 828 [('H3', '6879705f', ['title']), ...]

Two controls aimed at the disconnect assertion both failed to reach it. The first broke /serverload's re-add and killed the gate at the first /serverload, before the assertion ran. The second used a realistic bug shape (add_if_new matching (Sent, Wildcard) instead of (Sent, viewer)); two neighbouring checks went red and this one passed, because the admin already held the bars.

So that assertion is deterministic and green but its incremental value over the three earlier toggle checks is unproven, and it is labelled STATUS: UNPROVEN in the source next to itself, with both failed attempts written out and a description of the defect that would actually distinguish it (one needing a stale (Sent, dead_viewer) pair, since this is the only bar check that runs after a socket has died). Anyone later deciding whether to delete it as redundant gets the evidence rather than a guess.

Honest tally: 6 green runs of the fixed gate, 1 red baseline, 3 induced reds from deliberate breaks, and 1 control that would not fire.

Not redundant with diff_check, and this was measured

Making operation() return None for a title-only change freezes every countdown on every screen:

30.36s FAIL a second passing that moves only the countdown's label ... : 0 of 0 retitle(s)
35.00s PASS 780 boss bar packet(s) went out this run ...
35.00s PASS no boss bar packet said anything the client already knew: 0 of 780 []
35.00s PASS and a whole bar was only ever resent where two or more fields moved at once: 0 of 780 []
35.00s PASS and nothing arrived for a bar the client had never been given: 0 of 780 []

diff_check passes all four rules; this check is the only one in the suite that goes red. Its rules are all "no packet did X", so they can only judge packets that exist, and the volume guard does not help because progress updates dominate the count. The two claims are opposites and each is blind exactly where the other looks. That reasoning is written next to the assertion so it is not resolved the wrong way later.

bedwars / shared engine

boss_bar.rs is engine code every mode shares, so worth stating: a bedwars client sees nothing different, because no runtime code changed. The whole Rust diff is one insertion hunk, @@ -490,0 +491,83 @@ mod tests, and #[cfg(test)] starts at line 410. operation() is untouched.

Who reads the path: crates/hyperion/src/egress/mod.rs:50-51 imports BossBarModule and ServerLoadModule unconditionally, so every event including bedwars gets the drive system and the /serverload bars (an engine command, crates/hyperion-clap/src/lib.rs:621). events/smash/src/adapter.rs:520-578 is the only game-mode consumer. events/bedwars/src/ has no boss bar reference at all.

Caveat: because bedwars draws no bars of its own, no bedwars gate exercises this path, and the new wire-level coverage is smash-only. The truth-table unit test is the mode-independent part.

Verification

• nix build .#checks.x86_64-linux.smash-hud-e2e -L on vc1-nix: 50 PASS, 0 FAIL, rc=0. Seven green runs total against a 1-for-1 red baseline.
• cargo test -p hyperion --lib egress::boss_bar: 5 passed. Watched the new test fail under the induced break (left: "add" right: "update_name").
• cargo fmt --check -p hyperion and cargo clippy -p hyperion --lib --tests: clean.
• Gate is also 20 s shorter (56.5 s to 35.0 s of client time).

Unrelated, filed separately

• ENG-11405: checks.*.{fmt,lint,test,ci,deny} are writeShellApplication derivations that build the script rather than run it, so nix flake check cannot fail on them. This briefly made me report a clean lint that was not.
• Pre-existing clippy errors at HEAD in events/smash/src/module/visuals.rs:418 and :525, owned by another change in flight.

[github-actions]

Benchmark Results for general

ray_intersection/aabb_size_0.1                     [  18.8 ns ...  18.7 ns ]      -0.20%
ray_intersection/aabb_size_1                       [  18.7 ns ...  18.7 ns ]      -0.08%
ray_intersection/aabb_size_10                      [  18.6 ns ...  18.6 ns ]      -0.04%
ray_intersection/ray_distance_1                    [   1.3 ns ...   1.3 ns ]      -0.27%
ray_intersection/ray_distance_5                    [   1.3 ns ...   1.3 ns ]      +0.14%
ray_intersection/ray_distance_20                   [   1.3 ns ...   1.3 ns ]      -0.59%
overlap/no_overlap                                 [  16.1 ns ...  15.9 ns ]      -1.22%*
overlap/partial_overlap                            [  16.1 ns ...  15.9 ns ]      -1.08%*
overlap/full_containment                           [  14.4 ns ...  14.5 ns ]      +0.79%*
point_containment/inside                           [   5.4 ns ...   5.4 ns ]      +0.25%
point_containment/outside                          [   5.7 ns ...   5.7 ns ]      +0.40%
point_containment/boundary                         [   5.7 ns ...   5.7 ns ]      +0.43%

Comparing to ec8c9fd