Skip to content

feat(benchmarks): phase 6C - publish allocator scaling curves #186

Description

@zackees

Parent phase: #183
Implementation guide: #177
Design authority: #176
Work package: 6C of 6
Blocker: #185 must be merged with handoff
Branch: feat/benchmark-phase-6c-scaling
PR count: exactly one

Context

A single physical-core point can hide serialization, false sharing, remote-free collapse, and oversubscription behavior. This issue adds a versioned scaling protocol with explicit CPU topology/affinity and fixed per-worker work.

Metric version and scenario subset

Add throughput-scaling-v1 for:

  • tiny-fixed-64;
  • small-log-mixed;
  • cross-thread-producer-consumer;
  • random-ownership;
  • thread-churn.

Thread points, deduplicated after topology resolution:

  1. 1;
  2. 2;
  3. 4;
  4. available physical cores;
  5. available logical CPUs;
  6. 2x available logical CPUs.

A point is omitted only when identical to an earlier numeric point; store aliases so the chart explains deduplication.

CPU topology contract

Resolve from the process's allowed CPU set/cpuset, not host-wide counts unavailable to the runner.

Record:

  • allowed CPU IDs;
  • package/core/thread sibling mapping;
  • physical-core count within allowed set;
  • logical count within allowed set;
  • affinity support and exact worker-to-CPU mapping;
  • runner/container CPU quota where observable;
  • NUMA nodes and worker placement;
  • governor/frequency/power metadata when observable.

Prefer one hardware thread per physical core before siblings. For oversubscription, round-robin workers over the allowed mapping.

If topology is ambiguous or requested affinity cannot be applied/verified, mark the scaling metric invalid on a stable host and informational-with-warning on hosted runners; do not silently label logical CPUs as physical cores.

Work normalization

Use fixed work per worker:

  1. calibrate one-thread upstream operations per worker to the protocol duration;
  2. freeze that per-worker count across all thread points and allocators;
  3. total operations = per-worker count x worker count;
  4. use identical per-worker deterministic streams by worker ID inside paired blocks;
  5. record requested/completed counts exactly.

Do not hold total operations constant across thread points; that would shorten high-thread samples and mix scaling with timer noise.

Keep allocator order paired/near-balanced independently for every scenario/thread point. Minimum 15 blocks per point.

Metrics

Per allocator/scenario/thread point:

  • aggregate throughput;
  • per-worker throughput;
  • absolute elapsed time;
  • speedup = throughput(N) / throughput(1);
  • parallel efficiency = speedup / N;
  • p50/IQR/min/max/noise across blocks;
  • voluntary/involuntary context switches where observable.

Relative allocator comparison remains paired against upstream at the same thread point.

Do not connect scaling lines across different topology fingerprints, affinity policies, per-worker operation counts, allocator pins, or metric versions.

Scaling chart

Replace pending scaling panel with one facet per scenario:

  • x-axis: numeric worker count with alias labels;
  • y-axis: speedup and/or aggregate throughput, clearly identified;
  • one line per allocator;
  • ideal linear reference line only on speedup plot;
  • absolute values available in table/tooltips/HTML;
  • physical/logical/oversubscribed markers;
  • runner/topology/affinity metadata;
  • informational label.

Never use one global y-scale when it hides scenario-specific collapse; use consistent semantics and clearly labeled facet scales.

RED -> GREEN tests

  • cpuset fixture excludes host CPUs outside allowed set;
  • SMT topology fixture selects physical cores before siblings;
  • dedup logic preserves aliases for 1/2/4/physical/logical;
  • failed affinity verification invalidates as specified;
  • per-worker count remains constant and total scales exactly;
  • same worker ID produces same stream across allocator/thread runs;
  • speedup/efficiency golden calculations;
  • zero/negative throughput rejection;
  • topology/affinity/operation-count mutation changes metric key;
  • chart cannot connect incompatible keys;
  • oversubscribed point is labeled, not treated as additional physical cores;
  • incomplete allocator/point/block matrix remains pending.

Workflow

Use a weekly/manual scaling workflow with:

  • <=60-minute hard timeout;
  • four allocators sequential/interleaved on one host;
  • five scenario subset, deduplicated thread points, >=15 blocks;
  • no parallel allocator processes;
  • raw topology/affinity/sample artifacts retained 30 days;
  • existing least-privilege sealed publication path;
  • default-branch-only writes.

If the full matrix exceeds 60 minutes, post measured breakdown and stop. Do not drop allocator/block validity. A reviewed suite-version change may reduce scenario subset only through #176.

Validation/evidence

Run fixture tests, repository gates, and one full default-branch scaling collection. Post run/runtime, resolved topology/aliases, affinity verification, raw artifact digest, metric key, panel URL, and branch/Pages digest equality.

Non-goals

  • NUMA cross-socket sweep beyond recorded default placement;
  • hardware counter attribution;
  • Windows/macOS scaling publication;
  • stable-host performance acceptance;
  • arbitrary Cartesian thread counts.

Acceptance criteria

  • Latency phase handoff is verified.
  • Topology uses allowed cpuset and affinity is recorded/verified.
  • Exact deduplicated thread points and aliases follow contract.
  • Fixed per-worker work and exact totals are proven.
  • Four allocators/five scenarios/all points have >=15 complete blocks.
  • Speedup/efficiency and paired effects pass golden tests.
  • Scaling chart labels physical/logical/oversubscribed points honestly.
  • Incompatible topology/key lineages never connect.
  • Live panel replaces pending with matching branch/Pages digest.
  • Workflow budget/repository gates/handoff pass.

Decisions

  • Fixed per-worker work measures aggregate scaling without shrinking samples.
  • Use allowed cpuset rather than host-wide CPU counts.
  • Deduplicate numeric thread points but retain semantic aliases.
  • Keep a focused five-scenario scaling suite rather than a Cartesian expansion.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions