Skip to content

fix(benchmarks): finish Phase 5B README graphics and seal benchmark-stats #200

Description

@zackees

Context

Phase 5 is operational but not finished cleanly enough to merge the README graphics work.

Verified on 2026-08-11:

Item Current state
Scheduled producer Run #31481003209 succeeded from main at d9307184
Generated branch benchmark-stats exists at 89b5ec4f
Pages Dashboard is built and its deployment run #31483479971 succeeded
Real graphics benchmark-throughput.png and benchmark-history.png return HTTP 200 from both the raw branch and Pages; the byte sizes agree (7,211 and 4,565 bytes)
Advanced graphics memory, latency, scaling, and pprof-tax PNGs exist only as explicit pending placeholders
Default README Links the dashboard/branch/data and shows the development smoke SVG, but does not embed the two live headline graphics
README PR #199 is open but CONFLICTING, one commit ahead and two commits behind main, with no check rollup

The generated branch is not sealed correctly. Its manifest says .nojekyll is part of the site, but .nojekyll is absent from the branch. Conversely, repository files and directories such as .clud/, .github/, .gitattributes, and .gitignore remain in the branch even though ci/benchmark_report.py::SITE_FILES is intended to be the exact publication allowlist.

PR #199 identifies the dotfile transfer problem, but its proposed shopt -s dotglob followed by rm -rf ./* can also match and delete the linked worktree's .git file. Its explicit upload globs are also not the supported control for hidden artifacts: actions/upload-artifact excludes hidden files by default and documents include-hidden-files: true as the opt-in: https://github.com/actions/upload-artifact#uploading-hidden-files

This issue supersedes the now-stale snapshot in #194 and records the recovery path from the live state.

Proposal

1. Repair the Phase 5B PR in place

Update PR #199 / branch docs/benchmark-phase-5-readme on top of current main rather than creating a second Phase 5B PR.

2. Make publication an exact, fail-closed tree replacement

Replace the generated worktree contents without a dotglob deletion that can remove .git.

A safe shape is:

  1. create the detached linked worktree;
  2. remove tracked contents with a Git-aware operation such as git rm -r -f . (or an equivalently tested approach that preserves the worktree administrative file);
  3. copy the complete validated site with cp -a "$GITHUB_WORKSPACE/site-temp/." . or equivalent so .nojekyll is included;
  4. git add -A;
  5. before commit and before push, assert that the staged file set is exactly SITE_FILES;
  6. after push, query the remote branch tree and repeat the exact allowlist/digest audit.

For the cross-job site artifact, set include-hidden-files: true on actions/upload-artifact after validation has proven that the site contains only the allowlisted files. Keep if-no-files-found: error.

Extend the final publication audit so a successful run is impossible when:

  • .nojekyll is absent;
  • any source/config directory such as .github/ or .clud/ survives;
  • any unexpected file exists;
  • the branch tree differs from the downloaded sealed site;
  • branch and Pages payload/manifest digests differ.

3. Use a hybrid README link strategy

Use the canonical generated branch as each Markdown image source, but make a click open the user-facing Pages dashboard:

[![Throughput benchmark chart](https://raw.githubusercontent.com/zackees/mimalloc-pprof/benchmark-stats/benchmark-throughput.png)](https://zackees.github.io/mimalloc-pprof/#throughput)

Do the same for history. Add stable id="throughput" and id="history" anchors in generated HTML, or link to the dashboard root until those anchors ship.

Keep explicit text links to the benchmark-stats branch, latest.json, and the dashboard near the images. This gives README rendering a direct canonical artifact, sends normal clicks to the readable report with tables and methodology, and keeps the branch available for provenance/audit rather than making it the primary user interface.

4. Verify live state, then close the right trackers

Immediate Phase 5B closeout:

  1. merge corrected PR fix(benchmarks): seal stats publication and surface headline graphics #199;
  2. run one eligible default-branch publication;
  3. verify the remote tree allowlist, .nojekyll, image rendering, manifest/digest equality, and Pages links;
  4. post the run URL, branch SHA, manifest digest, and checks on this issue;
  5. close this issue and close handoff: complete benchmark dashboard publication pipeline and close #177 #194 as superseded/completed.

Advanced benchmark images are a separate serial program:

#184 memory/RSS
  -> #185 transaction latency
    -> #186 scaling
      -> #187 pprof tax
        -> close #183

Those issues are not blockers for publishing the real throughput/history images. They are blockers for replacing and embedding their corresponding pending panels as real results. After #183 is complete and the final acceptance audit passes, close #177 and then #176. The broader optimization program #170 can remain independent.

Do not reopen already-closed #182 solely to track these corrections; this issue is the explicit post-closure Phase 5B recovery record.

Acceptance criteria

  • RED evidence reproduces both current defects: hidden .nojekyll is lost across the site artifact boundary, and repository dot-directories survive the generated-branch replacement.
  • GREEN tests prove hidden-file upload is explicit, the worktree administrative .git file survives cleanup, .nojekyll survives transfer, and the staged/remote branch tree equals SITE_FILES exactly.
  • PR fix(benchmarks): seal stats publication and surface headline graphics #199 is based on current main, conflict-free, has a valid conventional commit, and passes the required c-unit platform matrix (including MI_PPROF=OFF), rust-native, and applicable Python/workflow gates.
  • The corrected publication workflow is merged before the verification run.
  • A new eligible default-branch run succeeds through measurement, artifact audit, branch publication, Pages deployment, and publication audit.
  • benchmark-stats contains only the allowed generated site files, including .nojekyll; it contains no .clud/, .github/, source tree, or unrelated dotfile.
  • Branch and Pages expose digest-identical current payloads; the run URL, branch SHA, and manifest digest are posted here.
  • README.md embeds the live throughput and history graphics, with accessible alt text; image clicks open the relevant Pages report/anchor.
  • README keeps visible direct links to the dashboard, benchmark-stats, and latest.json.
  • The development smoke SVG remains explicitly labeled non-headline.
  • Memory, latency, scaling, and pprof-tax placeholders remain clearly pending and are not embedded as measured results before feat(benchmarks): phase 6A - measure Linux RSS and fragmentation honestly #184-feat(benchmarks): phase 6D - quantify pprof compilation and runtime tax #187 close in order.
  • Live HTTP checks for the README image sources, click targets, branch, and latest.json return successful responses after merge.
  • Evidence is posted before this issue and handoff: complete benchmark dashboard publication pipeline and close #177 #194 are closed.

Decisions

Related issues

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions