Skip to content

The record is usually the only place the rejected alternative is written down, so Stage 0 holds - #829

Merged
MongLong0214 merged 8 commits into
mainfrom
cdeb-v4-qualification
Aug 22, 2026
Merged

The record is usually the only place the rejected alternative is written down, so Stage 0 holds#829
MongLong0214 merged 8 commits into
mainfrom
cdeb-v4-qualification

Conversation

@MongLong0214

Copy link
Copy Markdown
Owner

Stacked on #828. Completes CDEB-Fresh v4 Stage 0.

Six candidates qualified out of 241. The threshold — 48 across at least three repositories with twelve each — was registered before the census ran and taken unchanged from the owner's PRD. It is not met, and nothing here moves it.

repository raw provenance hidden viable oracle delivery bounded qualified
agent-control-plane 35 3 17 27 30 28 33 1
agent-operator-score 59 2 31 35 56 41 58 1
gitseed 104 4 32 62 56 42 71 2
logic-pro-mcp 43 8 22 19 41 43 43 2

Every gate except provenance passes in the hundreds. Provenance passes 17 of 241, and 190 candidates fail with insufficient-provenance.

The blocker is the corpus, not the instrument

Two blind reviewers, given only the commit prose with every record removed, were asked to quote any rejected alternative and the reason it was rejected. They found a rejection for 169 of 207 candidates. For most it was not the candidate's own — a different ruling from the same commit, or nothing corresponding at all. The Ruled-out: trailer is, for most of these decisions, the only place the specific rejection exists.

That matters because gold copied from the record makes the benchmark measure its own instrument. If the rejection is not independently written down, gold cannot be written.

A preregistered robustness arm closes the obvious alternative explanation: 60 candidates, 15 per repository, reviewed again with the commit's full diff as well as its message — 13% versus 10% on the same candidates. The packet is not too narrow.

Reviewer machinery

Both reviewers are independent sessions of one model family (a second family refused with HTTP 402; the owner directed the substitution), so agreement bounds reliability from above. Where the pair split, a third blind vote decides by majority rather than an adjudicator who already knows how the pair voted.

Agreement by gate: G2 0.91, G3 0.74, G4 0.91, G5 0.93, G7 0.99. When both reviewers found a rejection they quoted near-identical text 45% of the time — that is how correlated two models of one family actually are, and it is published rather than left implicit.

Guard coverage: 13 bound → 23

The five new properties are the ones whose absence would have been invisible: a terminal study resolving as active, a packet still carrying a record line, a harness failure published as zero delivery, a reviewer answering three of ten, and a split pair rounding into a pass.

Verification

56 v4 tests pass, both typechecks clean, guard ratchet exits 0 at 23 bound with no failures. Full suite 3561 passed with 4 cdeb-smoke timeouts under 14 concurrent reviewer sessions; that file passes 6/6 alone.

RESULT.md is generated from the artifacts and its --check is a test, so the prose cannot drift from the counts.

Not done

No pilot, no measured run, no randomization, no README headline, no synthetic identity. measured_run_allowed is still false and there are still zero measured product-effect rows.

…e decision was recoverable

So the reviewer's evidence is the commit's prose with every CommitLore trailer and note
removed -- what the record was written from, not the record. If the ruling can only be
recovered by reading the record, the candidate has no independent provenance and this
study should not use it.

The mechanical half of that runs here. Of 241 enumerated decisions, 207 have surviving
ordinary prose and a resolvable scope:

    agent-control-plane   35 audited   35 pass
    agent-operator-score  59           58 pass    1 no resolvable scope
    gitseed              104           71 pass   33 subject-and-record only
    logic-pro-mcp         43           43 pass

Gitseed's 33 are commits whose whole message is a subject line and a record. There is
nothing to review behind them, which is a fact about the corpus rather than a fault in the
candidates.

Two guards exist because their absence would be invisible. `assertPacketsCarryNoAnchor`
refuses a packet containing the benchmark's own key, since a reviewer who can see the
anchor can tell which decisions are being watched. `assertRedactionDidWork` refuses an
audit in which no record-backed candidate had a trailer removed -- a redaction that
removed nothing produces a clean leak report for every candidate, and the report reads
identically whether the redaction worked or never ran.

Packets are built once per repository rather than once per candidate. The existing
per-candidate builder re-materializes the bundle each call, and logic-pro-mcp's is 87MB.

Record-Id: r-v4provenance
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: using the rendered record as gold | it proves the record is readable, which is the thing under test, and any effect measured that way is the instrument measuring itself
Ruled-out: judging independent recoverability here | that is a reading of the prose, and a program that scored it would be encoding one reviewer's judgment as a mechanical fact
Ruled-out: calling buildSourcePacket per candidate | it re-materializes the sealed bundle each time, so 241 candidates would clone 87MB once per logic-pro-mcp decision
Limit: `Provenance: reconstructed` and `migrated` are the only benchmark-authored signals available mechanically. A record written during benchmark work but labelled `authored` is indistinguishable from an ordinary one here, and only the reviewers can catch it
Verified: 5 tests pass, both typechecks clean. Each guard is proved by arrival: the anchor check passes on real packets and throws once an anchor is appended to one, and the inertness check passes on a real audit and throws when the removal count is zeroed
CommitLore-Version: 2.0.0
…nd reviewer would have been handed the answer

The leak check on the Stage A packets fired before a single reviewer ran. Two candidates'
"redacted" ordinary source contained a complete CommitLore record -- `Ruled-out:` line,
reason, `Record-Id:` and all -- and eleven carried at least one record line.

The cause is not a bug in the product's redaction. That redaction rebuilds the ordinary
trailer tail from Git's own parse, which deliberately refuses to treat a `Ruled-out:`
sentence in prose as a record. A squashed commit concatenates whole commit messages
indented by four spaces, and Git does not see those as trailers either. Correct for the
product, wrong for a blind evidence packet.

So a second pass removes any CommitLore key line wherever it appears, with its folded
continuations, and a separate check refuses a packet that still holds one. The bias is
deliberately reversed here: removing a prose sentence that merely looks like a trailer
costs a sentence, and leaving a record in costs the answer to the question being asked.

83 lines came out of 11 packets. Every packet is now clean by the check that found the
problem.

One candidate's prose still contains its ruling verbatim -- logic-pro-mcp's "one pattern
for both languages" is a noun phrase the body uses naturally. That is a G2 pass, not a
leak, and it is reported rather than stripped.

Record-Id: r-v4redactionsecondpass
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: changing the product's redaction to match | it is right for the product, where a prose sentence that looks like a trailer must not be treated as a record, and the benchmark's opposite bias does not belong in shipped behaviour
Ruled-out: excluding the eleven affected candidates | the contamination is in the packet, not in the decision, and dropping them would lose real corpus to a fixable tooling gap
Ruled-out: stripping the one prose echo of a ruling | the body states the rejected approach in its own words, which is the independent support G2 is asking about
Limit: the key list is the vocabulary this repository uses. A record written under an extension key would survive both passes, and only the packet check would catch it -- by name, which is the same list
Verified: 8 tests pass, both typechecks clean. The finding arrived before the fix: the packet builder refused to emit Stage A batch 01, naming the candidate whose ruling was visible. After the second pass, zero packets contain a CommitLore key line, and the fixture reproducing the squashed-commit shape fails without it
CommitLore-Version: 2.0.0
…le reason this study exists

The predecessor's delivery gate asked whether an expected `Record-Id` appeared in the bytes
the hook forwarded. Under the owner's estimand that is the wrong question, and it is the
question that emptied two of four repositories before any measurement.

So the probe drives the same surface -- `commitlore inject --hook-input` against a real
`PreToolUse` edit payload, at the frozen v1.2.0 dist, at the shipping default budget of 800
tokens -- and reads the forwarded bytes for content: the ruling, the reason, the path
scope, the lifecycle. Identity is recorded beside the result and never conditions it.

Across the 207 candidates with independent ordinary source:

    delivered                    154
      carrying a Record-Id        69
      carrying none               85
    ruling visible               157
    reason visible               157
    scope correct                157
    stale delivered as current     0

85 id-less decisions delivered is the finding. `logic-pro-mcp` has no `Record-Id` anywhere
in its history and its decisions still reach an agent about to edit the paths they cover.

Two things about the numbers are worth stating. The scope result is not vacuous: 71 of the
out-of-scope probes returned a non-empty payload, so the injector does forward other
records for an unrelated path -- it just never forwarded *this* candidate's ruling there.
And the first run of this probe reported 0 delivered of 207, which read exactly like a
finding. The extracted release tree had no installed dependencies and the CLI exited 1
before reading a record. Every content field is false in both cases, so the distinction is
now made by a check rather than by whoever reads the summary.

Record-Id: r-v4delivery
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: keeping the Record-Id visibility gate | it measures the instrument's own restriction, and under it 43 of logic-pro-mcp's 43 decisions fail for a field that did not exist when they were written
Ruled-out: a single substring match as the gate | a record sharing one phrase would pass, so the gate needs ruling, reason, scope and lifecycle together, with the out-of-scope probe as the part that can actually fail
Ruled-out: running the working-tree dist | it has drifted from v1.2.0, so it would measure behaviour nobody shipped
Limit: this shows the content can reach an agent that edits one of the decision's own paths at the shipping budget. It says nothing about whether an agent will edit those paths, use a tool the matcher covers, or act on what it reads -- those are the questions a measured run would ask
Verified: 6 tests pass including an end-to-end probe in which a record with no `Record-Id` at all has its ruling and reason delivered through the shipping hook. All 207 real probes exit 0. The injector guard is proved by arrival: it throws on the exact stderr the broken first run produced
CommitLore-Version: 2.0.0
…ten down, so Stage 0 holds

Six candidates qualified out of 241. The threshold was 48 across at least three
repositories with twelve each, registered before the census ran and taken unchanged from
the owner's PRD. It is not met, and nothing here moves it.

    repository            raw  provenance  hidden  viable  oracle  delivery  bounded  qualified
    agent-control-plane    35           3      17      27      30        28       33          1
    agent-operator-score   59           2      31      35      56        41       58          1
    gitseed               104           4      32      62      56        42       71          2
    logic-pro-mcp          43           8      22      19      41        43       43          2

Every gate except provenance passes in the hundreds. Provenance passes 17 of 241, and 190
candidates fail with `insufficient-provenance`.

The blocker is the corpus, not the instrument. Two blind reviewers, given only the commit
prose with every record removed, were asked to quote any rejected alternative and the
reason it was rejected. They found *a* rejection for 169 of 207 candidates. For 190 of
those it was not the candidate's own -- a different ruling from the same commit, or
nothing corresponding at all. The `Ruled-out:` trailer is, for most of these decisions,
the only place the specific rejection exists.

That matters because gold copied from the record makes the benchmark measure its own
instrument. If the rejection is not independently written down, gold cannot be written.

A preregistered robustness arm closes the obvious alternative explanation. Sixty
candidates, fifteen per repository, reviewed again with the commit's full diff as well as
its message: 13% versus 10% on the same candidates. The packet is not too narrow. The
rejection is not written outside the record.

Reviewer machinery is worth stating. Both reviewers are independent sessions of one model
family -- a second family refused with HTTP 402 and the owner directed the substitution --
so the agreement figures bound reliability from above. Where the pair split, a third blind
vote decides by majority rather than an adjudicator who already knows how the pair voted.
Agreement by gate: G2 0.91, G3 0.74, G4 0.91, G5 0.93, G7 0.99. When both reviewers found
a rejection they quoted near-identical text 45% of the time, which is how correlated two
models of one family actually are.

Guard coverage moves from 13 bound to 23. The five new properties are the ones whose
absence would have been invisible: a terminal study resolving as active, a packet still
carrying a record line, a harness failure published as zero delivery, a reviewer answering
three of ten, and a split pair rounding into a pass.

Record-Id: r-v4qualification
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: adjudicating the 92 split gates myself | the study operator reading their own corpus, already knowing how the pair voted, is the least blind reader available; a third blind vote costs one more session and is a vote rather than an override
Ruled-out: averaging or passing an unresolved disagreement | it would put a candidate in the corpus that no two reviewers agreed on, and the disagreement would stop being visible
Ruled-out: relaxing the quote-correspondence floor after seeing 8% | the floor was fixed in code and in the deviation record before any overlap was computed, and moving it now would let the count choose the method
Ruled-out: accepting any rejection found in the same commit | it qualifies candidate X on evidence about decision Y, which is how a corpus fills up without meaning anything
Limit: G3 and G4 were judged from the commit message, the changed paths and the ruling. Neither reviewer read the current code or ran a test, so both are informed judgements about a maintenance task rather than measurements of one. G5 classifies whether an oracle could be written; none was built
Limit: this says nothing about whether recording decisions helps an agent. It says the four surveyed repositories cannot supply gold that is independent of the records being tested, which is a fact about these repositories and this gate
Verified: 56 v4 tests pass, both typechecks clean, guard ratchet exits 0 at 23 bound with no failures. Full suite 3561 passed with 4 cdeb-smoke timeouts under 14 concurrent reviewer sessions; that file passes 6/6 alone. RESULT.md is generated from the artifacts and its --check is a test, so the prose cannot drift from the counts
CommitLore-Version: 2.0.0
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown

CommitLore — record lint

Trailers: clean — 8 commits in origin/main..a74aaa58ad35b3551b77a81ff6a8823ef11eb641
Active constraints: 136 limits · 168 ruled-out · 17 warnings — from 160 records over 26 changed paths

Active constraints for the paths this PR touches

Limits (136)

  • r-v4claimnarrowed a74aaa5 — G2 as implemented bounds the qualified count from above for the whole gate, so the HOLD stands. It does not answer whether complete independent gold could be written, because scope and lifecycle recovery were never tested
  • r-v4provtiers e9b167a — this reports the tier split. It takes no position on whether P2 evidence should ever be admitted, which is not Stage 0's call
  • r-v4qualification b8ff1b9 — G3 and G4 were judged from the commit message, the changed paths and the ruling. Neither reviewer read the current code or ran a test, so both are informed judgements about a maintenance task rather than measurements of one. G5 classifies whether an oracle could be written; none was built
  • r-v4qualification b8ff1b9 — this says nothing about whether recording decisions helps an agent. It says the four surveyed repositories cannot supply gold that is independent of the records being tested, which is a fact about these repositories and this gate
  • r-v4delivery d2ff392 — this shows the content can reach an agent that edits one of the decision's own paths at the shipping budget. It says nothing about whether an agent will edit those paths, use a tool the matcher covers, or act on what it reads -- those are the questions a measured run would ask
  • r-v4redactionsecondpass 9222978 — the key list is the vocabulary this repository uses. A record written under an extension key would survive both passes, and only the packet check would catch it -- by name, which is the same list
  • r-v4provenance 62f0ac4Provenance: reconstructed and migrated are the only benchmark-authored signals available mechanically. A record written during benchmark work but labelled authored is indistinguishable from an ordinary one here, and only the reviewers can catch it
  • r-v4stage0id 6b427af — this proves a terminal study cannot be resolved as active through the declaration. It does not stop a caller that names a study root directly, which is why the measured-run gate is separate and still shut
  • r-v3manifestsync c195b48 — this proves the manifest matches this checkout. It does not prove the pinned image produced the committed dist on any other machine, which is what the pinned digest is for
  • r-v3terminalseal 7754f1a — the placeholder row remains in the ledger and always will. This makes it legible, not absent, and a reader who takes digests on faith rather than reading the deviation is still misled
  • r-v3terminalseal 7754f1a — the canonical digest binds the artifact list it is given. A transition that names too few artifacts is bound to a partial set, and nothing here decides what the right set is for a future study
  • r-v3terminalseal 7754f1a — guard coverage is unchanged -- thirteen exclusion kinds remain uncovered and one scan inert, recorded in the mutation baseline
  • r-guardratchet 26b1989 — the baseline is a floor. A guard can bind its claim against the one mutation recorded for it and still miss a different violation of the same claim
  • r-guardratchet 26b1989 — the reasons are written by the same author as the claims, so a gap reasoned narrowly can look more settled than it is
  • r-guardratchet 26b1989 — thirteen exclusion kinds remain uncovered and one scan inert; this records them and repairs neither
  • r-guardmutation 551921d — a mutation proves a guard reacts to one specific edit. A guard can bind its claim for that edit and miss a different violation of the same claim, so coverage here is a floor and not a proof
  • r-guardmutation 551921d — the claims were written by the same author as the guards, so a claim stated too narrowly produces a control that passes for a property nobody wanted
  • r-guardmutation 551921d — thirteen exclusion-index kinds remain inert; this change makes that visible and does not repair it
  • r-coauthoredcasing 06354f9 — the exemption stays a fixed pair. A standardised trailer this repository has not met -- Reviewed-by:, say -- is still refused as unknown-key, which is deliberate but will read as the same bug to whoever hits it next
  • r-democopycanonicalbuild 8ce56bc — the bundle changed only because one output string did; nothing about the build is different
  • r-release120 b073960 — the passive notice only speaks once a check has landed, so the first invocation after this install says nothing however out of date the next release finds it. That is the trade the zero-latency design buys, and the answer arrives on the following command
  • r-lazysignaturemode f7dc7ef — a scan that reads even one commit still asks, so the cost returns on any invocation that has catching up to do. That is the invocation that can afford it
  • r-rebuildworknobodyreads 1a66b26 — the deadline is still only checked between batches and before the expensive half of one, so a late batch of 1024 commits can overshoot by whatever that batch costs -- bounded by one batch rather than by the whole scan. And an unsigned-mode index now carries '' where it carried git's verdict, so a reader wanting the cached status without turning signature mode on no longer gets it; none exists
  • - 1ca0ac9 — the other tracked dist files auto-merged rather than conflicting, and a line-wise merge of generated JavaScript is not something to trust on its own -- the canonical rebuild is what makes them correct, not git's resolution
  • r-rebuildopensdamaged 70dc155 — this covers a full-text table that will not rebuild. Damage that makes createSchema or the first meta read fail still surfaces through the open's own catch, which is where it belonged already; nothing here widens that
  • r-initsayswhatitpinned c7de40d — the report names the pinned version and the newer one, but not whether the pinned checkout is intact -- doctor compares the running build against the pinned one and is where that question belongs
  • r-upgradeperforms b1e75c9 — nothing here can tell a current that resolves to the right tag over a checkout whose contents are wrong. install.sh verifies a reused checkout's manifest and tag, and doctor compares the running build against the pinned one; step 4's failure text names doctor for exactly that reason
  • r-doctorreleasefreshness 174e120latestReleaseSync cannot signal a process group, so a git that spawns an SSH client which then hangs is bounded by spawnSync's timeout on the child alone. The async path exists because the notice cannot afford that and this report can
  • r-passiveupdatenotice a783a95 — the notice speaks only when a check has already landed, so on a cold cache the first invocation says nothing however out of date it is. That is the trade the zero-latency property buys, and the answer arrives on the next command rather than the first
  • r-upgradereadonly ccc634cupgrade accepts --check but performs no upgrade in this build, and --check is therefore the only behaviour. T-1606 makes the bare form act; until then the command names the install line rather than running it
  • r-integrityoffread 2584678commitlore index --rebuild still cannot open a structurally damaged index -- openIndex rebuilds the FTS table on open and throws first (commitlore index --rebuild cannot open the index it is meant to rebuild #785). That predates this change and the documented remedy has never run in that state; test/index-corruption.test.ts opens the database directly to work around it and says so
  • r-pluginawaredelivery 46c4169 — this does not clean up dual installs already on disk -- somebody in that state keeps paying twice until they remove one by hand. It also reads Claude Code's private state, which has already changed shape once (the registry is on version: 2); when that breaks, it breaks toward writing the hook, which is the direction chosen on purpose
  • r-hookmatcherunify fa4373d — neither installer knows the other exists -- init writes the settings.json hook unconditionally, so a user who follows the README to the plugin and then runs commitlore init carries two PreToolUse hooks running the same command. Unifying does not create that double fire, but it widens the overlap from Edit and Write to all five; the partial overlap it replaces was worse to diagnose, because the same user saw records twice on an edit and once on a read
  • r-rebuildschema a6d577e — this recreates the file whenever the recorded version differs, so a downgrade discards an index a newer build wrote rather than reading what it can from it
  • r-partialsilence 9553e2c — this says the scan was cut short, not which records were missed -- the payload cannot name what it never read
  • r-rel114 9692b6d — the README restructure and the mobile hero redesign are not in this release, so the four READMEs remain long and the hero's labels remain small at 375px
  • r-builderpin cb1515f — nothing checks that the pinned digest still exists upstream, so a digest deleted from the registry surfaces as a build failure rather than as a clear message
  • r-canonmerge761 6a88f2f — this proves the bundle matches this tree; whether this tree is what a reviewer wants is what the pull request is for
  • r-rel113 17a1301#749 question 1 stays open -- a fix that lives in the hook reaches a repository only on its next visit, and nothing on this machine knows which repositories exist
  • r-machinescope e46af2a — this is one check's classification, not a scope field -- another machine-scoped check added later will default to claiming attention again, and nothing here would notice
  • r-upgraderebind 49765af — this reaches a repository only when its stub already carries the arm, so one installed before this still needs hooks install once -- the same boundary After an upgrade the hook says "cannot find the CLI" when the CLI is fine and containment refused it #746 recorded, and the reason that command stays the named remedy
  • r-746message 5dda01b — the stub is written into .git/hooks at install time, so a repository wired before this keeps the old text and gets the old sentence until hooks install runs there; installing a corrected release is not enough
  • r-746narrow 5dda01bcommitlore.node is only tested for -x, so a recorded interpreter that exists but cannot run this bundle still reaches neither arm and falls through to the absence message
  • r-rel112 ad6fee3 — the readback confirms the link, not that the interpreter behind it runs -- doctor remains the check for that
  • r-rel112 ad6fee3 — this repairs the installer; a machine already upgraded to 1.1.0 or 1.1.1 keeps its stale current until the installer is re-run, which is why the note names the command to check
  • r-detectrule728 f1784ce — this records the rule, it does not enforce it -- a new host added with the wrong test still compiles, and Plant a host fixture in Windows CI so this class can't hide behind an empty runner #722's planted fixture is where that could be caught
  • r-detectwhy728 f1784ce — still recorded, not enforced -- a new host added with the wrong test compiles
  • r-rel111 8c29f5d — Hermes still fails on that machine for a cause that is not this one and is not yet named (Windows: every detected host fails to wire — the temp filename carries the whole path, and hasCommand cannot see a .cmd #716)
  • r-rel111 8c29f5d — a zero-byte .cursor/mcp.json on the tester's machine is a user file; the installer read the file it says it reads and reported the true reason
  • r-rel111why 8c29f5d — this changes the note, not the behaviour -- the behaviour shipped in the merged branch and is already covered by artifact:verify
  • r-canon720 7bf5b82 — this adds no judgement about the Windows behaviour -- it makes the branch buildable, and the live evidence on the pull request is still the only evidence for that path
  • r-canon720b 7bf5b82 — this rebuilds, it does not judge -- the Windows behaviour still rests on the live evidence recorded on the pull request, and Hermes and the first-run probe timeout are still open on Windows: every detected host fails to wire — the temp filename carries the whole path, and hasCommand cannot see a .cmd #716
  • r-winargv716 7bf5b82 — Does not change executable-only Claude detection or the pre-existing MCP probe shell path; a new canonical dist rebuild is not included from Windows
  • r-winexec716 7bf5b82 — Does not change MCP probing or rebuild generated artifacts.
  • r-rel110 d9a041f — this release does not make host wiring work on Windows -- detection still cannot see a .cmd and spawn still cannot run one (Windows: every detected host fails to wire — the temp filename carries the whole path, and hasCommand cannot see a .cmd #716)
  • r-rel110 d9a041f — 1.0.0 through 1.0.2 have no CHANGELOG entries; a pointer to the releases page stands in rather than reconstructing them
  • r-rellock110 d9a041f — nineteen version surfaces was already wrong before this -- the lockfile makes it twenty-one, and the count is only ever known after the gate says so
  • r-relmanifest110 d9a041f — this is the release commit's own repair, not a fix -- the next release will need the same regeneration for the same reason
  • r-namefile716 f728e69 — only failures name their file; a successful wire still reports no path
  • r-namefile716 f728e69 — this makes the cursor question answerable on the next Windows run -- it does not answer it, and the outcome (failed) was never in doubt
  • r-overlay709 7e08cbf — unattended is an input to the effective digest but not to the defaults digest -- M-UX: capture leaves the user's workflow #511's exclusion rests on a file's identity being its own bytes, which an overlay breaks
  • r-overlay709 7e08cbf — a broken overlay falls back to the built-in defaults, not to the committed file -- layering onto a policy nobody could read states an effective policy no file states
  • r-wintmp716 f0ed513 — this fixes the write, not the detection -- codex, hermes and claude-code failed for the second cause and still will
  • r-wintmp716 f0ed513 — no Windows machine has run this fix; the guard proves the name, not the install
  • r-pend710 b452535 — the wording is checked, not the severity — an abandoned draft still warns
  • r-rel102 25c11ed — an installer-boundary fix reaches nobody until it is released
  • r-codexok 9021dd9 — a requested integration that failed is not a healthy host
  • r-697codx ae6245f — one owner per host step, in the place both installers call
  • r-693curr 14909c3 — a hook records a path that does not name a release
  • r-693mut2 14909c3 — a rejection test names what does the rejecting
  • r-rel101 b65e34f — a distribution-boundary fix reaches nobody until it is released
  • r-689host c0e909b — a host is wired or reported undetected, never absent
  • r-686skil adbe186 — a permanent config never records a path that belongs to one invocation
  • r-682herm 2e64424 — recognition is by field, never by formatting
  • r-680ver 47359a1 — an assertion that reads the source it checks proves nothing
  • r-rel100 47359a1 — a published install URL must resolve the moment it is published
  • r-status550 de514ea — a setting and a behaviour never share one word
  • r-590gate 63e48fa — the preregistered verdict is the authority for published M5 figures
  • r-g1build 63e48fa — identity travels as version and digest, never as a path
  • r-g1e2e1 63e48fa — parity is only measured across process boundaries
  • r-gateplan 63e48fa — a plan that lives only in a session is lost at the next compaction
  • r-631cov 92c1b37 — coverage describes the index, history describes the sources
  • r-638bnd 2607bae — git reports the message's origin, not the commit's
  • r-cleanrebuild664 da8948d — the manifest binds dist to committed source, so any uncommitted edit anywhere in src makes a local verification meaningless — this is a property of the check, not of this branch
  • r-formatnotmachine661 2cc274d — verified locally only; whether the warning list's numbering also shifts when the check flips is a question only CI answers, because the runners are where the two runs actually diverged
  • r-selfscan661 ccab7f4 — ancestry is only knowable while the ancestor lives; a server orphaned by a doctor that has already exited is indistinguishable from any other session's, and is correctly counted as one
  • r-liveruntime660 6a221dbps is the seam, so this reports nothing on win32 and says so rather than claiming a clean machine
  • r-committedat650 f077870 — this pins the spelling, not the clock; nothing here makes two gits agree about anything else in %cI
  • r-prosetrailer647 30e40c7 — this changes the advice, not the outcome; the commit is still refused, which is correct while git reads the line as a trailer
  • r-repointsays629 c7ab87e — this reports the move, it does not verify the new target runs — hooks status owns that check and is unchanged here
  • r-childtreediag640 69c98a0 — the diagnosis is printed, not asserted — the next failure explains itself but the step still cannot say which outcome is acceptable
  • r-childtreepre640 69c98a0 — windows-latest is the only evidence for this path; nothing here was verified locally
  • r-coldstart640 69c98a0 — this measures the launcher's own child; it does not measure when cmd.exe started, so a stall before node is attributed to the same interval
  • r-launchernode640 69c98a0 — the earlier commit raising the probe budget to 15s stands on its own measurement — a healthy probe used 4478ms of 5000ms — but its message claimed that budget explained this flakiness, and it did not
  • r-launcherwhy640 69c98a0 — the deadline explanation is two independent diagnoses and a margin measurement, not a reproduction — the confirming evidence is the next run's own output
  • r-mcpidentity572 69c98a0 — Windows behaviour here rests on windows-latest runs and nothing else; a slower machine than three times the measured passing case will still be told it could not be verified, which is at least true
  • r-nodedrive640 69c98a0 — this reproduces the spawn, not the whole probe; a silent result narrows the cause without naming the fix
  • r-probefacts572 69c98a0 — the close-stdin outcome is measured, not guaranteed by the shell -- a platform whose shell defers the redirect past the parent's first write would produce the timeout code instead, and this case would then need its own seam rather than a looser assertion
  • r-proberace572 69c98a0 — this leaves one outcome unpinned, so a regression that swapped closed-input for timed-out on that fixture would not be caught here -- the four other cases still pin their codes exactly
  • r-reclaimnogate640 69c98a0 — windows-latest is the only evidence for this path, and one green run does not settle a race — this needs to hold across runs before The MCP probe's Windows behaviour is unverified: PATHEXT resolution and child-tree cleanup #640 can be called closed
  • r-sawinput640 69c98a0 — this records the arrival, not the response; if bytes arrive and nothing comes back, the next question is the probe's reader
  • r-slownotbroken640 69c98a0 — fifteen seconds is calibrated against one Windows runner's measurement; a machine slower than three times that will still be told it could not be verified, which is at least now true
  • r-mirrorsays632 aafd5ab — this states the retry, it does not verify it — nothing here checks that the following push actually published the note
  • r-keyringgen653 8b066c7 — the generation identifies the keys the verifier can list, not every reason git might reach a different verdict; expiry and revocation are outside what this binds
  • r-preflight002 0dca998 — MCP capture advertisement requires package manifest, SPEC, and schema to be available in the active runtime
  • r-recheck002 0dca998 — a readiness answer is only as fresh as the request that asked for it; nothing here prevents an asset vanishing between the check and the work
  • r-canon605 f474cf4 — esbuild resolves a platform-specific binary
  • r-rel0820 59c6730 — release versions must agree across manifests, lockfile roots, installer pins, and the runtime CLI
  • r-engfloor01 fe83524 — the parser covers the range shapes npm packages actually publish -- comparators like >=22 <23, and pre-release identifiers, are read by their first version and not by their bounds
  • r-nodefloor1 f4c924f — this bounds the version, not the feature -- a Node that ships node:sqlite behind a flag, or removes it, is not detected here
  • r-release081 ffe702a — the capture half reaches a host that surfaces MCP instructions; one that ignores that field still needs --agents-md, and nothing detects which kind a host is
  • r-cdeb10reg 48bd5a8 — wrong-path viability, deterministic oracle feasibility, code disclosure, bounded implementation, and unproven ordinary or benchmark authorship cannot be decided from history and remain undecided for human review
  • r-cdeb08an 60db89f — the paired bootstrap describes resampling stability within these five frozen repositories and thirty frozen tasks, and says nothing about any other repository, task or agent population
  • r-rel071 af8e0ab — 0.7.0 stays published with its notes amended to name the defect at the top; retracting a tag people may already have installed trades a known-bad version for an unknown one
  • r-rel070 d4a4d8b — the README's behaviour claim now rests on M5 while the generated numbers block beneath it still publishes M4, which is The README's generated numbers block still publishes M4; M5 measured the thing the README leads with #480 rather than a release-time edit
  • r-cdebver01 ce7b278 — the schemas freeze protocol 1.2.0 constants -- thresholds, matrix size, category names -- so a protocol change is a schema change and CI notices
  • r-m5analysis 3450656 — the script enforces the row count, not the identity of the rows; a run that produced 1,160 rows under a changed harness would satisfy it, which is what harness_commit and dist_digest on each row are for
  • r-benchscope 67f4375 — nothing checks the shape of the eight metric-row files. This gate names them and steps over them, and bench/deterministic/types.ts is the only definition that family has -- there is no JSON schema for it, so drift on that side is still invisible
  • r-benchscope 67f4375 — the pre-provenance exemption reads started_at, which is data on the row rather than a fact about the file. A row that misreported it would be held to the shorter list of requirements; that is a deliberate falsification rather than the omission this fixes, and nothing here detects it
  • r-3c9d52 dc9e769 — the sweep is two git log calls per path and the delivery phase runs git log --follow on every tracked path, so a full run over the four externals is hours rather than minutes on one machine
  • r-rel060 e999b9d — the install one-liner in all four READMEs now points at a tag that does not exist until this is tagged, so the window between merging to main and pushing v0.6.0 is one where the documented install is broken
  • r-release051 19810d2 — the hook is written at install time, so no release repairs a repository that already has one; every release touching hook behaviour has to restate what does
  • r-convertreadme e12c816 — a README claim about the default workflow is only true if the shipped skill performs it, and the skill currently requires the user to name CommitLore first
  • r-release050 ad402c7 — the hook is written at install time, so a corrected release never reaches a repository that already has one; every release fixing hook behaviour has to say what repairs an existing install
  • r-rel041notes 71efe1f — 0.4.1 makes the installer honest about a verification it cannot complete rather than fixing the kill, so an upgrading user may still see the unverified message instead of a version
  • r-rel040notes 5d57a72 — the 26.3-point density gap quoted in the notes is measured at this head and will drift with merge volume; it is illustrative of the denominator problem rather than a stable figure
  • r-rel021a a79e350 — v0.2.0 remains on the remote with no release attached
  • r-fix93pkg 9c4a396 — package.json remains a development artifact (build, typecheck, dependency floor) -- it is not read as a distribution manifest by anything in this repository
  • r-relinstall c6e1d04 — never tested against the real GitHub release infrastructure (no release exists yet — that is the owner's action) — verified against a locally built SEA binary, a hand-made SHA256SUMS, and a local HTTP server standing in for GitHub's release-asset redirects, which is everything this repository lets a change verify before a tag exists.
  • r-det058 695cdf6 — the suite must need no model, agent, network or uncommitted benchmark input
  • r-7a3e91 cf859e4 — better-sqlite3 stays external because it is native — the bundle degrades to --no-index without it, which only works because r-6f2a08 made that load lazy first
  • r-4a8e15 49e12c7 — git's grammar requires a subject before a trailer block, so a serialized block is not by itself a parseable message
  • r-7e5f02 e5f5e00 — npm installs through an engine mismatch, so the ecosystem's own signal cannot be relied on to stop anything
  • r-5a8c04 c46a577 — git owns the definition of a trailer block, so any behavior we cannot get from interpret-trailers is behavior we must not invent
  • r-c0f4e2 3d249cd — npm gitlore is held by an active same-domain CLI, so the owner's first-choice name was not available
  • r-b2e7f1 00d348d — Parsing must delegate to git interpret-trailers -- reimplementing the block rules would drift from the rest of the git ecosystem

Ruled out (168)

  • r-v4claimnarrowed a74aaa5 — answering the review in a comment and leaving the document as it was | the document is the record and the comment is not, so the overclaim would keep being read while the correction sat somewhere else
  • r-v4claimnarrowed a74aaa5 — collecting owner testimony now to close the P2 gap | it would be produced after the counts were visible, which the tier policy forbids, and the honest statement is that the route is untested
  • r-v4claimnarrowed a74aaa5 — re-running the review with the full four-part G2 question | the counts are visible now, so choosing the instrument at this point lets the result pick its own method; the gap is recorded instead
  • r-v4claimnarrowed a74aaa5 — discarding the review because one of its figures was wrong | it reported 41 where the artifact gives 43, and every load-bearing objection checked out
  • r-v4provtiers e9b167a — collecting owner testimony to fill the gap | it would be produced after the corpus counts were visible, which is the one condition the tier policy forbids
  • r-v4qualification b8ff1b9 — adjudicating the 92 split gates myself | the study operator reading their own corpus, already knowing how the pair voted, is the least blind reader available; a third blind vote costs one more session and is a vote rather than an override
  • r-v4qualification b8ff1b9 — averaging or passing an unresolved disagreement | it would put a candidate in the corpus that no two reviewers agreed on, and the disagreement would stop being visible
  • r-v4qualification b8ff1b9 — relaxing the quote-correspondence floor after seeing 8% | the floor was fixed in code and in the deviation record before any overlap was computed, and moving it now would let the count choose the method
  • r-v4qualification b8ff1b9 — accepting any rejection found in the same commit | it qualifies candidate X on evidence about decision Y, which is how a corpus fills up without meaning anything
  • r-v4delivery d2ff392 — keeping the Record-Id visibility gate | it measures the instrument's own restriction, and under it 43 of logic-pro-mcp's 43 decisions fail for a field that did not exist when they were written
  • r-v4delivery d2ff392 — a single substring match as the gate | a record sharing one phrase would pass, so the gate needs ruling, reason, scope and lifecycle together, with the out-of-scope probe as the part that can actually fail
  • r-v4delivery d2ff392 — running the working-tree dist | it has drifted from v1.2.0, so it would measure behaviour nobody shipped
  • r-v4redactionsecondpass 9222978 — changing the product's redaction to match | it is right for the product, where a prose sentence that looks like a trailer must not be treated as a record, and the benchmark's opposite bias does not belong in shipped behaviour
  • r-v4redactionsecondpass 9222978 — excluding the eleven affected candidates | the contamination is in the packet, not in the decision, and dropping them would lose real corpus to a fixable tooling gap
  • r-v4redactionsecondpass 9222978 — stripping the one prose echo of a ruling | the body states the rejected approach in its own words, which is the independent support G2 is asking about
  • r-v4provenance 62f0ac4 — using the rendered record as gold | it proves the record is readable, which is the thing under test, and any effect measured that way is the instrument measuring itself
  • r-v4provenance 62f0ac4 — judging independent recoverability here | that is a reading of the prose, and a program that scored it would be encoding one reviewer's judgment as a mechanical fact
  • r-v4provenance 62f0ac4 — calling buildSourcePacket per candidate | it re-materializes the sealed bundle each time, so 241 candidates would clone 87MB once per logic-pro-mcp decision
  • r-v4stage0id 6b427af — keeping a hardcoded list of terminal study ids in the resolver | the list and the studies drift apart silently, and the drift shows up as a terminated study resolving cleanly
  • r-v4stage0id 6b427af — leaving the declaration at null and passing the study root explicitly everywhere | every caller then carries the choice, and the one caller that forgets picks a default nobody reviewed
  • r-v4stage0id 6b427af — reusing the v3r1 study directory under a new name | §4.3 requires a new study id, and a renamed directory keeps the qualification verdicts this estimand discards
  • r-v3manifestsync c195b48 — keeping the generator out of package.json to avoid touching the manifest | the check has to run somewhere the repository actually invokes, and hiding it from the script list would leave the drift guard unwired
  • r-v3terminalseal 7754f1a — correcting the placeholder digests in place | the correction is indistinguishable from the mistake it repairs, and the row is historical evidence rather than a working value
  • r-v3terminalseal 7754f1a — recomputing digests for the historical rows from today's artifacts | the artifacts have changed since, so the result would be a number that never bound anything, wearing the authority of one that did
  • r-v3terminalseal 7754f1a — hand-maintaining evidence-matrix.md beside the JSON | two copies of the same claims disagree eventually and the disagreement is silent
  • r-v3terminalseal 7754f1a — leaving cdeb-fresh-v3r1 as the default study root | a terminated study as a fallback is how a measured run gets attempted against a study that ended
  • r-guardratchet 26b1989 — leaving the job in the gate while it is red | branch protection would block every merge until someone deleted the job, and deleting it removes the only thing that can see these gaps
  • r-guardratchet 26b1989 — allowing an improvement without updating the baseline | the record would drift below the measurement, and a baseline that overstates the gaps is as useless as one that understates them
  • r-guardratchet 26b1989 — recording gaps as a count instead of per property with a reason | a count cannot distinguish a control nobody wrote from one that cannot exist, and those need different work
  • r-guardmutation 551921d — fixing the inert guards in the same change | a runner that has never reported a real failure is not known to report one, and the red run is the evidence that it can
  • r-guardmutation 551921d — indexing guard functions instead of claims | that reproduces the exact failure this exists to stop, because the control comes back out of the implementation it is meant to test
  • r-guardmutation 551921d — treating an unexpressible control as a skip | it is indistinguishable from a control nobody attempted, and both were silently green before
  • r-guardmutation 551921d — folding this into the check job | it runs vitest once per mutation, so it belongs in its own job where its cost is visible
  • r-coauthoredcasing 06354f9 — adding Co-Authored-By to the set as a second entry | the next casing git or an editor emits is refused again, and the set would be a list of spellings someone has to keep guessing at
  • r-coauthoredcasing 06354f9 — lowercasing every key before the whole check | KNOWN_KEYS is SPEC vocabulary and SPEC matches it case-sensitively; widening that is a protocol change, not a bug fix
  • r-detectrule728 f1784ce — give claude-code a config-directory fallback for symmetry | it would report a host detected and then perform no wiring, which is the false success this installer exists to avoid
  • r-detectwhy728 f1784ce — give claude-code a config-directory fallback for symmetry | it would report failed and set ok false on a machine whose only trace of the host is a leftover config, failing an install that had nothing wrong with it
  • r-rel111 8c29f5d — claim Windows host wiring works | two of the four detected hosts wire, and a release note that rounds that up is the false green this release exists to remove
  • r-winargv716 7bf5b82 — shell: true | wrapper and user paths would become a shell injection surface
  • r-winexec716 7bf5b82 — shell: true | wrapper and config paths must not become unchecked shell input.
  • r-rel110 d9a041f — fold the second Windows cause into this release | it arrived as Fix Windows host resolution and batch spawning (#716) #720 with real Windows evidence and needs its own judgement, and holding this back would make the note about what is still broken false in both directions
  • r-rellock110 d9a041f — replace the version string throughout the lockfile | it matches four dependencies that are really at 1.0.2, and nothing in the suite would have caught it
  • r-namefile716 f728e69 — add a path field to HostResult | the summary schema is pinned at v1 and detail is already the free-text field a human reads
  • r-overlay709 7e08cbf — let an overlay only narrow permissions | it solves the contributor who wants less, and the one who wants more still edits the tracked file, which is the reported failure
  • r-overlay709 7e08cbf — write a .gitignore entry for the overlay | a tool that hides a file on a repository's behalf has decided for the repository what it may not see
  • r-wintmp716 f0ed513 — use basename() | correct on Windows, unfalsifiable on the platform CI actually runs
  • r-pend710 b452535 — drop the warning for the non-staged case | a stale draft is still worth surfacing, and doctor reports a consumed capture as never written to history #584 already showed that going quiet on a healthy-looking path is how this check loses its meaning
  • r-cleanrebuild664 da8948d — rebuild again and hope | the first rebuild was real; what was wrong was the tree it read, and running it once more from the same tree would have produced the same mismatch
  • r-formatnotmachine661 2cc274d — revert Report the MCP servers that are answering, not the ones registered #661 | it would take the check and its regression back out, and rebuilding both costs more than the line this change adds while main stays red
  • r-selfscan661 ccab7f4 — mark the probe's child through the environment | ps reports arguments, not environment, so the marker would be invisible exactly where the decision is made
  • r-selfscan661 ccab7f4 — exclude by direct parent only | the server is a grandchild of whatever probed it, so one link is not the relationship that matters
  • r-liveruntime660 6a221db — rebase the original branch | it predates the probe rewrite and the sidecar, so 29 of its conflicts were in code those changes already resolved differently
  • r-liveruntime660 6a221db — compare reported versions between runtimes | a copied or stale install can declare the same version as a current one, so equality there proves nothing about identity
  • r-liveruntime660 6a221db — canonicalise the totals line in the snapshot | it would hide a real change in the check set behind a token, which is what that snapshot exists to catch
  • r-pinnedreport660 6a221db — canonicalise the totals line as well | it would hide a real change in the check set behind a token, which is the thing that snapshot exists to catch
  • r-pinnedreport660 6a221db — leave the check out of the default registry to keep the report stable | a check nobody runs reports nothing, and the stale runtimes it names are on real machines rather than in fixtures
  • r-committedat650 f077870 — normalise every timestamp to UTC | an offset carries where the commit happened, and discarding it to gain uniformity loses a fact the record is entitled to keep
  • r-committedat650 f077870 — canonicalise at the JSON boundary only | the index stores the same field, so the two routes would then disagree with each other rather than with git
  • r-prosetrailer647 30e40c7 — treat sentence-like lines as prose rather than trailers | SPEC §2.1 B3 gives git the trailer boundary, and re-deciding it here would make our parse and git's disagree about the same message
  • r-prosetrailer647 30e40c7 — enforce key validity only when every line in the block parses as Key: value | same objection, and it would silently stop validating a mistyped Record-Id sitting beside a prose line
  • r-prosetrailer647 30e40c7 — a multi-word value alone as the prose signal | measured against the conformance fixture, which it misclassified — the fixture caught the heuristic, and the heuristic changed rather than the fixture
  • r-repointsays629 c7ab87e — drop the word unchanged | the file genuinely is unchanged and that is worth saying; the defect was that it was the only thing said
  • r-repointsays629 c7ab87e — report the recorded target on every install | on a first install there is nothing to compare against, and a line that always appears stops carrying information
  • r-bundleprobe636 69c98a0 — sidecar MCP helper | it is outside the declared single-file runtime
  • r-childtreediag640 69c98a0 — guess between the two suspects and change the fixture again | the previous change was a reasonable guess that did not hold, and a second guess costs another Windows round trip with no more information than the first
  • r-childtreepre640 69c98a0 — weaken or delete the failing step | it is the only coverage of Windows child-tree reclamation in the repository, and no unit test touches taskkill
  • r-coldstart640 69c98a0 — read the node-spawn driver's silence as proof the spawn loses bytes | it shares the same five-second window, so slowness would make it silent too and I would have read a timing result as a delivery result
  • r-initiator636 69c98a0 — asset readiness preflight | F-002 and prepare_capture fails with ENOENT on a dev-hash asset root that is never installed #633/commitlore_prepare_capture fails with ENOENT on a 0.6.0 spec path; the Claude plugin cache only has 0.8.0 and the codex cache has 0.8.2 #635 own that separate runtime check
  • r-launchernode640 69c98a0 — keep the hard-coded path and widen the timeout | no budget reaches an interpreter that is not on disk, and three attempts had already shown that
  • r-launchernode640 69c98a0 — treat this as the product defect | a registration naming an interpreter that does not exist should be reported unhealthy, and the probe running the registered command verbatim is the behaviour to keep
  • r-launcherwhy640 69c98a0 — revert to the hard-coded path | it works on this image and testing against the Node the job installed is still the more honest fixture; the correction here is to the reason, not to the code
  • r-mcpid572 69c98a0 — a separate MCP protocol probe | reuse the Both installers still skip a host whose registration cannot run, and exit 0 #595 implementation to prevent drift
  • r-mcpidentity572 69c98a0 — widen the budget and keep the wording | the message would still send an operator to repair something that works, just less often
  • r-mcpidentity572 69c98a0 — retry the probe | a retry pays another full budget to answer a question the first attempt already answered, and doubles doctor's worst case
  • r-mcpidentity572 69c98a0 — gate the child-tree assertion on the server being healthy first | reclamation runs on every probe exit including the timeout, so requiring health made the precondition depend on the behaviour under test and could not tell a broken product from a broken fixture
  • r-nodedrive640 69c98a0 — instrument mcp-probe.ts itself | product code should not carry diagnostics for one platform's fixture, and the mechanism is reproducible without touching it
  • r-probefacts572 69c98a0 — accepting either code in one case | it makes the regression pass whether or not the probe still separates the two facts, which is the property under test
  • r-proberace572 69c98a0 — accepting either code in both fixtures | that is what makes the pair meaningless, since neither would then hold its own fact
  • r-proberace572 69c98a0 — forcing determinism by having the fixture signal readiness before closing stdin | the probe writes immediately on spawn, so there is no point at which a fixture could report ready first without changing the protocol
  • r-probesync572 69c98a0 — treating stdout as completion | spawnSync waits for helper exit after stdout
  • r-reclaimnogate640 69c98a0 — keep gating on probe status and retry the step | the gate races on cmd.exe stdin ownership, so retries would convert a real signal into a slower coin flip
  • r-reclaimnogate640 69c98a0 — mark the path unsupported | reclamation is implemented and unconditional in the source; what was missing was a test that could reach it
  • r-sawinput640 69c98a0 — drive the wrapper through a pipe instead of a file as the discriminator | PowerShell closes the pipe after writing, so it would test EOF rather than the held-open pipe the probe uses
  • r-slownotbroken640 69c98a0 — widen the window and keep the wording | the message would still tell an operator to repair something that works, just less often
  • r-slownotbroken640 69c98a0 — retry the probe | a retry pays another full budget to answer a question the first attempt already answered — the server was not reached in time — and doubles doctor's worst case
  • r-slownotbroken640 69c98a0 — keep the sidecar's outer bound as a constant | it was 7000ms against a 5000ms budget, so raising one without the other would kill the helper before it could answer and report that death as the server's fault
  • r-stub572 69c98a0 — retaining the fixed one-line stub | it is not an MCP server and the new probe correctly rejects it
  • r-trimdiag640 69c98a0 — keep the drivers for the next investigation | they are cheap to write again from the pull request comment, and a permanent cost for a question already answered is the wrong trade
  • r-winprobe640 69c98a0 — process manager framework | taskkill is a bounded Windows helper
  • r-mirrorsays632 aafd5ab — fail the push when the mirror cannot be published | a record that could not be mirrored is not a reason to withhold code, and the hook's contract is that it never blocks
  • r-mirrorsays632 aafd5ab — keep one sentence and lengthen it | the two outcomes need opposite actions from the reader, and one sentence covering both is what left it silent on the actionable half
  • r-keyringgen653 8b066c7 — memoize the generation per process | a long-running server would keep grading against the keyring it started with, which is the shelf-life defect just removed from the MCP capture gate
  • r-keyringgen653 8b066c7 — stop caching signature_status | it is read per graded record, and recomputing it per query costs the git log the index exists to avoid
  • r-f002onf001 0dca998 — keep the local packageVersion reader | it is a second answer to a question F-001 now owns, which is the divergence this pair of findings exists to remove
  • r-preflight002 0dca998 — code-only tool advertisement | a stale runtime can expose capture after its SPEC is gone
  • r-recheck002 0dca998 — keep the startup snapshot and document the limitation | the runtimes this finding came from had all outlived their snapshot, so documenting it would describe the defect rather than remove it
  • r-recheck002 0dca998 — a filesystem watcher or a daemon | a stat at the boundary answers the same question without a process to supervise
  • r-oneinstall001 31cf0d1 — filename equality | two shipped entrypoints of one install are one runtime
  • r-runtime001 31cf0d1 — version-only comparison | equal version strings still allow different entrypoints and package roots
  • r-runtime001 31cf0d1 — filename equality | two shipped entrypoints of one install are one runtime
  • r-signer597 3dc75e7 — signer email or author header matching | either is commit-controlled metadata and does not bind the verified key to repository authority
  • r-canon605 f474cf4 — remove the legacy dist sidecars now | that changes test harnesses and needs a separate reviewed reduction
  • r-rel0820 59c6730 — rebuild dist | the CLI reads package.json at runtime and a rebuild changes the bundle digest without changing behaviour
  • r-engfloor01 fe83524 — adding semver as a dependency to parse this | one regex over a handful of published shapes does not justify a runtime dependency in a check that runs before install
  • r-cdeb08an 60db89f — discovering row files under the result directory | an unregistered file contaminates the matrix while leaving every stopping rule looking satisfied
  • r-cdeb08an 60db89f — filling or dropping unavailable usage | both change a token aggregate without evidence, one by inventing a number and one by redefining the population
  • r-rel071 af8e0ab — deleting the v0.7.0 tag and re-cutting it | rewriting a published ref breaks every clone that already fetched it, and the release notes can carry the truth without that
  • r-rel071 af8e0ab — reading the flag with a nullish check and documenting the commander default | the documentation would sit in this file while the defect sits in every install, which is the arrangement that produced this
  • r-rel070 d4a4d8b — bumping the manifests first and the README pins after | the readme suite pins them to package.json, so the intermediate commit is one where CI is red and the documented install is wrong
  • r-rel070 d4a4d8b — rebuilding dist for the version change | the version is read at runtime, and a rebuild would move the digest every M5 row records without changing a byte of behaviour
  • r-cdebver01 ce7b278 — default-in discovery over the CDEB tree like the legacy gate | that gate's enemy was the unregistered file, CDEB's is the unregistered input, and bench:m5 reads every jsonl in bench/results, so the registered analysis would run over M1, M4 and files marked non-citable #441 showed a glob passing a stopping rule on contamination
  • r-cdebver01 ce7b278 — validating with the draft-07 Ajv entry and downgrading the schemas | the 2020-12 entry ships in the same package, and weakening six schemas to fit a meta-schema is backwards
  • r-m5analysis 3450656 — Computing the table now and not looking | the file would exist, and a number that exists is a number someone reads
  • r-m5analysis 3450656 — Leaving the analysis to be written after the run | that is the freedom this document was written to give up, and writing it afterwards is indistinguishable from writing it to fit
  • r-m5analysis 3450656 — Asserting the M1 control in a vitest suite instead of --validate | it belongs to the analyser, and a reader checking the verdict should be able to re-derive the control with the same command that produced the verdict
  • r-benchscope 67f4375 — A declared list of result files, in the gate or in a manifest | opt-in under another name, and a file left off it is silently ungated -- which is precisely the failure bench/verify.mjs is not run by anything, so the result schema drifted unnoticed #392 reports
  • r-benchscope 67f4375 — Validating only the three files that pass today | a description of the present rather than a rule, and it goes stale the first time a matrix is added or the schema moves
  • r-benchscope 67f4375 — A naming convention on results filenames | the runner's default output is an ISO stamp plus hex and the metric writers' is a family name plus an ISO stamp, but every run-record file in the tree was named by hand through --out (t702-m2, m5-off-design-20-tasks), so the convention is enforced nowhere and cannot carry a gate
  • r-benchscope 67f4375 — A marker file dropped beside each gated result | still a step somebody has to remember, and a forgotten step is the defect being fixed
  • r-benchscope 67f4375 — Widening result.schema.json to describe both row families | they share no required field, so the union would accept a run record with no task and a metric row with no metric, which is a schema that rejects nothing
  • r-benchscope 67f4375 — Making harness_commit and dist_digest optional in the schema | weakens the gate for every future row in order to admit six historical files, and Make per-turn token usage observable, and say what it still cannot price #390's drift was an additionalProperties failure that a laxer schema catches later rather than sooner
  • r-benchscope 67f4375 — Skipping the six pre-provenance files whole | cheaper to write, but it stops checking types, patterns and the closed property set on 380 rows in order to excuse two absent fields
  • r-3c9d52 dc9e769 — reimplementing the scorer for the external corpus | two scorers that drift by a line produce two numbers that look comparable and are not, which is the failure No-index query at 100k regressed from 7.7s to 90s — cause not yet identified #163 already recorded here
  • r-3c9d52 dc9e769 — replacing the per-path --follow chain with one forward pass over a rename graph | it is much faster and it is a different attachment predicate from the one DECISION-DELIVERY.md §3 registered, so the two corpora would stop being comparable
  • r-3c9d52 dc9e769 — turning the census notes option on by default | every measurement registered before it existed read commit messages only, and a denominator that silently grew would invalidate them without saying so
  • r-rel060 e999b9d — 0.5.2 | three observable behaviour changes and one narrowed acceptance rule are not a patch, and an upgrader reading the number would not go looking
  • r-rel060 e999b9d — Publishing the delivery figure in the README | the measurement is one corpus and one repository, and the README already carries more numbers than a first-time reader can weigh; docs/evidence.md and bench/ hold it
  • r-release051 19810d2 — Waiting for 0.5.2 to carry index ingests any key: value line as a trailer; doctor reports 106 records where git has 0, and context serves commit subjects to the agent #335 | false decision context reaching an agent is the inversion of the product claim, and a stable release should not hold it
  • r-release051 19810d2 — 0.6.0 | no command is added and nothing breaks; calling it minor would hide a correctness fix behind a feature number
  • r-release051 19810d2 — Filtering Verified: because a report listed it as noise | it is vocabulary, and separating a release-note use from a record use needs a guess that can discard real records
  • r-convertreadme e12c816 — Adding "Keep coding normally" now | the commit skill still triggers on explicit requests and teaches harvest; shipping the sentence first repeats the doc-ahead-of-code failure this session fixed twice
  • r-convertreadme e12c816 — Writing testimonials to fill the social-proof section | one real field report exists, and fabricated or padded quotes would cost the claim honesty that is this project's strongest asset
  • r-convertreadme e12c816 — Moving the benchmark block and protocol detail out of the README now | the generated block is byte-compared by check-readme-numbers and the guard figures are asserted inside the limitations slice; the move is worth doing and is worth its own change
  • r-convertreadme e12c816 — Dropping "decision authority" everywhere | it is the right term in the architecture sections; it is only the wrong term for a first screen
  • r-release050 ad402c7 — 0.4.2 | a new command and a platform support claim are not a patch, and calling them one hides both from anyone reading version numbers to decide whether to upgrade
  • r-release050 ad402c7 — 1.0.0 | nothing here breaks an existing install, and the milestone that closed is Gate B rather than a stability commitment
  • r-release050 ad402c7 — Describing Windows as supported without the caveat | the word is the one most likely to be read as more than it is, and the release that earns it is the right place to bound it
  • r-nobin284 ab2f08f — Keeping the binary classification arm for the installer's wrapper | the wrapper is a shell script that execs node, so trusting it by name would re-admit any extensionless executable called commitlore while the bundle it runs is the thing with an interpreter to check
  • r-nobin284 ab2f08f — Relaxing the dogfood Evidence check to allow a missing path | it exists to catch a citation nobody can follow, and dropping the check to accommodate a deletion would retire the guarantee instead of correcting the question it asked
  • r-nobin284 ab2f08f — Leaving src/core/paths.ts alone because ADR-0026's inventory did not name it | it was the only remaining code that read files out of a compiled artifact, so leaving it would have left the invariant false while the test asserting it passed
  • r-nobin284 ab2f08f — Keeping /dist/commitlore in gitignore in case a binary returns | an ignore rule for an artifact nothing produces is a claim that something might, and the ADR says otherwise
  • r-rel041notes 71efe1f — folding this into the next feature release | the documented install is broken for upgraders now, and every day it stays tagged is a day the one-liner in the README fails for them
  • r-rel041notes 71efe1f — moving the v0.4.0 tag to the fixed commit | a tag users may already have fetched must not change under them; the fix gets its own version
  • r-rel040notes 5d57a72 — describing 0.4.0 as making agents safer | no study here measures agent behaviour, and the recovery protocol is registered and unrun; the release delivers a loop, not an outcome
  • r-rel040notes 5d57a72 — omitting guard's figures from the release notes because they are unflattering | the notes are read by people deciding whether to trust the tool, and a quality limit hidden at release time is discovered in use
  • r-rel021a a79e350 — retagging v0.2.0 | the tag is already on the remote and moving a published tag breaks anyone who fetched it
  • r-rel0200a a074754 — bumping ci.yml's "v0.1.0 was published with zero attached assets" comments | those describe a historical fact about the actual v0.1.0 release, not a version this project declares; the check they document (releases/latest/download/SHA256SUMS returning 200) is written to start exercising the real path automatically the day any release ships assets, v0.2.0 included, with no workflow edit
  • r-rel0200a a074754 — touching docs/adr/ADR-0001-scope-v010.md, docs/tickets/release.md, bench/VERDICT-M1.md, HANDOFF.md, bench/ROUTE-GAP.md | planning and historical-record prose that names v0.1.0 as a past decision or measurement subject, not a live version carrier
  • r-rel0200a a074754 — changing test/mcp.test.ts's CommitLore-Version: 0.1.0 fixture trailer | that's protocol-version content inside a synthetic seed commit (what an old commit's trailer looked like), unrelated to and never asserted against package.json's version
  • r-fix93pkg 9c4a396 — repointing bin at dist/commitlore.mjs instead of removing it | that entry exists only to serve a package-manager install ADR-0011 already replaced with a git clone; repointing it keeps maintaining a path to an install mode that will never happen
  • r-fix93pkg 9c4a396 — also removing files now | the issue's own "After B-09 · Single static binary — remove the Node runtime dependency #39" note groups bin/files/dependencies as one cleanup once the static binary lands; files alone is inert without a registry rather than actively broken, so doing it now means doing it twice
  • r-relinstall c6e1d04 — guessing the current version to build the asset URL directly | would need either the GitHub API (rate-limited, needs no-auth headers handled correctly) or trusting a redirect's final Location header parsing. Downloading the fixed-URL SHA256SUMS first and reading the real asset name back out of it needs neither and is what the checksum step has to fetch anyway.
  • r-relinstall c6e1d04local for scoping — not POSIX per se, but supported by dash, bash, and every shell this script is realistically piped into (verified directly, see Verified) | not used in the end; the script has few enough variables that scoping was not needed, only noted here because it was considered.
  • r-det058 695cdf6 — add the 100k history suite to CI | CI budget and deterministic local measurement are separate decisions
  • r-7a3e91 cf859e4 — inlining spec/SPEC.md and the schema into the bundle | SPEC.md would need a codegen step that itself needs a drift guard, and the package-root walk removes the reason to want it
  • r-7a3e91 cf859e4 — replacing the tsc output with the bundle | test/cli.test.ts, test/hooks.test.ts and test/mcp.test.ts import dist internals by path
  • r-0d4b81 8005227 — a longer quickstart that demonstrates context, limits, ruled-out, warnings and stale | an agent calls those itself once the MCP server is registered, so listing them teaches the human a workflow that is not theirs
  • r-4a8e15 49e12c7 — make the parser accept a bare block | it would disagree with git, and delegating to git is the one thing SPEC section 2 requires
  • r-7e5f02 e5f5e00 — hold Node 20 and downgrade both dependencies | paying maintenance cost to keep supporting an unpatched runtime
  • r-7e5f02 e5f5e00 — floor at 24 | excludes Node 22 users today for no benefit, and 22 outlives v0.1.0
  • r-7e5f02 e5f5e00 — drop engines entirely | the failure moves from install time to runtime, where it is someone else's confusing bug
  • r-5a8c04 c46a577 — regex trailer parsing | B3 prose false-positives are the failure mode this protocol exists to prevent
  • r-5a8c04 c46a577 — a custom GitError subclass | the convention here is new Error with structured context, and callers read GitResult
  • r-c0f4e2 3d249cd — GitLore published as git-lore | the binary and search results still collide with the existing gitlore tool
  • r-c0f4e2 3d249cd — keep Annals | the sound problem does not decay, and with code near zero this is the cheapest moment the project will ever have
  • r-c0f4e2 3d249cd — rename code and spec first, documents later | the drift window makes every artifact written in it wrong
  • r-b2e7f1 00d348d — line-matching Key: prefixes | prose containing a colon line parses as a record and feeds agents false context (verified B3)
  • r-b2e7f1 00d348d — content-hash Record-Id | must survive rebase and squash, which a hash of the commit does not

Warnings (17)

  • r-cdeb08an 60db89f (claim) — the orchestrator must freeze 180 opaque analysis_inputs.row_files and write canonical rows only at those paths; a row written elsewhere will be refused rather than found
  • r-rel071 af8e0ab (claim) — any option declared with a non-nullish default has the same shape, and nothing checks for it -- this fix repairs one site, not the class
  • r-rel070 d4a4d8b (claim) — between the promotion merge and the tag push, every README install one-liner points at a tag that does not exist -- tag promptly and confirm the URL rather than assuming
  • r-cdebver01 ce7b278 (claim) — every failure fixture is the one valid row with a single mutation, so keep the valid-row builder in lockstep with result.schema.json -- a builder drift fails all eleven cases at once and reads as a broken verifier
  • r-3c9d52 dc9e769 (claim)budgeted_log_coverage is not recall — it counts commits, not records, and only the calibration row at b3f5692 says how far apart the two are on a repository where both are known
  • r-nobin284 ab2f08f (claim) — an extensionless COMMITLORE_BIN is now ignored rather than honoured -- the hook falls through to the recorded install and to PATH, so a caller who pointed it at the wrapper still gets a working hook, by a different route
  • r-release030 510d70e (claim) — Tagging and publishing remain separate release steps.
  • r-rel0200a a074754 (claim) — scripts/commitlore-bootstrap.sh is orphaned -- no hooks.json entry invokes it, and its npm-install strategy contradicts ADR-0011. It still carries a live version default, now bumped for consistency, but nothing exercises it. Worth a follow-up issue: either wire it up correctly or delete it.
  • r-det058 695cdf6 (claim) — guard precision is against the frozen archived labels, not a new hand-adjudication
  • r-7a3e91 cf859e4 (claim) — hardcoding ../ counts back to the package root is what broke this — new code reads assets through installedPath(), never through import.meta.url
  • r-0d4b81 8005227 (claim)claude mcp add commitlore -- commitlore mcp is Claude Code's syntax — other MCP clients register a stdio server their own way
  • r-4a8e15 49e12c7 (claim) — canonical output is a block, not a message -- prepend a subject before parsing it back, or the round-trip silently returns nothing
  • r-7e5f02 e5f5e00 (claim) — raising the floor again needs an ADR superseding ADR-0010 -- a package.json-only change will pass tests and lie to users
  • r-5a8c04 c46a577 (claim) — do not add --grep anywhere under src/ -- test/source-guards enforces its absence and will fail the build
  • r-c0f4e2 3d249cd (claim) — ADR-0008 and ADR-0009 keep the literal string Annals on purpose -- mechanical substitution there destroys the decision trail
  • r-c0f4e2 3d249cd (claim) — the residual grep for lore_query reports a false positive because commitlore_query contains it as a substring, so check the prefix
  • r-b2e7f1 00d348d (claim) — spec/fixtures and spec/contract-cases are the contract -- prose in SPEC.md explains them, it does not define them

git log --follow accepts exactly one pathspec, so renames are not followed for 26 paths; query one path at a time to follow its rename chain

withheld the content of 1 record(s) graded blocked: a Verified trailer matching an injection pattern is reported, never quoted (SPEC §7)

Trailer violations fail this check. Active constraints are informational — they are what the repository already decided, not a verdict on this PR.

@MongLong0214
MongLong0214 changed the base branch from cdeb-v4-delivery to main August 21, 2026 23:52
@MongLong0214

Copy link
Copy Markdown
Owner Author

Retargeted to main and updated. This branch's history already contains #827 (provenance) and #828 (delivery), so merging this one lands all three sets of commits with their own messages and records. Merging them individually would need a fresh full-suite run per PR after each branch update; this is the same content in one CI cycle.

#827 and #828 stay open until this merges, then close as already-merged.

…to say which

The result reported P1 counts and said nothing about P2, the owner-attested tier. Read
alone, an absent tier could mean owner testimony was collected and rejected, or that none
was collected at all. Those are different claims and only one of them is true here.

    P1           17
    P2            0
    unsupported 224

No owner testimony was collected in Stage 0, so P2 is empty by construction. Whether an
attested candidate may enter a primary corpus is a decision for a later preregistration,
and the report now says that rather than leaving a zero to be interpreted.

Record-Id: r-v4provtiers
Provenance: authored
Certainty: firm
Blast: local
Undo: easy
Ruled-out: collecting owner testimony to fill the gap | it would be produced after the corpus counts were visible, which is the one condition the tier policy forbids
Limit: this reports the tier split. It takes no position on whether P2 evidence should ever be admitted, which is not Stage 0's call
Verified: 18 tests pass including the generator's --check against the committed Markdown
CommitLore-Version: 2.0.0
…rrow one the evidence supports

An adversarial review of this study's own result, run before publication, confirmed the
HOLD reproduces and refuted the sentence built on top of it. It was right on five counts
and they are all now fixed in the record rather than in a reply.

The result said the record is "the only place" the rejection exists, and that the
robustness arm "rules out" the alternative explanation. Neither follows. The pool is what
the `Ruled-out:` trailer discovers, not a census of these repositories' decisions. Pull
requests, issues, design documents, comments and tests were never searched. The arm
broadened the packet in one direction, by one commit's diff, on 60 candidates, and moved
the count from 6 to 8. Owner testimony, which the preregistration permits as an independent
tier, was never collected -- so the P2 route to gold is untested, not closed.

Two protocol changes were made and not recorded, which is the part that matters most.
§8 names an ADJUDICATOR for split pairs and the implementation takes a third blind vote;
that decided 19 Stage A and 73 Stage B gates and appears in the artifacts as `adjudicated`.
And G2 registered four components -- decision, reason, scope, lifecycle -- while the
implementation compares only the quoted alternative against the ruling. The quoted reason
is never compared, and scope and lifecycle recovery are never tested. Both are now
deviations. Neither moves the verdict: reason-quote overlap passes 15 against the ruling's
17 at the same floor, and a candidate that fails the alternative comparison cannot pass the
full gate either.

The floor itself does more work than the prose admitted. 159 pairs found *a* rejection and
17 matched *this* one, so the correspondence rule -- not bare absence -- separates most of
them. The sensitivity is published now:

    floor 0.200  46      floor 0.300  24      floor 0.340  17  (registered)
    floor 0.250  39      floor 0.333  24      floor 0.500  14

The verdict survives all of them: the most generous floor still passes fewer than the
registered 48, before the other six gates take their share.

The delivery gate needed three bounds named too. Scope is tested against one non-touched
path. Lifecycle is not read from the payload -- an active decision counts correct whenever
its ruling is visible, so that field only discriminates the superseded cases.
`before_first_mutation` is true by construction because the payload is a synthetic
`PreToolUse` event, not an observation of a real agent.

The review is committed verbatim beside the result. A review that survives only in the
changes it caused cannot be checked by a later reader.

Record-Id: r-v4claimnarrowed
Provenance: authored
Certainty: firm
Blast: system
Undo: easy
Ruled-out: answering the review in a comment and leaving the document as it was | the document is the record and the comment is not, so the overclaim would keep being read while the correction sat somewhere else
Ruled-out: collecting owner testimony now to close the P2 gap | it would be produced after the counts were visible, which the tier policy forbids, and the honest statement is that the route is untested
Ruled-out: re-running the review with the full four-part G2 question | the counts are visible now, so choosing the instrument at this point lets the result pick its own method; the gap is recorded instead
Ruled-out: discarding the review because one of its figures was wrong | it reported 41 where the artifact gives 43, and every load-bearing objection checked out
Limit: G2 as implemented bounds the qualified count from above for the whole gate, so the HOLD stands. It does not answer whether complete independent gold could be written, because scope and lifecycle recovery were never tested
Verified: the reviewer's numeric claim was recomputed before acting -- 43 candidates with nonzero overlap below the floor, 7 of them at or above 0.333, against its 41 and 7. 18 tests pass, both typechecks clean, and the generator's --check holds the prose to the artifacts
CommitLore-Version: 2.0.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant