Skip to content

Check ledger rule citations as verbatim excerpts, fix the stale ones - #632

Merged
derek73 merged 4 commits into
masterfrom
fix/ledger-citation-excerpts
Oct 11, 2026
Merged

derek73 merged 4 commits into
masterfrom
fix/ledger-citation-excerpts

Conversation

@derek73

@derek73 derek73 commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Why

test_citations_are_verbatim_excerpts swept only *.py, and only the colon form's first quote. So the rule citations in tools/differential/expected_since_*.toml — which AGENTS.md puts under the excerpt discipline — were checked by nothing, and so was every citation in another shape. #631's review found five stale ledger quotes by hand. Measured 2026-10-10 by running this PR's check over the tree before it (8e6e524, master after #631): 27 more stale quotes and 5 colon citations quoting nothing; review found 14 more beyond the check's reach (below).

Scanner (tests/v2/test_doc_citations.py)

  • Sweeps the ledgers beside the code (this module's own illustrative comments excluded).
  • Reads a quote in every shape the tree writes: rules.md#X: "…", rules.md#X -- "…", rules.md#X's Accepted clause ("…"), a verb between (rules.md#S2 consumes "…"), and every quote chained on by "and"/comma/dash.
  • Quotable text: a rule's statement and its Accepted: clauses (not example or pointer lines); a section letter's Background (rules.md#H Background); a mechanism's Contract statement; a decisions entry's text. A decisions citation may paraphrase, but what it quotes must be verbatim.
  • Fix: decisions.md#X: was looked up in rules.md's statements, so decisions.md#P6: "…" was checked against rules.md#P6.
  • A reference with no quote must still name an existing rule, section, mechanism or decisions entry. Lowercase decisions keys are matched; the ### differential-ledger, … arc headings merge onto their one key.
  • [...] elision (fragments verbatim, in order, non-empty); -- reads as —, a nested ' as "; #: continuations and hyphen-wrapped IDs followed.
  • test_implemented_matches_citing_modules keeps its population: colon-form .py citations only.
  • Guards: a reach test asserting every lead shape is live ('s, parenthesis, dash, bare words, plus ledgers, chained, decisions, elided); a test pinning what is quotable (Accepted clause, a Background's wrapped line) and what is not (example lines, a wrapped no-boundary: marker); and _RETIRED_EXCERPTS, the recorded negative control for the quotable text.
  • Stated limit: a quote is checked when it stands within four words of its ID or is chained to one that does. A quote further off reads like a name written in double quotes — checking every quote in the paragraph was measured at 139 failures, nearly all names — so it is not checked, and AGENTS.md says to keep a quote beside its ID. Case folds, so emphasis in capitals is unchecked.

Stale citations fixed

Citation Problem
C1, S2 (title run), T3 reworded rule / unquoted paraphrase (first commit)
P5, P2 reworded by #614
M2 (4 rows) #601 replaced the maiden walk with the clause-free trailing run; each reading re-verified by parse
S2 "BEGINS a name" now "OPENS THE STRING"
rules.md#A1 "hesitate too" (4 ledgers) wrong document — it is AGENTS.md's convention
decisions.md#S2 Accepted (test_pieces.py) wrong document — rules.md#S2's Accepted clause
decisions.md#S2 paraphrase (4 ledgers) now quotes the headline and names #544's second exception
A1 segmenter clause (_script_segment.py) one-word drift
C1 paired initials (5 ledgers), N3 (3), P2 (1) beyond the check's reach, found by the second review; #563 made C1's speaking word an unambiguous suffix
P2 (#424) and P5 (#425) landing paragraphs now say they describe the rule as it then stood, cite today's text separately, and point at the dated notes that record later moves

T3: measured, 王·Smith never divides (the dot has a Latin neighbour) and 王Smith reads the same — a mixed-script word W4 never reaches, so O5 decides; the old comment described a mechanism that isn't there. The M2 rule whose names #601 moved now names all four (three comma names to fix(#601), Jane Doe Jr. nee Smith Ma to fix(#601/#602)) in a dated note rather than rewriting its history. No issue strings changed.

AGENTS.md's ledger-citation sentence now names the check, what is quotable, its reach, and the measured dated count.

Reviewed twice: a four-agent first round, then a second round on the fix commit (scanner + prose), whose findings are f481c721.

Verification

🤖 Generated with Claude Code

@derek73 derek73 self-assigned this Oct 10, 2026
derek73 added a commit that referenced this pull request Oct 10, 2026
The first commit checked only the colon form's first quote, and the
review of #632 found the rest of the class: `rules.md#X -- "..."`,
`rules.md#X's Accepted clause ("...")`, `rules.md#S2 consumes "..."`
and a second quote chained by "and" all passed unread, 23 of them
stale. The scanner now:

- reads a quote after the ID in any of those shapes (_LEAD_RE), and
  every quote chained to it; a reference with no quote must still
  name a real rule, section, mechanism or decisions entry
- quotes against a rule's statement AND its Accepted: clauses (example
  and pointer lines excluded), a section letter's Background
  (`rules.md#H Background`), a decisions entry's text
- matches lowercase decisions keys, merges the `### differential-
  ledger, ...` arc headings onto their one key, rejects empty "[...]"
  fragments, reads `--` as the em dash and a nested ' as ", follows
  `#:` comment continuations and IDs wrapped at a hyphen
- skips this module's own illustrative comments
- records the retired excerpts as a negative control, and asserts the
  sweep reaches each shape (non-colon, chained, decisions, elided)

The colon form alone still defines a citing module for implemented:.

Fixed: P5 (the reserve's numeral clause and the join-after-the-run
wording, #614), P2 (S2's once-read run), M2 (#601 replaced the walk
with the clause-free trailing run; the 'Jane Doe Jr. nee Smith Ma'
paragraph gains a dated note that #601 moved it to fix(#601/#602)), S2
("OPENS THE STRING"), A1's segmenter clause, a decisions.md#S2 citation
of rules.md#S2's Accepted clause, and four ledgers crediting AGENTS.md's
"a reader would hesitate too" to rules.md#A1. Four ledgers' paraphrase
of decisions.md#S2 now quotes its headline and names #544's second
exception. With Accepted text quotable, the first commit's H5 and S2
rewrites go back to the original, more exact Accepted-clause quotes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
derek73 added a commit that referenced this pull request Oct 11, 2026
Scanner (tests/v2/test_doc_citations.py):
- a section's Background is read whole, not its first line; a wrapped
  no-boundary: marker's continuation lines are no longer quotable
- each citation records its lead, and the reach test asserts every
  lead shape is live ('s, parenthesis, dash, bare words) rather than
  any one of them; a new test pins what is quotable (an Accepted
  clause, a Background's wrapped line) and what is not (an example
  line, a no-boundary: continuation)
- the docstring states the limits: case folds, and a quote more than
  four words from its ID and chained to nothing is not checked -- the
  measured alternative, every quote in the paragraph, flags 139, nearly
  all names written in double quotes
- _RETIRED_EXCERPTS drops its P6 row (no version ever routed a
  decisions quote to a rule) and says which direction it guards

Ledgers: fourteen stale quotes the check does not reach, found by the
review -- C1's paired-initials sentences in all five (#563 made the
speaking word an UNAMBIGUOUS suffix), N3 in three, P2 in one. Dated
landing paragraphs for P2 (#424) and P5 (#425) now say they describe
the rule as it then stood instead of quoting today's text as their
reason; the M2 rule's opener names all four names #601 moved, not one.

AGENTS.md: the count is measured, not summed from commit messages --
27 stale quotes and 5 quoteless colon citations, the current check
over e4b653a -- and the sentence states the four-word reach.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
derek73 and others added 4 commits October 10, 2026 20:32
test_citations_are_verbatim_excerpts swept only *.py, so the
`# rules.md#X: "..."` comments in tools/differential/expected_since_*.toml
-- which AGENTS.md puts under the same excerpt discipline -- were
checked by nothing; #631's review found five stale by hand. The sweep
now reads the ledgers too. They are not "citing modules":
test_implemented_matches_citing_modules still counts .py files only.

Two scanner corrections the ledgers were the first input to reach:
- a decisions.md#X citation was looked up in rules.md's statements
  (the regex accepted the doc name and dropped it), so a decisions.md#P6
  quote was checked against rules.md#P6. It now must name a real
  decisions entry, and a quote OPENING it must be verbatim in that
  entry; a paraphrase pointer stays legal, the entry being a record
  rather than a statement.
- an excerpt may elide with "[...]"; each fragment must be verbatim
  and in order.

The fifteen failures (five citations, copied across ledgers):
- H5 and S2 (1.4.0) quoted Accepted clauses; now quote statements that
  carry the same claim.
- C1's "The part is read as its words stand" is now a longer sentence.
- 2.2.0/2.3.0 S2 paraphrased with no quote; now quotes the sentence.
- T3 paraphrased a mechanism that is not there: '王·Smith' is one word
  (the dot has a Latin neighbour) and '王Smith' reads the same; it now
  cites T3 and W4 for why no script order applies.

Adds a reach assertion so a glob matching nothing cannot pass the
excerpt test vacuously, and an elision-order test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first commit checked only the colon form's first quote, and the
review of #632 found the rest of the class: `rules.md#X -- "..."`,
`rules.md#X's Accepted clause ("...")`, `rules.md#S2 consumes "..."`
and a second quote chained by "and" all passed unread, 23 of them
stale. The scanner now:

- reads a quote after the ID in any of those shapes (_LEAD_RE), and
  every quote chained to it; a reference with no quote must still
  name a real rule, section, mechanism or decisions entry
- quotes against a rule's statement AND its Accepted: clauses (example
  and pointer lines excluded), a section letter's Background
  (`rules.md#H Background`), a decisions entry's text
- matches lowercase decisions keys, merges the `### differential-
  ledger, ...` arc headings onto their one key, rejects empty "[...]"
  fragments, reads `--` as the em dash and a nested ' as ", follows
  `#:` comment continuations and IDs wrapped at a hyphen
- skips this module's own illustrative comments
- records the retired excerpts as a negative control, and asserts the
  sweep reaches each shape (non-colon, chained, decisions, elided)

The colon form alone still defines a citing module for implemented:.

Fixed: P5 (the reserve's numeral clause and the join-after-the-run
wording, #614), P2 (S2's once-read run), M2 (#601 replaced the walk
with the clause-free trailing run; the 'Jane Doe Jr. nee Smith Ma'
paragraph gains a dated note that #601 moved it to fix(#601/#602)), S2
("OPENS THE STRING"), A1's segmenter clause, a decisions.md#S2 citation
of rules.md#S2's Accepted clause, and four ledgers crediting AGENTS.md's
"a reader would hesitate too" to rules.md#A1. Four ledgers' paraphrase
of decisions.md#S2 now quotes its headline and names #544's second
exception. With Accepted text quotable, the first commit's H5 and S2
rewrites go back to the original, more exact Accepted-clause quotes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scanner (tests/v2/test_doc_citations.py):
- a section's Background is read whole, not its first line; a wrapped
  no-boundary: marker's continuation lines are no longer quotable
- each citation records its lead, and the reach test asserts every
  lead shape is live ('s, parenthesis, dash, bare words) rather than
  any one of them; a new test pins what is quotable (an Accepted
  clause, a Background's wrapped line) and what is not (an example
  line, a no-boundary: continuation)
- the docstring states the limits: case folds, and a quote more than
  four words from its ID and chained to nothing is not checked -- the
  measured alternative, every quote in the paragraph, flags 139, nearly
  all names written in double quotes
- _RETIRED_EXCERPTS drops its P6 row (no version ever routed a
  decisions quote to a rule) and says which direction it guards

Ledgers: fourteen stale quotes the check does not reach, found by the
review -- C1's paired-initials sentences in all five (#563 made the
speaking word an UNAMBIGUOUS suffix), N3 in three, P2 in one. Dated
landing paragraphs for P2 (#424) and P5 (#425) now say they describe
the rule as it then stood instead of quoting today's text as their
reason; the M2 rule's opener names all four names #601 moved, not one.

AGENTS.md: the count is measured, not summed from commit messages --
27 stale quotes and 5 quoteless colon citations, the current check
over e4b653a -- and the sentence states the four-word reach.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e4b653a lived on #631's branch, which the squash merge leaves to be
deleted; 8e6e524, master after #631, has the identical tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derek73
derek73 force-pushed the fix/ledger-citation-excerpts branch from f481c72 to c200fff Compare October 11, 2026 03:34
@derek73
derek73 changed the base branch from fix/issue-627-lone-particle-count to master October 11, 2026 03:34
@derek73 derek73 added this to the 2.4 milestone Oct 11, 2026
@codecov

codecov Bot commented Oct 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.01%. Comparing base (6185e6f) to head (c200fff).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #632   +/-   ##
=======================================
  Coverage   99.01%   99.01%           
=======================================
  Files          46       46           
  Lines        4487     4487           
=======================================
  Hits         4443     4443           
  Misses         44       44           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@derek73
derek73 merged commit 505d6de into master Oct 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant