Skip to content

Commit 505d6de

Browse files
derek73claude
andauthored
Check rule citations in every shape, ledgers included; fix the stale ones (#632)
* Check ledger rule citations as verbatim excerpts, fix the stale ones test_citations_are_verbatim_excerpts swept only *.py, so the `# rules.md#X: "..."` comments in tools/differential/expected_since_*.toml -- which AGENTS.md puts under the same excerpt discipline -- were checked by nothing; #631's review found five stale by hand. The sweep now reads the ledgers too. They are not "citing modules": test_implemented_matches_citing_modules still counts .py files only. Two scanner corrections the ledgers were the first input to reach: - a decisions.md#X citation was looked up in rules.md's statements (the regex accepted the doc name and dropped it), so a decisions.md#P6 quote was checked against rules.md#P6. It now must name a real decisions entry, and a quote OPENING it must be verbatim in that entry; a paraphrase pointer stays legal, the entry being a record rather than a statement. - an excerpt may elide with "[...]"; each fragment must be verbatim and in order. The fifteen failures (five citations, copied across ledgers): - H5 and S2 (1.4.0) quoted Accepted clauses; now quote statements that carry the same claim. - C1's "The part is read as its words stand" is now a longer sentence. - 2.2.0/2.3.0 S2 paraphrased with no quote; now quotes the sentence. - T3 paraphrased a mechanism that is not there: '王·Smith' is one word (the dot has a Latin neighbour) and '王Smith' reads the same; it now cites T3 and W4 for why no script order applies. Adds a reach assertion so a glob matching nothing cannot pass the excerpt test vacuously, and an elision-order test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Read every citation shape the tree writes; fix what that exposes The first commit checked only the colon form's first quote, and the review of #632 found the rest of the class: `rules.md#X -- "..."`, `rules.md#X's Accepted clause ("...")`, `rules.md#S2 consumes "..."` and a second quote chained by "and" all passed unread, 23 of them stale. The scanner now: - reads a quote after the ID in any of those shapes (_LEAD_RE), and every quote chained to it; a reference with no quote must still name a real rule, section, mechanism or decisions entry - quotes against a rule's statement AND its Accepted: clauses (example and pointer lines excluded), a section letter's Background (`rules.md#H Background`), a decisions entry's text - matches lowercase decisions keys, merges the `### differential- ledger, ...` arc headings onto their one key, rejects empty "[...]" fragments, reads `--` as the em dash and a nested ' as ", follows `#:` comment continuations and IDs wrapped at a hyphen - skips this module's own illustrative comments - records the retired excerpts as a negative control, and asserts the sweep reaches each shape (non-colon, chained, decisions, elided) The colon form alone still defines a citing module for implemented:. Fixed: P5 (the reserve's numeral clause and the join-after-the-run wording, #614), P2 (S2's once-read run), M2 (#601 replaced the walk with the clause-free trailing run; the 'Jane Doe Jr. nee Smith Ma' paragraph gains a dated note that #601 moved it to fix(#601/#602)), S2 ("OPENS THE STRING"), A1's segmenter clause, a decisions.md#S2 citation of rules.md#S2's Accepted clause, and four ledgers crediting AGENTS.md's "a reader would hesitate too" to rules.md#A1. Four ledgers' paraphrase of decisions.md#S2 now quotes its headline and names #544's second exception. With Accepted text quotable, the first commit's H5 and S2 rewrites go back to the original, more exact Accepted-clause quotes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix #632's second-round review: reach, controls, stale prose Scanner (tests/v2/test_doc_citations.py): - a section's Background is read whole, not its first line; a wrapped no-boundary: marker's continuation lines are no longer quotable - each citation records its lead, and the reach test asserts every lead shape is live ('s, parenthesis, dash, bare words) rather than any one of them; a new test pins what is quotable (an Accepted clause, a Background's wrapped line) and what is not (an example line, a no-boundary: continuation) - the docstring states the limits: case folds, and a quote more than four words from its ID and chained to nothing is not checked -- the measured alternative, every quote in the paragraph, flags 139, nearly all names written in double quotes - _RETIRED_EXCERPTS drops its P6 row (no version ever routed a decisions quote to a rule) and says which direction it guards Ledgers: fourteen stale quotes the check does not reach, found by the review -- C1's paired-initials sentences in all five (#563 made the speaking word an UNAMBIGUOUS suffix), N3 in three, P2 in one. Dated landing paragraphs for P2 (#424) and P5 (#425) now say they describe the rule as it then stood instead of quoting today's text as their reason; the M2 rule's opener names all four names #601 moved, not one. AGENTS.md: the count is measured, not summed from commit messages -- 27 stale quotes and 5 quoteless colon citations, the current check over e4b653a -- and the sentence states the four-word reach. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Name the count's comparator by its master commit e4b653a lived on #631's branch, which the squash merge leaves to be deleted; 8e6e524, master after #631, has the identical tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 6185e6f commit 505d6de

9 files changed

Lines changed: 623 additions & 349 deletions

File tree

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Three committed contributor docs carry the parser's normative rules and their re
2828

2929
**Counting claims.** A bare count in prose is either an assertion or a liability, keyed by who observes its staleness: asserted counts (a test holds the number) fail CI at change time — the useful kind; dated snapshots ("51 sites at spec time") cannot go stale; standing present-tense prose counts are the forbidden class — promote to an assertion, add a date, or state the invariant and let a test count. After changing how many times something runs, sweep for counts, not for the thing's name.
3030

31-
**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the differential gate's classified summary, be verified against rules.md examples, or -- for a view the gate cannot see -- carry a recompute recipe stored with the design entry the bullet cites; never write one from memory. The classified summary covers the CONTRACT tier plus whatever radar diffs a rule classifies; a radar corpus's unmatched diffs are listed under UNCLASSIFIED (radar) and are not in it, so a claim quantified from the summary alone is silent about them. The gate compares the seven role fields, `_ambiguities` and -- since #484 -- `initials()`, under the `_initials` pseudo-field; so an initials-only change DOES show in the classified summary. But that pseudo-field sees only the names whose initials moved WHILE EVERY ROLE AND EVERY AMBIGUITY KIND STAYED PUT -- main()'s roles-identical guard keeps it out of any diff a role or a report is already in -- so a count taken from it is a FLOOR on initials movement, not the population: measured 2026-09-02 at 2.1.0 → tree on the v2 surface, 83 of the 1120 compared ENTRIES (1116 distinct names; seven entries carry a declared order rather than the default, and three strings are compared under more than one) changed their `initials()` string and only 28 were visible under `_initials`, the other 55 having moved a role as well. A bullet about how many names' initials changed still needs the recompute recipe. `capitalized()` and any other render view stay invisible to the gate (decisions.md#R4, #R3), and for those the first two sources still cannot reach a claim: a gate run is byte-identical across the change, and an example line witnesses an output without counting anything. A recipe names the corpus files, the policy sweep, and -- the part that is easy to omit and fatal -- THE COMPARATOR, which must be something the shipped tree is not: #408's first recipe said to compare `initials()` against a folded-first partition, which is what `initials()` now IS, so it reproduced 0 where the bullet claimed 660 and was the only stated provenance for the number. Run the recipe as written before shipping the bullet. Cross-version numbers (a released wheel, the pre-change tree) are dated snapshots under Counting claims, since nothing in the repository re-runs them. Per-rule ledger toml comments asserting PARSER behavior cite rule IDs under the excerpt discipline; free prose is for ledger mechanics only (owned by tools/differential/README.md).
31+
**Release-log claims.** Quantified or universal behavior claims in release bullets must come from the differential gate's classified summary, be verified against rules.md examples, or -- for a view the gate cannot see -- carry a recompute recipe stored with the design entry the bullet cites; never write one from memory. The classified summary covers the CONTRACT tier plus whatever radar diffs a rule classifies; a radar corpus's unmatched diffs are listed under UNCLASSIFIED (radar) and are not in it, so a claim quantified from the summary alone is silent about them. The gate compares the seven role fields, `_ambiguities` and -- since #484 -- `initials()`, under the `_initials` pseudo-field; so an initials-only change DOES show in the classified summary. But that pseudo-field sees only the names whose initials moved WHILE EVERY ROLE AND EVERY AMBIGUITY KIND STAYED PUT -- main()'s roles-identical guard keeps it out of any diff a role or a report is already in -- so a count taken from it is a FLOOR on initials movement, not the population: measured 2026-09-02 at 2.1.0 → tree on the v2 surface, 83 of the 1120 compared ENTRIES (1116 distinct names; seven entries carry a declared order rather than the default, and three strings are compared under more than one) changed their `initials()` string and only 28 were visible under `_initials`, the other 55 having moved a role as well. A bullet about how many names' initials changed still needs the recompute recipe. `capitalized()` and any other render view stay invisible to the gate (decisions.md#R4, #R3), and for those the first two sources still cannot reach a claim: a gate run is byte-identical across the change, and an example line witnesses an output without counting anything. A recipe names the corpus files, the policy sweep, and -- the part that is easy to omit and fatal -- THE COMPARATOR, which must be something the shipped tree is not: #408's first recipe said to compare `initials()` against a folded-first partition, which is what `initials()` now IS, so it reproduced 0 where the bullet claimed 660 and was the only stated provenance for the number. Run the recipe as written before shipping the bullet. Cross-version numbers (a released wheel, the pre-change tree) are dated snapshots under Counting claims, since nothing in the repository re-runs them. Per-rule ledger toml comments asserting PARSER behavior cite rule IDs under the excerpt discipline, and since 2026-10-10 tests/v2/test_doc_citations.py checks them as it checks code. A ledger comment is copied from ledger to ledger and reworded by nothing: measured 2026-10-10 by running the check as it now stands over the tree before it (8e6e524a, master after #631), 27 quotes were stale and 5 colon citations quoted nothing, beyond the five #631's review found by hand -- most in a shape the colon-only check could not read (`rules.md#X -- "..."`, `rules.md#X's Accepted clause ("...")`, a second quote chained by "and"). A rule's statement and its `Accepted:` clauses are quotable, its example lines are not; `rules.md#H` names the H section, whose Background is quotable; a decisions.md citation may paraphrase, but what it quotes must be verbatim; and a reference with no quote must still name something that exists. The check's reach is a quote within four words of the ID or chained to one that is: a quote further off reads exactly like a name written in double quotes, so it is NOT checked, and the second review of #632 found 14 such quotes stale by hand. Keep a quote beside its ID. Quotes of AGENTS.md itself are not checked at all. Free prose is for ledger mechanics only (owned by tools/differential/README.md).
3232

3333
**Writing the user docs (docs/*.rst) has its own AGENTS.md too.** `docs/AGENTS.md` carries the style the docs are written in — tables as indexes, subheadings per task, measured claims, recipe doctests — distilled from PR #588. It loads the same way as the docs/design/ one below; if your tool does not do nested discovery, read it before editing a `.rst` file under docs/.
3434

‎nameparser/_pipeline/_script_segment.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -715,9 +715,9 @@ def _split_surname_site(state: ParseState) -> ParseState:
715715
for j in state.segments[0]):
716716
return state
717717
# No try/except around the call: rules.md#A1's Accepted clause
718-
# ("a user-supplied segmenter's own error propagates"). The two
719-
# checks below are that same doctrine, curated,
720-
# and they are where the line this module draws is easiest to state:
718+
# ("a user-supplied segmenter's own error, which propagates"). The
719+
# two checks below are that same doctrine, curated, and they are
720+
# where the line this module draws is easiest to state:
721721
# a PROTOCOL VIOLATION BY THE SEGMENTER AUTHOR RAISES, while an
722722
# ADAPTER'S DEFENSE AGAINST ITS LIBRARY DECLINES. Both checks here
723723
# are the first kind -- a wrong answer TYPE and an answer indexing

‎tests/v2/pipeline/test_pieces.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -426,7 +426,7 @@ def test_the_walks_own_leading_piece_never_anchors_what_follows_it(
426426

427427
def test_a_reserve_kept_leading_piece_beside_a_genuine_family_loss(
428428
) -> None:
429-
# decisions.md#S2's Accepted boundary ("an unambiguous suffix is
429+
# rules.md#S2's Accepted clause ("an unambiguous suffix is
430430
# consumed even when that leaves no family name at all", 'Smith
431431
# Jr.' -> family='') applies just the same when the LEADING piece
432432
# is itself listed suffix vocabulary rather than an ordinary name:

0 commit comments

Comments
 (0)