Skip to content

Commit 33da23a

Browse files
derek73claude
andcommitted
Fix #626's review findings: the sixth emitter, P1's fold, a wider guard
The comma report on the word after a family comma hardcoded "read as the given name", and with nothing before the comma H1 moves that word to the family behind a trailing title (`, Ma Dr.`). It now defers its field to assemble like the other five, so its given-side wording reads "a given name". P1's family-first fold is a third rule moving reported words (`de Kim Ma` under FAMILY_FIRST); the comments and docs that named only H1 and P6 now name all three, and "H1's swap" is H1's move. Assemble withdraws a field_tail report in an emptied name instead of shipping half a sentence, and FIELD_READING is asserted complete over Role. The sweep's phrase list covers the settled phrasings too (the suffix run's, the numeral fork's, H4's "the name", P6's), states that locale rows are not checked, and a second test records how many claims it checks (1151) so it cannot go vacuous. Rows `, Ma Dr.` (fix(#316)) and `de Kim Ma` under FAMILY_FIRST (fix(#289)), both bisected; the `Kim Min Do` note no longer claims both family-first orders. Unit tests for assemble's join; two test_assign names now say "field". decisions.md's entry carries a recompute recipe and re-measured figures: 2,725 of 89,092 claims wrong on master over 53,517 texts (comma shapes with an empty head and a trailing title added), 0 on the tree; control 11 failures on master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 2a52a55 commit 33da23a

10 files changed

Lines changed: 180 additions & 52 deletions

File tree

‎AGENTS.md‎

Lines changed: 2 additions & 2 deletions
Large diffs are not rendered by default.

‎docs/design/decisions.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,10 @@ Entry conventions:
1818
- 2026-09-20 #397 (second review) — A REPORT MUST NOT CONTRADICT THE READING BESIDE IT, and the shape could not arise before this cycle. `classify`'s `conjunction-or-initial` fork offers exactly two readings and its detail says which it took ("it is read as an initial"). Where the rest of the parse then roles that letter SUFFIX, the fork resolved to NEITHER branch: `JOHN QUINCY SMITH I` carried the connective-or-initial beside a `suffix-or-name` saying the same token reads as a generational suffix, and `john smith i`, `HENRY I`, `SMITH, JOHN I`, `JOSEP CAROD I ROVIRA III` and `abdul e i` were the same. `i` is the first word that is both a MARKED connective and suffix vocabulary, so no parse before #397 could reach it — which is why the rule is written down now rather than having been.
1919
DECIDED: the report is WITHDRAWN where the role is SUFFIX or TITLE, in `_pipeline/_assemble.py`, the one place that sees final roles and the ambiguity list together — and beside a withdrawal that was already there, for a report whose referents did not survive assembly. That is the narrow reading of mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE rather than an exception to it: classify still owns the fork and still takes it where it is taken; what goes is a report whose subject the parse went on to read as something else. Emitting later was the alternative and was rejected as more invasive for the same answer — the fork's inputs (the case class, the own-words span) live in classify and moving the emitter moves them.
2020
MEASURED: over the review's 43,925-name grid under eight configurations, 9,806 rows on 1,637 distinct names lose exactly one `conjunction-or-initial` and nothing else — no report of any other kind moves and none is gained. On the deduped corpus-union-cases population the only name affected is `JOHN QUINCY SMITH I`, a row this round added, so the ledger's three-name `fix(#397) the Catalan link reports a connective-or-initial in a one-case name` rule is untouched: all three of its names role the letter MIDDLE and all three still report. The invariant is `tests/v2/test_properties.py::test_no_report_contradicts_the_reading_beside_it`, which states it twice — no connective-or-initial on a token roled SUFFIX or TITLE, and no token named by two reports whose details assert different readings — and fails on 8,204 rows at `dc3bdf9c`, 0 here.
21-
- 2026-10-08 #626 — A REPORT NAMES THE FIELD ITS WORD LANDS IN. Five of assign's reports worded the field from the role the word held at assign (`read as a given name`), and two rules still move a word after that: H1 moves a lone unit behind a title to the family, and P6's attachment takes a trailing name word into the family at the end of a name read family-first. So `Attorney General of Minnesota` reported 'General of Minnesota' "read as a given name by convention" with the unit in the family (since 2.3.0, #491's join clause), and `Kim Min Do` under FAMILY_FIRST told the caller 'Do' was "read as a middle name" with 'Do' in the family (this cycle, since #289 made the Title-case 'Do' a name word). #614's second review had fixed one instance of the class (`Freiherr von Berg Ma`) by moving its report to the stage that knows the outcome.
22-
DECIDED: the DECISION stays at its site and the WORDING of the field moves to assemble, the one place that knows final roles (the same reasoning as the 2026-09-20 withdrawal above). An emitter whose detail names a field writes the sentence up to the field and hands the rest as `PendingAmbiguity.field_tail`; assemble joins them around `FIELD_READING` of the first referent's final role. All five such emitters in assign use it, including the three that were right on every measured input (H4's all-suffix name, O5's lone word, the leading ambiguous particle), so no later rule can make them wrong. A placeholder in the detail was declined: token text can carry any characters, and a name containing the placeholder would corrupt its own report. Reports worded at a site whose field is fixed (a comma's credential reading, P6's own attachment report) keep their literal wording.
23-
MEASURED 2026-10-08 on master (26cdb891), each report's field phrase compared against its tokens' final roles: over a generated grid of 51,835 names (the corpus names plus heads × trailing words, comma shapes and random runs) under all three orders, 2,689 of 66,208 field claims named the wrong field, in two emitters, title-or-name's join branch (1,575) and the trailing peel's picks (1,114); on the tree, 0. Fingerprint against master, over the differential corpora, the case-table names and a generated comma grid (13,090 names) under six policies (the three orders, `lenient_comma_suffixes=False`, a `/` suffix delimiter and `CapsSuffixes.OFF`): no role, kind, initials or rendering moves; 40 details change, every one `read as a given name` → `read as a family name` on title-or-name, ten names under each of the four given-first policies. Pinned by `test_a_report_names_the_field_its_word_lands_in` in tests/v2/test_cases.py over every case row under all three orders, whose recorded control is 10 failures on master.
21+
- 2026-10-08 #626 — A REPORT NAMES THE FIELD ITS WORD LANDS IN. Six of assign's reports worded the field from the role the word held at assign (`read as a given name`), and three rules in post_rules still move a word after that: H1 moves the name behind a title to the family, P1's family-first fold takes the particle run and one unit and lays the rest out by the order, and P6's attachment takes a trailing name word into the family at the end of a name read family-first. So `Attorney General of Minnesota` reported 'General of Minnesota' "read as a given name by convention" with the unit in the family (since 2.3.0, #491's join clause); `Kim Min Do` under FAMILY_FIRST told the caller 'Do' was "read as a middle name" with 'Do' in the family, and `de Kim Ma` under FAMILY_FIRST said 'Ma' was a middle name with 'Ma' the given name (both this cycle, since #289 made the Title-case word a name word); and `, Ma Dr.` said 'Ma' after the comma was "read as the given name" with 'Ma' in the family (this cycle, since #316's trailing title walk left the comma's word to H1). #614's second review had fixed one instance of the class (`Freiherr von Berg Ma`) by moving its report to the stage that knows the outcome.
22+
DECIDED: the DECISION stays at its site and the WORDING of the field moves to assemble, the one place that knows final roles (the same reasoning as the 2026-09-20 withdrawal above). An emitter whose detail names a field a later rule could change writes the sentence up to the field and hands the rest as `PendingAmbiguity.field_tail` (`""` where the field ends the sentence); assemble joins them around `FIELD_READING` of the first referent's final role. All six such emitters in assign use it -- H4's all-suffix name, title-or-name's join clause, O5's lone word, the trailing peel's picks, the leading ambiguous particle and the comma report on the word after a family comma -- including those right on every measured input, so no later rule can make them wrong. A report in a name emptied for want of content is withdrawn rather than shipped half-worded (no emitter reaches that today). A placeholder in the detail was declined: token text can carry any characters, and a name containing the placeholder would corrupt its own report. Reports worded where the field is settled keep their literal wording: a comma's credential reading, the suffix run's and the numeral fork's, H4's "the name" and P6's own "joins the family". The comma report's given-side wording moves from "the given name" to "a given name" with the rest.
23+
MISSED, then caught in review: the first draft converted five emitters and its grid had no empty-head comma with a trailing title, the shape AGENTS.md's comma sweep already asks for; four of the review's five readers found the sixth emitter on `, Ma Dr.`. The draft also named only H1 and P6 as movers; P1's fold was found by stepping the stages.
24+
MEASURED 2026-10-08, each report's field phrase compared against its tokens' final roles. RECIPE: texts are the 1,536 names of `tools/differential/corpus*.jsonl` and 347 `tests/v2/cases.py` texts (the rows that existed at #614 and are not corpus names; the whole table moves the totals by a few claims), plus every head in {'John', 'John Smith', 'John Q. Smith', 'Mary Ann Smith', 'John van Smith', 'Juan de la Vega', 'John van', 'anh van', 'Jan Freiherr von Berg', 'Freiherr von Berg', 'Freiherr St', 'St', 'Dr. St', 'Freiherr', 'Dr.', 'Mr. and Mrs.', 'Attorney General of', 'abdul salam', 'Juan y Garcia', 'Jane Doe nee van der Berg', 'Dr. John Smith', 'JOHN SMITH', 'john smith', 'Smith', 'de Mesnil', 'Dr. de Mesnil', 'Kim Min', 'Kim Min Jun', 'Ortega y Gasset', 'van', 'Do', ''} followed by one or two words drawn from {PhD, MA, Ma, DO, Do, do, vd, mc, Jr, III, V, V., VI, X.Y.Z., Ph. D., Esq., Prof., Sir, Ed, van, and, Jones, ba, John, Smith, Minnesota, de, St, Freiherr}, `{head}, {word}`, `Smith, {head} {word}`, `{word}, {head}`, `, {w1} {w2}` and `(Bob), {w1} {w2}` for every pair of those words, and 20,000 random head-plus-three-or-four-word draws (seed 626) -- 53,517 texts; each parsed under the three orders; a claim is a match of `(reads? as|joins) (<phrase>)` for the phrases `_FIELD_CLAIMS` in tests/v2/test_cases.py lists plus "the given name", and it is wrong when its tokens' roles are not among the fields the phrase names. On master (26cdb891) 2,725 of 89,092 claims were wrong, in three emitters: title-or-name's join clause (1,575), the trailing peel's picks (1,114) and the comma report (36); on the tree, 0. The comparator is master, which words the field at assign; the tree words it at assemble. Fingerprint against master over the differential corpora, master's case-table names and a seeded comma grid (13,090 names) under six policies (the three orders, `lenient_comma_suffixes=False`, a `/` suffix delimiter and `CapsSuffixes.OFF`): no role, kind, initials or rendering moves; the details that change are 6,395 comma reports on 1,086 names reworded from "the given name" to "a given name", 85 on 15 names now naming the family they land in, and 40 title-or-name join reports on 10 names now naming the family. Pinned by `test_a_report_names_the_field_its_word_lands_in` in tests/v2/test_cases.py, over every case row under all three orders except the locale rows (a row with its own policy only as declared), whose recorded control is 11 failures on master, and by `test_the_field_sweep_sees_the_claims_it_checks`, which records how many claims that sweep checks.
2425

2526
### A2 — the empty name
2627

‎nameparser/_pipeline/_assemble.py‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
"""Not a stage: converts the final ParseState into a public ParsedName.
22
33
Consumes: tokens (all roles set), dropped, ambiguities (by index;
4-
a report naming a field is worded from its word's final role, #626).
4+
one carrying ``field_tail`` is worded from its word's final role,
5+
#626).
56
Produces: a validated ParsedName -- the constructor re-checks every
67
invariant (span order/bounds, ambiguity subset), so a pipeline bug
78
that would produce an invalid result dies HERE, not in a renderer
@@ -114,7 +115,13 @@ def assemble(state: ParseState) -> ParsedName:
114115
for t in materialized)):
115116
continue
116117
detail = pending.detail
117-
if pending.field_tail is not None and materialized:
118+
if pending.field_tail is not None:
119+
# A report naming a field, in a name emptied above (no
120+
# emitter of one reaches that today: every referent is a
121+
# name word): no field is left to name, so it goes rather
122+
# than ship the sentence's first half.
123+
if not materialized:
124+
continue
118125
detail = (f"{detail}{FIELD_READING[materialized[0].role]}"
119126
f"{pending.field_tail}")
120127
ambiguities.append(Ambiguity(pending.kind, detail, materialized))

‎nameparser/_pipeline/_assign.py‎

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -363,9 +363,10 @@ def _assign_main(seg_idx: int, state: ParseState,
363363
#
364364
# Every bare ambiguous acronym the FINAL peel had to resolve is one
365365
# coin-flip each, in either direction, so the report collects
366-
# rather than overwrites. Deferred to after assignment because the
367-
# wording reads the role back, and which role "not peeled" means
368-
# depends on name_order. (The roman-numeral fork needs no such
366+
# rather than overwrites. Deferred to after assignment because what
367+
# a pick is declined as depends on the role it took, and which role
368+
# "not peeled" means depends on name_order; the field itself is
369+
# worded by assemble (#626). (The roman-numeral fork needs no such
369370
# deferral and is reported here.)
370371
#
371372
# Read once in group, after P3's connective joins and ahead of the
@@ -456,8 +457,9 @@ def _assign_main(seg_idx: int, state: ParseState,
456457
# `suffix-or-name`" (history: decisions.md#H4) -- only the
457458
# word made into a name reports, and no name piece survived
458459
# the peel, so the carve-out above made the first post-nominal
459-
# the name. Its field is worded by assemble, as every report
460-
# naming one is: see the particle emitter below.
460+
# the name. Its field is worded by assemble, as every assign
461+
# report naming a field a later rule could change is: see the
462+
# particle emitter below.
461463
if peeled.names == 0 and field_undecided:
462464
text = " ".join(tokens[i].text for i in head)
463465
ambiguities.append(PendingAmbiguity(
@@ -551,8 +553,9 @@ def _assign_main(seg_idx: int, state: ParseState,
551553
# the loops above read `order`, which is _effective_order's
552554
# answer and not necessarily name_order's (a script_orders
553555
# entry overrides it), and a later rule may still move the
554-
# word -- P6's attachment, H1's swap -- as it did to every
555-
# report worded here before #626 ('Kim Min Do' under
556+
# word -- H1's move behind a title, P1's family-first fold,
557+
# P6's attachment -- as it did to three of the reports
558+
# worded at assign before #626 ('Kim Min Do' under
556559
# FAMILY_FIRST was told 'Do' was a middle name).
557560
token = tokens[head[0]]
558561
ambiguities.append(PendingAmbiguity(
@@ -853,11 +856,13 @@ def assign(state: ParseState) -> ParseState:
853856
if state.pieces[1] and len(state.pieces[1][0]) == 1:
854857
i = state.pieces[1][0][0]
855858
if not tokens[i].tags.isdisjoint(_AMBIGUOUS_CREDENTIAL_TAGS):
859+
# the field is assemble's to word: with nothing before
860+
# the comma and a title after the word, H1 moves it to
861+
# the family (', Ma Dr.', #626's review)
856862
ambiguities.append(PendingAmbiguity(
857863
AmbiguityKind.SUFFIX_OR_NAME,
858864
f"{tokens[i].text!r} after the comma is also an "
859-
f"ordinary name word; read as the given name",
860-
(i,)))
865+
f"ordinary name word; read as ", (i,), field_tail=""))
861866
# Segment 1 is read before segment 0's wholly-family pass. It
862867
# consumes piece tags and text only -- nothing segment 0's
863868
# read writes. (A title standing after the comma behind a whole

‎nameparser/_pipeline/_state.py‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -132,11 +132,13 @@ class PendingAmbiguity:
132132
after tokenize set ``indices`` directly and leave ``origin`` None.
133133
134134
``field_tail`` is for a report that names the field its word was
135-
read into. A later stage may still move the word (P6's attachment,
136-
H1's swap), so the emitter writes ``detail`` up to the field and
137-
the rest of its sentence here, and assemble joins them around the
138-
FIELD_READING of the first referent's final role -- the one place
139-
that knows it (#626).
135+
read into. post_rules may still move the word (H1's move behind a
136+
title, P1's family-first fold, P6's attachment), so the emitter
137+
writes ``detail`` up to the field and the rest of its sentence
138+
here -- ``""`` where the field ends the sentence, which is why
139+
assemble tests ``is not None`` -- and assemble joins them around
140+
the FIELD_READING of the first referent's final role, the one
141+
place that knows it (#626). Until then ``detail`` is a fragment.
140142
"""
141143

142144
kind: AmbiguityKind
@@ -152,6 +154,9 @@ class PendingAmbiguity:
152154
Role.MIDDLE: "a middle name", Role.FAMILY: "a family name",
153155
Role.SUFFIX: "a suffix", Role.NICKNAME: "a nickname",
154156
Role.MAIDEN: "a maiden name"})
157+
# every role a referent can hold, or a new one is a KeyError at the
158+
# first parse whose report lands in it
159+
assert FIELD_READING.keys() == set(Role)
155160

156161

157162
@dataclass(frozen=True, slots=True)

‎tests/v2/cases.py‎

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8233,11 +8233,29 @@ def _check_cjk_shape_purity(self) -> None:
82338233
"Kim Min Do",
82348234
{"given": "Kim", "middle": "Min", "family": "Do"},
82358235
ambiguities=("suffix-or-name",), classification="fix(#289)",
8236-
notes="#626: under both family-first orders the order sweep "
8237-
"reads P6 attaching the Title-case 'Do' to the family "
8238-
"after assign read it as a name word ('Do Kim' under "
8239-
"FAMILY_FIRST); the report was worded at assign and "
8240-
"said 'middle'. 1.4.0 read suffix 'Do'"),
8236+
notes="#626: under FAMILY_FIRST the order sweep reads P6 "
8237+
"attaching the Title-case 'Do' to the family ('Do Kim') "
8238+
"after assign read it as a middle name; the report was "
8239+
"worded at assign and said 'middle'. Under "
8240+
"FAMILY_FIRST_GIVEN_LAST nothing moves it: given 'Do'. "
8241+
"1.4.0 read suffix 'Do'"),
8242+
Case("a_pick_the_family_first_fold_moves_is_reported_where_it_lands",
8243+
"de Kim Ma", {"family": "de Kim", "given": "Ma"},
8244+
ambiguities=("suffix-or-name",), classification="fix(#289)",
8245+
policy=Policy(name_order=FAMILY_FIRST),
8246+
notes="#626's review: P1's family-first fold takes the particle "
8247+
"run and one name unit, and lays the rest out by the "
8248+
"order, so 'Ma', which assign read as a middle name, "
8249+
"lands in the given name -- a third rule moving a "
8250+
"reported word after assign. 2.3.0 read suffix 'Ma'"),
8251+
Case("a_comma_report_h1_moves_is_reported_in_the_family",
8252+
", Ma Dr.", {"title": "Dr.", "family": "Ma"},
8253+
ambiguities=("suffix-or-name",), classification="fix(#316)",
8254+
notes="#626's review: with nothing before the comma, the comma "
8255+
"report on 'Ma' was worded 'the given name' at assign, "
8256+
"and H1 then moved it to the family behind the trailing "
8257+
"title. The empty-head comma shape AGENTS.md's sweep "
8258+
"paragraph asks for; 1.4.0 read first 'Ma', suffix 'Dr.'"),
82418259
Case("lone_joined_unit_carrying_collision_set_title_vocabulary",
82428260
"Smith and King", {"given": "Smith and King"},
82438261
ambiguities=("title-or-name",),

0 commit comments

Comments
 (0)