Skip to content

Commit bc24386

Browse files
derek73claude
andauthored
Word a report's field from the word's final role (#626) (#628)
* Word a report's field from the word's final role (#626) Five of assign's reports named the field their word was read into from the role it held at assign, and two later rules still move a word: H1 takes a lone unit behind a title into the family, and P6's attachment takes a trailing name word into the family at the end of a name read family-first. So `Attorney General of Minnesota` said 'General of Minnesota' was "read as a given name" (since 2.3.0), and `Kim Min Do` under FAMILY_FIRST said 'Do' was "read as a middle name", each with the word in the family. The decision stays at its site; the field's wording moves to assemble, which alone knows final roles. An emitter naming a field writes its sentence up to the field and passes the rest as PendingAmbiguity.field_tail; assemble joins them around FIELD_READING of the first referent's final role. All five emitters use it. Measured on master over 51,835 names under all three orders: 2,689 of 66,208 field claims named the wrong field; on the tree, 0. No role, kind, initials or rendering moves on the fingerprint; 40 details change, all title-or-name 'given' -> 'family'. Guarded by test_a_report_names_the_field_its_word_lands_in over every case row under all three orders (control: 10 failures on master). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix #626's review findings: the sixth emitter, P1's fold, a wider guard The comma report on the word after a family comma hardcoded "read as the given name", and with nothing before the comma H1 moves that word to the family behind a trailing title (`, Ma Dr.`). It now defers its field to assemble like the other five, so its given-side wording reads "a given name". P1's family-first fold is a third rule moving reported words (`de Kim Ma` under FAMILY_FIRST); the comments and docs that named only H1 and P6 now name all three, and "H1's swap" is H1's move. Assemble withdraws a field_tail report in an emptied name instead of shipping half a sentence, and FIELD_READING is asserted complete over Role. The sweep's phrase list covers the settled phrasings too (the suffix run's, the numeral fork's, H4's "the name", P6's), states that locale rows are not checked, and a second test records how many claims it checks (1151) so it cannot go vacuous. Rows `, Ma Dr.` (fix(#316)) and `de Kim Ma` under FAMILY_FIRST (fix(#289)), both bisected; the `Kim Min Do` note no longer claims both family-first orders. Unit tests for assemble's join; two test_assign names now say "field". decisions.md's entry carries a recompute recipe and re-measured figures: 2,725 of 89,092 claims wrong on master over 53,517 texts (comma shapes with an empty head and a trailing title added), 0 on the tree; control 11 failures on master. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix the fix commit's review: the opt-in movers, #629's route, the recipe The patronymic rotations (O1, O2) and middle_as_family's fold (O3) also move a word assign reported, under opt-in policies no three-order grid reaches, so reverting the leading-particle emitter passed the whole suite. Rows `Van Ivan Petrovich` (East Slavic), `Van Ali Veli oglu` (Turkic) and `Do Bishop Do` (FAMILY_FIRST + middle_as_family) pin them, and every mover list now names them. The sweep's claim count moves 1151 -> 1155 with the rows, its control on master 11 -> 14, and the sweep reads its population from `_swept_rows` rather than a copy of the skip. decisions.md#A1 gains `Open: #629` (segment's "consumed as suffix"), which mechanisms.md and the release log now qualify; AGENTS.md says the guard covers only the phrasings it lists. The `, Ma Dr.` report dates from #289, not #316; the recipe names the 6f15f21 case table and the collapsed empty-head space, and scopes its figures to the three orders. A mixed-role unit test pins assemble's first-referent choice; the 2026-09-17 entry notes the rewording; the release bullet adds the patronymic example, which 2.3.0 also misreported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Correct #626's recipe and A1's interacts list (third review) The recipe's case table is 0c54662's, the parent of #614's first commit (6f15f21 has the same table but is not that parent), and the claim that today's table moves the wrong count by at most two was false: the rows the reviews added read 6 more on master. The random draws, the stripping and the comma forms are now stated exactly enough to rebuild the 53,517 texts. rules.md#A1's interacts list names the movers the entry names: P1 and O1 to O3 beside H1 and P6. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Simplify #626's change: one swept-row list, shared helpers The two order-sweep invariants walk one `_SWEPT_ROWS` list, so the row filter is stated once and neither test skips (582 -> 110 skips in the suite); `_NAME_FIELDS` is `NAME_ROLES`; test_assign's `_reported` reads through `run` rather than a hand-listed stage chain. In assign, title-or-name's two branches share one constructor, the particle emitter's comment points at `PendingAmbiguity.field_tail` for the movers instead of carrying a third copy of the list, and its one-use `token` binding is inlined. No behavior change: fingerprint byte-identical on every common name, field grid 0 of 89,092, five gates exit 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 26cdb89 commit bc24386

13 files changed

Lines changed: 360 additions & 76 deletions

File tree

‎AGENTS.md‎

Lines changed: 2 additions & 2 deletions
Large diffs are not rendered by default.

‎docs/design/decisions.md‎

Lines changed: 7 additions & 1 deletion
Large diffs are not rendered by default.

‎docs/design/mechanisms.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ Problem shape. A retired default rides in on old pickles, but a user may have de
114114

115115
## AMBIGUITY-AT-THE-DECISION-SITE — emit where the branch is taken
116116

117-
Problem shape. An ambiguity report should fire exactly when the parse chose between live readings — no more, no less — and the choosing happens in code, not in vocabulary. Contract statement. Emit at the site that takes the branch, not where an ambiguous tag sits; when a fork's two branches are decided in different stages, EVERY deciding stage carries an emitter; and a branch that runs but changes nothing is not a decision and must not report. How it works. PARTICLE_OR_GIVEN fires from assignment for a lone leading particle and from grouping when a title shifts it off the front — for two years only the first site emitted. And keying on
117+
Problem shape. An ambiguity report should fire exactly when the parse chose between live readings — no more, no less — and the choosing happens in code, not in vocabulary. Contract statement. Emit at the site that takes the branch, not where an ambiguous tag sits; when a fork's two branches are decided in different stages, EVERY deciding stage carries an emitter; and a branch that runs but changes nothing is not a decision and must not report. The decision is the site's, but a FIELD the report names is not: a later rule can still move the word, so the emitter leaves the field to assemble (`PendingAmbiguity.field_tail`), which words it from the final role (#626; a comma-structure report on a part past the second comma still names "suffix" for a part holding a title, #629). How it works. PARTICLE_OR_GIVEN fires from assignment for a lone leading particle and from grouping when a title shifts it off the front — for two years only the first site emitted. And keying on
118118
"the code got here" instead of "the outcome differed" once reported
119119
a fork for all ambiguous particles on "Dr. Van Jr.". Lives in. The AmbiguityKind emitters across _pipeline/ (rule A1 is the observable contract). Reach for it when. Adding any ambiguous vocabulary or any new fork — count the deciding sites, then count the emitters.
120120

‎docs/design/rules.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2429,7 +2429,9 @@ A1. Rationale: a caller can only act on doubt that is reported.
24292429
contradicts: a letter both a connective and an initial, read as
24302430
the generation it also spells, is neither of the two the
24312431
connective-or-initial fork offered, and only that fork's report
2432-
goes — the generation's own stands.
2432+
goes — the generation's own stands. And a report that names the
2433+
field its word was read into names the field the word holds in
2434+
the result, whatever rule moved it after the report was made.
24332435
"Van Johnson" → ambiguities=("particle-or-given",)
24342436
"JOHN QUINCY SMITH I" → ambiguities=("suffix-or-name",)
24352437
"Jane „JD Smith" → ambiguities=("unbalanced-delimiter",)
@@ -2439,7 +2441,7 @@ A1. Rationale: a caller can only act on doubt that is reported.
24392441
segmenter's own error, which propagates — a user-code error is
24402442
not a content error. (Needs the optional extra to demonstrate,
24412443
so no example line.)
2442-
history: decisions.md#A1 · interacts: P3, S2 · implemented: nameparser/_pipeline/_assemble.py, nameparser/_pipeline/_state.py
2444+
history: decisions.md#A1 · interacts: H1, O1, O2, O3, P1, P3, P6, S2 · implemented: nameparser/_pipeline/_assemble.py, nameparser/_pipeline/_state.py
24432445

24442446
A2. Rationale: an input with no name content names nobody, and
24452447
saying so beats inventing fields from punctuation.

‎docs/release_log.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ Release Log
3636

3737
- **Fix a comma part read wholly as suffixes reporting a particle in it as chained onto a name.** ``parse("John Smith, Jr., Freiherr von Richthofen").ambiguities`` names ``comma-structure`` alone, where 2.0 through 2.3 also named ``particle-or-given`` for ``von`` -- a word the same parse had put in the suffix, so the report described a reading it never made. This fix moves no field; the title change below (#603) moves ``Freiherr`` to the title. A part the parser consumes as suffixes, after a suffix comma or past the second comma, reports what the part is and nothing about its words as names, and the particle chain's credential-acronym report this release adds (above) keeps the same bound: ``John Smith, PhD Do Ma`` reports the comma's decision once and not again for ``Ma``. See the 2026-09-28 bullet of the ``C1`` entry in ``docs/design/decisions.md``
3838

39+
- **Fix an ambiguity report naming the field its word was first placed in rather than the one it ends up in.** ``parse("Attorney General of Minnesota").ambiguities`` says ``'General of Minnesota'`` was "read as a family name by convention", where 2.3.0 said "read as a given name" of a name whose family it was, and a leading ``Van`` that a patronymic rule rotates into the family name (``Van Ivan Petrovich`` under ``Policy(patronymic_rules=frozenset({PatronymicRule.EAST_SLAVIC}))``) is reported as read as a family name where 2.3.0 said a given one. A report that names the field one word was read into now names the field the word holds in the result, whatever rule moved it after the report was made. This fix moves no field and no report kind. See the #626 entry under ``A1`` in ``docs/design/decisions.md`` (closes #626)
40+
3941
- **A credential after the name core starts a suffix run to the end of its part.** ``HumanName("John Smith PhD Jones")`` gives suffix ``PhD Jones`` and reports ``suffix-or-name`` for the name word it took, where 2.3.0 gave middle ``Smith PhD``, last ``Jones``; ``Smith, John PhD Jones`` gives suffix ``PhD Jones`` where 2.3.0 gave middle ``Jones``, suffix ``PhD``. A title in the run is a title, so ``Eric H. Holder Jr. Attorney General`` gives title ``Attorney General``, suffix ``Jr.``, as the comma spelling ``Eric H. Holder, Jr. Attorney General`` already read, where 2.3.0 gave middle ``H. Holder Jr. Attorney``, last ``General``. Only an unambiguous credential or generational word starts the run, and only behind the name core -- two name words with no comma, or the given part after one. A credential that is also a name does not start one, nor does a surname particle or a single letter, so ``John Smith MA Jones`` and ``Mohamed Ali Abd Allah`` read as before, and a bare title word starts nothing: ``Mary Jane King Smith`` keeps middle ``Jane King``. See the ``S2`` entry of ``docs/design/decisions.md`` (closes #602)
4042

4143
- **Fix a comma part that opens with a credential being read as a given name.** ``HumanName("John Smith, PhD Jones")`` gives first ``John``, last ``Smith``, suffix ``PhD Jones`` and reports ``suffix-or-name`` for the name word it took, where 2.3.0 gave first ``PhD``, middle ``Jones``, last ``John Smith`` (and 1.4.0 title ``PhD``, first ``Jones``). With one word before the comma the whole part is suffixes: ``Doe, PhD Jones`` gives last ``Doe``, suffix ``PhD Jones``. A title word in the part stays a title. A word that is a title as well as a credential (``MD``, ``Ms``) opens nothing, so ``Smith, Ms Jane`` keeps title ``Ms``, first ``Jane``. An undeclared delimiter becomes one more word of the part: ``Steven Hardman, RN - CRNA`` gives first ``Steven``, last ``Hardman``, suffix ``RN - CRNA``, where 1.4.0 through 2.3.0 gave first ``RN``, middle ``-``, last ``Steven Hardman``. The split spelling now counts like the joined one, so ``Smith, John Ph. D. Jones`` gives suffix ``Ph. D. Jones``, where 2.3.0 gave middle ``Jones``, suffix ``Ph. D.``. See the ``C1`` entry of ``docs/design/decisions.md`` (closes #603)

‎nameparser/_pipeline/_assemble.py‎

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
"""Not a stage: converts the final ParseState into a public ParsedName.
22
3-
Consumes: tokens (all roles set), dropped, ambiguities (by index).
3+
Consumes: tokens (all roles set), dropped, ambiguities (by index;
4+
one carrying ``field_tail`` is worded from its word's final role,
5+
#626).
46
Produces: a validated ParsedName -- the constructor re-checks every
57
invariant (span order/bounds, ambiguity subset), so a pipeline bug
68
that would produce an invalid result dies HERE, not in a renderer
@@ -18,7 +20,7 @@
1820
"""
1921
from __future__ import annotations
2022

21-
from nameparser._pipeline._state import ParseState
23+
from nameparser._pipeline._state import FIELD_READING, ParseState
2224
from nameparser._types import (
2325
Ambiguity, AmbiguityKind, ParsedName, Role, Token,
2426
)
@@ -112,8 +114,17 @@ def assemble(state: ParseState) -> ParsedName:
112114
and any(t.role in (Role.SUFFIX, Role.TITLE)
113115
for t in materialized)):
114116
continue
115-
ambiguities.append(
116-
Ambiguity(pending.kind, pending.detail, materialized))
117+
detail = pending.detail
118+
if pending.field_tail is not None:
119+
# A report naming a field, in a name emptied above (no
120+
# emitter of one reaches that today: every referent is a
121+
# name word): no field is left to name, so it goes rather
122+
# than ship the sentence's first half.
123+
if not materialized:
124+
continue
125+
detail = (f"{detail}{FIELD_READING[materialized[0].role]}"
126+
f"{pending.field_tail}")
127+
ambiguities.append(Ambiguity(pending.kind, detail, materialized))
117128
return ParsedName(original=state.original,
118129
tokens=tuple(final.values()),
119130
ambiguities=tuple(ambiguities))

‎nameparser/_pipeline/_assign.py‎

Lines changed: 36 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -363,9 +363,10 @@ def _assign_main(seg_idx: int, state: ParseState,
363363
#
364364
# Every bare ambiguous acronym the FINAL peel had to resolve is one
365365
# coin-flip each, in either direction, so the report collects
366-
# rather than overwrites. Deferred to after assignment because the
367-
# wording reads the role back, and which role "not peeled" means
368-
# depends on name_order. (The roman-numeral fork needs no such
366+
# rather than overwrites. Deferred to after assignment because what
367+
# a pick is declined as depends on the role it took, and which role
368+
# "not peeled" means depends on name_order; the field itself is
369+
# worded by assemble (#626). (The roman-numeral fork needs no such
369370
# deferral and is reported here.)
370371
#
371372
# Read once in group, after P3's connective joins and ahead of the
@@ -435,8 +436,6 @@ def _assign_main(seg_idx: int, state: ParseState,
435436
# call budget on every parse otherwise (decisions.md#parse-cost).
436437
if peeled.names <= 1:
437438
head = pieces[name_pieces[0]]
438-
token = tokens[head[0]]
439-
assert token.role is not None
440439
# Both conventions here turn on a lone name word, so both
441440
# report at the site that places one
442441
# (mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE), and one
@@ -458,16 +457,16 @@ def _assign_main(seg_idx: int, state: ParseState,
458457
# `suffix-or-name`" (history: decisions.md#H4) -- only the
459458
# word made into a name reports, and no name piece survived
460459
# the peel, so the carve-out above made the first post-nominal
461-
# the name. The role comes off the token for the reason stated
462-
# at the particle emitter below.
460+
# the name. Its field is worded by assemble from the final role
461+
# (`PendingAmbiguity.field_tail`, #626).
463462
if peeled.names == 0 and field_undecided:
464463
text = " ".join(tokens[i].text for i in head)
465464
ambiguities.append(PendingAmbiguity(
466465
AmbiguityKind.SUFFIX_OR_NAME,
467466
f"{text!r} is post-nominal vocabulary with no name word "
468-
f"beside it; read as a {token.role.value} name rather than "
469-
f"a post-nominal, nothing else being left to be the name",
470-
tuple(head)))
467+
f"beside it; read as ", tuple(head),
468+
field_tail=" rather than a post-nominal, nothing else "
469+
"being left to be the name"))
471470
# One name piece off the peel: the convention placed a lone
472471
# name word. A suffix beside it is not a decision -- 'Smith
473472
# Jr.' and "'Smitty' Jones Jr." ARE this convention -- and the
@@ -489,16 +488,17 @@ def _assign_main(seg_idx: int, state: ParseState,
489488
# Wales'): the leading-title peel took the whole name.
490489
if any("vocab:title" in tokens[i].tags for i in head):
491490
text = " ".join(tokens[i].text for i in head)
491+
lone = len(head) == 1
492492
ambiguities.append(PendingAmbiguity(
493493
AmbiguityKind.TITLE_OR_NAME,
494494
f"{text!r} is title vocabulary and the only name word "
495495
f"the title peel left standing; read as the name by "
496496
f"convention rather than as more title"
497-
if len(head) == 1 else
497+
if lone else
498498
f"{text!r} is the only name unit and joins title "
499-
f"vocabulary to a name word; read as a "
500-
f"{token.role.value} name by convention",
501-
tuple(head)))
499+
f"vocabulary to a name word; read as ",
500+
tuple(head),
501+
field_tail=None if lone else " by convention"))
502502
# rules.md#O5: "a name of one name word that nothing else has
503503
# decided reads that word as the given name under the default
504504
# given-first order, and as the family name under a declared
@@ -517,16 +517,17 @@ def _assign_main(seg_idx: int, state: ParseState,
517517
ambiguities.append(PendingAmbiguity(
518518
AmbiguityKind.GIVEN_OR_FAMILY,
519519
f"{text!r} is the only name word and nothing else "
520-
f"decides it; read as a {token.role.value} name by "
521-
f"convention, which follows the read order",
522-
tuple(head)))
520+
f"decides it; read as ", tuple(head),
521+
field_tail=" by convention, which follows the read "
522+
"order"))
523523
for piece in peeled.picks:
524-
# every pick is in rest, so the loops above just gave it a role
524+
# every pick is in rest, so the loops above just gave it a
525+
# role; whether it was read as a suffix or a name settles what
526+
# it was declined as, and which field assemble names (#626)
525527
token = tokens[piece[0]]
526528
assert token.role is not None
527-
taken, declined = (
528-
("a suffix", "a name part") if token.role is Role.SUFFIX
529-
else (f"a {token.role.value} name", "a post-nominal"))
529+
declined = ("a name part" if token.role is Role.SUFFIX
530+
else "a post-nominal")
530531
# A word in the class by SHAPE (rules.md#S2, #S3) is in no
531532
# wordlist and may be written with its periods, so the listed
532533
# member's wording would misdescribe it on both counts (#563)
@@ -537,27 +538,23 @@ def _assign_main(seg_idx: int, state: ParseState,
537538
"and an ordinary name")
538539
ambiguities.append(PendingAmbiguity(
539540
AmbiguityKind.SUFFIX_OR_NAME,
540-
f"{token.text!r} {what}; read as {taken} rather than "
541-
f"{declined}",
542-
piece))
541+
f"{token.text!r} {what}; read as ", piece,
542+
field_tail=f" rather than {declined}"))
543543
# leading ambiguous particle read as a name (#121 surfaced)
544544
if name_pieces:
545545
head = pieces[name_pieces[0]]
546546
if (len(head) == 1 and len(name_pieces) > 1
547547
and "vocab:particle-ambiguous" in tokens[head[0]].tags):
548-
# the loops above gave the head piece its role from
549-
# `order`, which is _effective_order's answer and not
550-
# necessarily name_order's -- a script_orders entry
551-
# overrides it. So read the role off the token rather than
552-
# assume given, or re-derive it here; same reason as
553-
# SUFFIX_OR_NAME just above.
554-
token = tokens[head[0]]
555-
assert token.role is not None
548+
# the field is assemble's to word from the final role
549+
# (`PendingAmbiguity.field_tail`, #626), never assumed given:
550+
# `order` above is _effective_order's answer, not
551+
# necessarily name_order's, and a later rule may still move
552+
# the word ('Van Ivan Petrovich' under the East Slavic rule
553+
# rotates 'Van' into the family)
556554
ambiguities.append(PendingAmbiguity(
557555
AmbiguityKind.PARTICLE_OR_GIVEN,
558-
f"leading {token.text!r} may be a family-name "
559-
f"particle; read as a {token.role.value} name",
560-
tuple(head)))
556+
f"leading {tokens[head[0]].text!r} may be a family-name "
557+
f"particle; read as ", tuple(head), field_tail=""))
561558
return order
562559

563560

@@ -852,11 +849,13 @@ def assign(state: ParseState) -> ParseState:
852849
if state.pieces[1] and len(state.pieces[1][0]) == 1:
853850
i = state.pieces[1][0][0]
854851
if not tokens[i].tags.isdisjoint(_AMBIGUOUS_CREDENTIAL_TAGS):
852+
# the field is assemble's to word: with nothing before
853+
# the comma and a title after the word, H1 moves it to
854+
# the family (', Ma Dr.', #626's review)
855855
ambiguities.append(PendingAmbiguity(
856856
AmbiguityKind.SUFFIX_OR_NAME,
857857
f"{tokens[i].text!r} after the comma is also an "
858-
f"ordinary name word; read as the given name",
859-
(i,)))
858+
f"ordinary name word; read as ", (i,), field_tail=""))
860859
# Segment 1 is read before segment 0's wholly-family pass. It
861860
# consumes piece tags and text only -- nothing segment 0's
862861
# read writes. (A title standing after the comma behind a whole

‎nameparser/_pipeline/_state.py‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,12 +130,35 @@ class PendingAmbiguity:
130130
extract_delimited knows only a character offset, so it records that
131131
and tokenize resolves it to the containing token's index. Stages
132132
after tokenize set ``indices`` directly and leave ``origin`` None.
133+
134+
``field_tail`` is for a report that names the field its word was
135+
read into. post_rules may still move the word (H1's move behind a
136+
title, P1's family-first fold, P6's attachment, and under opt-in
137+
policies the patronymic rotations and `middle_as_family`'s fold),
138+
so the emitter
139+
writes ``detail`` up to the field and the rest of its sentence
140+
here -- ``""`` where the field ends the sentence, which is why
141+
assemble tests ``is not None`` -- and assemble joins them around
142+
the FIELD_READING of the first referent's final role, the one
143+
place that knows it (#626). Until then ``detail`` is a fragment.
133144
"""
134145

135146
kind: AmbiguityKind
136147
detail: str
137148
indices: tuple[int, ...] = ()
138149
origin: int | None = None
150+
field_tail: str | None = None
151+
152+
153+
#: How a report names the field a word was read into, by final role.
154+
FIELD_READING: Mapping[Role, str] = MappingProxyType({
155+
Role.TITLE: "a title", Role.GIVEN: "a given name",
156+
Role.MIDDLE: "a middle name", Role.FAMILY: "a family name",
157+
Role.SUFFIX: "a suffix", Role.NICKNAME: "a nickname",
158+
Role.MAIDEN: "a maiden name"})
159+
# every role a referent can hold, or a new one is a KeyError at the
160+
# first parse whose report lands in it
161+
assert FIELD_READING.keys() == set(Role)
139162

140163

141164
@dataclass(frozen=True, slots=True)

0 commit comments

Comments
 (0)