Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -742,7 +742,7 @@ for n in ('Smith, John','Smith, XYZ'): print(n, calls_for(off.parse, n), calls_f
COST, `sys.setprofile` call counts (mean of 50, after one warm-up) against `git archive origin/master`: `Smith, John` 183 → 183, `Smith, JOHN` 183 → 183, `Smith, XYZ` 182 → 182, `John Smith, PhD` 217 → 217, `John Smith, MA` 252 → 252, `John Smith, Ph. D.` 254 → 254, `John Smith, CPA` 217 → 217, `John Smith, MD PhD` 260 → 260, `John Smith, MBA CPA` 261 → 261, `Dr. Juan de la Vega III` 366 → 366, `John Smith, XYZ` 251 → 266, and an all-caps record with a capitalized given name and a credential, `LLOYD WEBBER, ANDREW PhD`, 338 → 364, a constant +26 that does not grow with the name's length (`'LLOYD'*64 + ' WEBBER, ANDREW PhD'` 653 → 679; the second draft's per-character contrast test had made that +350). Past the all-caps check the contrast is linear in the words before the comma, about four frames a word (the generator, the case test's call, and the own-words walk's fold and marker test; `'GAULLE ' + 'ap '*k + 'GAULLE, CHARLES'` is +7, +26, +38, +54 over master at k = 0, 1, 4, 8), and constant in a word's letters (`'de ' + 'GAULLE'*k + ', CHARLES'` 253 → 275, 343 → 365 and 631 → 653 at k = 1, 16, 64), where the capital-then-lowercase draft grew a frame per letter; all paid only once a caps word is in hand. The comma test is on by default, so three C-level checks go before any call: two or more words before the comma (the first draft lacked it, and `Smith, JOHN`, a common record format, paid +25 for a flip it can never make — the code review), the first word in capitals, and the lone-two-letter length. Every caller of the caps predicate asks first, in C, what it would decline anyway — alphabetic capitals, not a listed suffix word — so a listed credential never pays for the call (the review of the first fix found `John Smith, CPA` +4 and `John Smith, Ph. D.` +5 before these). The 2026-09-14 entry's recompute recipe predates the enum: its `on` is `Policy(unlisted_caps_suffixes=CapsSuffixes.EVERYWHERE)` now, and its `off=Parser()` is `Policy(unlisted_caps_suffixes=CapsSuffixes.OFF)`, `Parser()` being AFTER_COMMA.
- 2026-10-02 (Derek), #573 — a never-given particle opening the given part after a family comma leaves no word to spare; the member behind it is the surname it heads, unless mixed-case capitals say credential. The record is the P6 section's entry of the same date.
- 2026-10-02 (Derek), #578 — A PARTICLE BETWEEN A CREDENTIAL AND A TITLE-CASE WORD STAYS SILENT, AS IT READS. `Doe, Jane PhD vd Ma` reads given 'Jane', middle 'vd Ma', family 'Doe', suffix 'PhD', reporting nothing. No new rule decides that: S2's given-part slot already says a particle in front of a member takes the member out of the slot, and where the word behind reads as a name the two are one name, asked nothing and reporting nothing — `vd` reads as `van` does (`Doe, Jane PhD van Ma`, the same reading), and S2's own boundary example is `Doe, John van Ma`. The writing is malformed whichever reading is meant: a Title-case `Ma` behind a lowercase `vd` behind a `PhD` is neither a Dutch listing nor a credential run as anyone writes one. A reader leaning anywhere leans suffix (Derek: both words follow a known credential, and the writer likely did not know how suffixes are capitalized), but that is a guess about a writer's mistake, not a convention the writing carries. Declined: (a) reading `vd Ma` as suffixes, which needs code to make a credential speak across a particle — against P2 (a particle joins the word behind it) and #554 (`vd` speaks for nothing behind it); (b) reporting the fork, which needs an emitter on the family-comma path for a decision the particle chain makes before assign knows which read the part gets (AGENTS.md: prefer leaving a fork silent and documenting the omission over machinery for input nobody reads with confidence). Where the writing does speak the parser already follows it: `Doe, Jane PhD vd MA` reads suffix 'PhD vd MA' and reports (#573). Pinned by the case row `a_particle_after_a_credential_heads_a_title_case_name`.
- 2026-10-04 (Derek), #602 — A CREDENTIAL AFTER THE NAME CORE STARTS A RUN TO THE END OF ITS PART. An unambiguous Latin suffix word of two or more letters standing after two name words with no comma (a lone particle not counting toward the two), or after the given word in the part after a family comma, makes every later word of that part a suffix, except a title word, which reads as a title: `John Smith PhD Jones` → suffix 'PhD Jones', `Eric H. Holder Jr. Attorney General` → suffix 'Jr.', title 'Attorney General'. A name word the run absorbs is reported `suffix-or-name`, a reader hesitating over `Jones` there. Before this the three comma shapes disagreed — middle 'Smith PhD' with no comma, middle 'Jones' and suffix 'PhD' after a family comma, and suffix 'Jr., Jones' after a suffix comma (C2's tail) — and the no-comma reading made a credential a middle name, the one reading nobody intends. The suffix-comma path already read it this way, and the comma spelling of the Holder name always did.
- 2026-10-04 (Derek), #602 — A CREDENTIAL AFTER THE NAME CORE STARTS A RUN TO THE END OF ITS PART. An unambiguous Latin suffix word of two or more letters standing after two name words with no comma (a lone particle not counting toward the two; qualified by #627, below), or after the given word in the part after a family comma, makes every later word of that part a suffix, except a title word, which reads as a title: `John Smith PhD Jones` → suffix 'PhD Jones', `Eric H. Holder Jr. Attorney General` → suffix 'Jr.', title 'Attorney General'. A name word the run absorbs is reported `suffix-or-name`, a reader hesitating over `Jones` there. Before this the three comma shapes disagreed — middle 'Smith PhD' with no comma, middle 'Jones' and suffix 'PhD' after a family comma, and suffix 'Jr., Jones' after a suffix comma (C2's tail) — and the no-comma reading made a credential a middle name, the one reading nobody intends. The suffix-comma path already read it this way, and the comma spelling of the Holder name always did.
WHAT STARTS THE RUN is one predicate, `_pieces.starts_a_credential_run`, read by the trailing peel (`credential_run`, applied to every finished peel, so assign, P5's reserve, P2's chain stop and the maiden walk see one answer) and by the given-part walk. EXCLUDED, each with its reason: a TITLE WORD — 746 title words are in no suffix set (measured 2026-10-04: `len(L.titles - L.suffix_acronyms - L.suffix_words)` for the default lexicon), many of them surnames (`King`, `Bishop`, `Judge`), so `Mary Jane King Smith` keeps family 'Smith'; a title is read from the end of a name only behind a comma or with a period (H5, Derek). The AMBIGUOUS CLASS, which carries no evidence of which it is. A SINGLE LETTER, in any case: a lowercase `i` or `v` carries no `initial` tag, and found by the v1 off-switch test, `Josep Carod i Rovira` with `i` removed from the conjunctions read suffix 'i Rovira'. A CONNECTIVE. A word that is also PARTICLE or BOUND GIVEN-NAME vocabulary — found by the suite, not foreseen: `abd` is in the credential list (ABD) and heads `Abd Allah`, and `Mohamed Ali Abd Allah` read suffix 'Abd Allah' until it was excluded. The NON-LATIN HONORIFIC words, an honorific followed by a name word being a different question from a credential followed by one. A word that is also a title (`MD`, `Ms`, `Sr.`) does start the run: position decides, and after the name core it is the suffix.
DECLINED: lifting the credential out — `John Smith PhD Jones` as given 'John', middle 'Smith', family 'Jones', suffix 'PhD', the family-comma path's old reading. It keeps every name word in a name field, but it reads a credential as something written in the middle of a name, which is the reading the rule rejects.
REVERSED BY THIS ENTRY rather than edited: the #578 bullet above (`Doe, Jane PhD vd Ma` now reads suffix 'PhD vd Ma', the credential starting the given part's run), and two of the three shapes #544's ACCEPTED LIMITS kept out of the company's reach — a title standing between the credential and the member, and a member a particle chain had taken. The case rows `a_title_between_degree_and_member_keeps_it_a_name` and `a_member_the_particle_chain_took_is_out_of_reach` keep their ids and now pin the run. The third, the merged `Ph. D.`, stays out of reach: that piece is outside the walk the run is read over.
Expand All @@ -766,6 +766,10 @@ for n in ('Smith, John','Smith, XYZ'): print(n, calls_for(off.parse, n), calls_f
MEASURED 2026-10-08 against 4278693d (#621's merge), py3.11, `nameparser.__file__` asserted, with the #614 harnesses and recipe above: fingerprint 0 moves; connective grid 0; S2 stress grid 12, three garbage names under four policies, none in the head-plus-one-word slice -- `Freiherr von Berg Dr. X.Y.Z. VD` reads title 'Freiherr Dr.', suffix 'X.Y.Z. VD' (H5's trailing title, the class #614 moved; master family 'von Berg Dr. X.Y.Z.'), `Jan Freiherr von Berg VD V and ba I` reads family 'VD V and ba I' as adf6da88 did (master suffix 'I'), and `de Mesnil Ma do mc and Ph. D.` reports the 'do' the peel took beside adf6da88's 'Ma'. Five gates exit 0. Frames unchanged, 308 on 3.11 and 289 on 3.12-3.15 (decisions.md#parse-cost).
/SIMPLIFY (2026-10-08), measured. Each pass of `tail_reading` had rescanned for the second unit from the front, a C-level quadratic no frame guard saw: `Freiherr von` + `Berg `*n + `MA Dr. `*n read 5.77x for 4x the input against master's 4.08x; the lookup is now made once per read and handed to every pass (4.03x), and `test_the_second_unit_is_found_once_per_read` counts it. `_weighed` is pinned to the reading by `test_a_word_the_read_takes_is_never_one_the_count_joined` (no piece marked JOINED is ever in the run the read takes; 320 grid names fail with the test answering False). The gathering of an absorbed particle run reads the BOUND marks, and #602's run start one condition. Grids byte-identical, frames unchanged. Left for a follow-up, each moving readings: merging the tail's particle runs in group before the read, which would retire the BOUND stop and the gathering (80 grid names move, `Dr. mc mc`, and the chain's particle report needs re-keying); and the name start decided once, the item below.
LEFT OPEN, by Derek's scoping: reading a family comma's given part the same way, and deciding the name's start once for a title-particle head (#620's related paths, which change readings).
- 2026-10-10 (Derek), #627 — A LONE PARTICLE WITH THE RUN RIGHT BEHIND IT COUNTS AS A NAME WORD. #602's run start needs two name words before the credential, and "a lone particle is not a name word for the count", for `de Mesnil`, where the particle and the word behind it are one surname. Since #620 the count is taken in the chain's units, so a particle that opens a unit counts once (`van der`, `von Berg`) and a P3-joined phrase is one piece (`von und zu`); the lone particle stayed uncounted only where nothing joins it, which is exactly where a credential stands behind it. So `John von PhD Jones` read middle 'von PhD', family 'Jones' -- the chain later taking the credential as the particle's surname -- while `John van der PhD Jones` and `John von und zu PhD Jones` read family 'van der' / 'von und zu', suffix 'PhD Jones', and the comma spelling `Smith, John von PhD Jones` already read suffix 'PhD Jones'. The issue's question was which way to make them agree.
DECIDED (Derek): the particle stops at the credential, so it is a name word of its own and the rest is suffix: `John von PhD Jones` → family 'von', suffix 'PhD Jones'. The input is malformed (a particle is not written before a post-nominal), so the decision is whichever reading the shared count already gives the other forms, not a special case for this one: the existing clause gains its own limit, a lone particle counting when the run starts right behind it, in `run_start`'s one loop. The caller without the chain's units (`trailing_start`) shares the expression so the two cannot disagree, but no reading shows that half: it changes the position it returns for `Smith, John von PhD Jones` (2 where master gave 4), so group's chain takes its stopped path and asks `trailing_start` again, reaching the same reading at 16 more frames (339 → 355 on py3.11; a rare shape no frame guard sees, decided for the shared expression over a per-caller copy); a mutant ignoring the flag there passed the suite with the grid byte-identical (#627's reviews). The 1.4.0 and 2.3.0 wheels read middle 'von PhD', as they read `von und zu PhD`; the joined forms changed this cycle with #602.
Declined: counting every unit, a lone particle included. It moved 2,836 S2-grid parses rather than 2,432, the extra ones all `de Mesnil <credentials>`, where it counted the surname as two words, and it split the two callers: a leading `van` counted before another particle (`van la Smith Jr. Jones` → suffix 'Jr. Jones') but not before a name word (`van Smith Jr. Jones`), and the case row `van la Smith Secretary Jr. Dr. Smith` caught it.
MEASURED 2026-10-10 against 195c4081 (#630's merge), py3.11, `nameparser.__file__` asserted on each side. S2 stress grid (the #614 entry's recipe above, 324,552 parses): 2,432 role moves, no report-only move; every move is a lone particle -- in that grid `van`, or a dual `do`/`mc`/`vd` in any case -- with an unambiguous credential right behind it becoming the family (under the family-first orders the given name, the family staying first), the run taking the rest. The grid's comma shapes do not move; the review found the same move in shapes the grid lacks, as intended: other particles (`von`, `de`, `du`, `la`, `bin`), and a head before a suffix comma (`John von PhD Jones, Jr.` → family 'von', suffix 'PhD Jones, Jr.'). Fingerprint of every `tools/differential/corpus*.jsonl` name and `tests/v2/cases.py` text at 195c4081 under the three orders (1,925 texts, the seven fields and every report): 0 moves; this change adds one text, its own example, which moves. The five differential gates exit 0. Held by the case row `John von PhD Jones` (`fix(#627)`) against the row `van la Smith Secretary Jr. Dr. Smith`, whose lone particle has a name word behind it and still does not count, and the row `Freiherr von vd PhD Jones`, where the particle standing right before the credential is one the run took and the flag is not asked of it (a mutant asking it of every piece passed the suite until that row).

### indic-honorifics — the renunciate class and the Indic honorific vocabulary (2026-09-06, #346/#344/#343)

Expand Down
11 changes: 8 additions & 3 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -1160,9 +1160,12 @@ S2. Rationale: generational suffixes and credentials are recognized
lone member of the ambiguous class, which reads as the
credential. A lone particle is
not a name word for the count, 'de Mesnil' being one surname,
and a particle run with the words the chain joins to it is one
name word, 'der la' and 'la Smith Secretary' each counting once
(P2). A title word
unless the credential stands right behind it, leaving it nothing
to join: then it counts as a name word of its own ('John von PhD
Jones'). A
particle run with the words the chain joins to it is one name
word, 'der la' and 'la Smith Secretary' each counting once (P2).
A title word
never starts the run, and neither does a member of the ambiguous
class, a single letter, a connective, a word that is also
particle or bound given-name vocabulary, or a non-Latin honorific
Expand Down Expand Up @@ -1250,6 +1253,8 @@ S2. Rationale: generational suffixes and credentials are recognized
"Eric H. Holder Jr. Attorney General" → title="Attorney General"
"John Smith PhD Prof. Ma" → suffix="PhD Ma"
"Smith, John PhD Jones" → suffix="PhD Jones"
"John von PhD Jones" → family="von"
"John von PhD Jones" → suffix="PhD Jones"
"Smith, John PhD de Jr." → family="de Smith" · boundary
"Mary Jane King Smith" → family="Smith"
"Josep Carod i Rovira" → family="Carod i Rovira"
Expand Down
21 changes: 13 additions & 8 deletions nameparser/_pipeline/_pieces.py
Original file line number Diff line number Diff line change
Expand Up @@ -1189,7 +1189,8 @@ def run_start(rest: Sequence[int], names: int,
first credential that starts one (`starts_a_credential_run`) with
two name pieces in front of it, or `names` where none does -- with
no comma the family has to exist first. A lone particle does not
count toward the two, 'de Mesnil' being one surname. The position
count toward the two, 'de Mesnil' being one surname, unless the
run starts right behind it (#627). The position
alone, for readers that need only where the name ends
(`trailing_start`); `credential_run` builds the rest of the answer.

Expand All @@ -1209,21 +1210,25 @@ def run_start(rest: Sequence[int], names: int,
if names < 3:
return names
core = 0
lone = False
for p in range(names):
q = rest[p]
tags = tokens[pieces[q][0]].tags
if (core >= 2 and "vocab:suffix" in tags
if (core + lone >= 2 and "vocab:suffix" in tags
and tags.isdisjoint(_NOT_A_RUN_START)
and starts_a_credential_run(pieces[q], ptags[q], tokens)):
return p
# a lone particle is not yet a name: 'de Mesnil' is one surname
# -- unless it opens a chain run, which is then the surname
# (with no `units` every piece opens its own)
if (units is None or units[q] == OPENS) and not (
len(pieces[q]) == 1 and "particle" in tags
and (units is None or q + 1 == len(units)
or units[q + 1] == OPENS)):
core += 1
# (with no `units` every piece opens its own), or the run
# starts right behind it, leaving it nothing to join: then it
# counts as a name word of its own, as 'van der' and 'von und
# zu' do (#627, 'John von PhD Jones')
if units is None or units[q] == OPENS:
lone = (len(pieces[q]) == 1 and "particle" in tags
and (units is None or q + 1 == len(units)
or units[q + 1] == OPENS))
core += not lone
return names


Expand Down
23 changes: 23 additions & 0 deletions tests/v2/cases.py
Original file line number Diff line number Diff line change
Expand Up @@ -9998,6 +9998,29 @@ def _check_cjk_shape_purity(self) -> None:
"joining the two after the particle, and the leading "
"'van' is a lone particle: one name word before 'Jr.', "
"so #602's run does not start (#620)"),
Case("a_lone_particle_before_the_run_is_the_surname",
"John von PhD Jones",
{"given": "John", "family": "von", "suffix": "PhD Jones"},
ambiguities=("suffix-or-name",),
classification="fix(#627)",
notes="the credential stands right behind the lone particle, "
"leaving it nothing to join, so the particle is the "
"surname and counts toward #602's two name words, as "
"'van der' and 'von und zu' already did; the row above "
"is the contrast, a lone particle with a name word "
"behind it (1.4.0 and 2.3.0 middle 'von PhD')"),
Case("a_particle_run_before_the_run_stays_one_name_word",
"Freiherr von vd PhD Jones",
{"title": "Freiherr", "given": "von vd", "middle": "PhD",
"family": "Jones"},
ambiguities=("particle-or-given",),
classification="parity",
notes="the row above with a particle run: 'von vd' is one "
"name word, 'vd' a particle the run took, so only one "
"name word stands before 'PhD' and #602's run does not "
"start -- the lone-particle count is asked only of a "
"piece that opens a unit, not of 'vd' standing right "
"before the credential (#627's review)"),
Case("a_dual_in_the_chain_stays_a_name_word_behind_a_title",
"John van mc Prof.",
{"title": "Prof.", "given": "John", "family": "van mc"},
Expand Down
Loading
Loading