Skip to content

Commit a16927c

Browse files
derek73claude
andcommitted
fix(S2): #564 -- the name's contrast is a capital followed by lowercase (Derek)
Title case (`istitle()`) read too narrowly: interior capitals and elisions ('DiCaprio', 'IJzerman', 'al-Rashid', "d'Estaing", 'McDonald') are mixed-case names and kept a given 'XYZ'. Derek's criterion: a word of the name carries the contrast if it holds a capital directly followed by a lowercase letter, a leading Mc/Mac skipped where a capital follows it (_vocab.written_as_a_name). A lowercase prefix glued to a surname written in capitals ("d'ESTAING", 'al-ASSAD', 'McDONALD') has no such pair, so those records keep their given names; a lone capital and a lowercase-only word have none either. Weighed and declined, with measurements in decisions.md#S2: "contains a capital" (any lowercase word in an all-caps record then makes every word a contrast) and "not all capitals but contains a capital" (takes d'ESTAING and al-ASSAD). The accepted costs shrink to all-lowercase names and unlisted mixed-case titles. A parametrized test pins the predicate; case rows pin McDONALD and DiCaprio. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 3e03c08 commit a16927c

10 files changed

Lines changed: 96 additions & 39 deletions

File tree

‎docs/customize.rst‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -532,7 +532,8 @@ listed below.
532532
- ``CapsSuffixes``
533533
- Where an unlisted all-caps word of two or more letters, with no
534534
period in it, reads as a credential. The name must contrast it
535-
with a word written in Title case (``Smith``) that the vocabulary
535+
with a word holding a capital followed by a lowercase letter
536+
(``Smith``, ``DiCaprio``) that the vocabulary
536537
does not claim as a title, particle or credential: a record
537538
written wholly in capitals, or wholly in lowercase, keeps every
538539
word a name word. ``CapsSuffixes.AFTER_COMMA``, the default, reads

‎docs/design/decisions.md‎

Lines changed: 2 additions & 2 deletions
Large diffs are not rendered by default.

‎docs/design/rules.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1724,8 +1724,10 @@ C1. Rationale: a credential run after the comma means the name is in
17241724
little else speaks for them, since no one writes a person's
17251725
initials as two dotted groups. An unlisted all-caps word joins
17261726
the class in such a part only where the name carries the
1727-
contrast: one of its own words before the comma written in Title
1728-
case, the way a name is written (S2), with no period and not
1727+
contrast: one of its own words before the comma written the way a
1728+
name is written in mixed case, a capital directly followed by a
1729+
lowercase letter (a leading Mc or Mac before a capital aside),
1730+
with no period and not
17291731
claimed by the vocabulary as a title, particle, connective,
17301732
credential or generation. A word written wholly in lowercase never
17311733
carries it, so a record that writes its surname in capitals keeps

‎docs/release_log.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Release Log
2222

2323
- **New Policy field unlisted_dotted_suffixes, on by default: a dotted acronym nobody has listed is read by position.** ``HumanName("John Smith X.Y.Z.")`` gives suffix ``X.Y.Z.`` where every release gave last ``X.Y.Z.``, while ``Jack X.Y.Z.`` keeps its surname, the same words-to-spare rule a listed acronym takes -- and both readings are reported. After a comma the count is of the words before it, and two dotted single letters are the exception: they are how a person's initials are written, and two words before a comma may be one surname, so ``García Márquez, G.J.`` keeps first ``G.J.`` and last ``García Márquez`` and reports the fork, unless an unambiguous post-nominal in front of the initials that is not also a title, or another unlisted dotted word beside them, says otherwise (``John Smith, PhD X.Y.`` gives suffix ``PhD X.Y.``, while ``García Márquez, Ms G.J.`` keeps title ``Ms``, first ``G.J.``). Three letters or more read by the count, so ``John Smith, X.Y.Z.`` gives suffix ``X.Y.Z.`` -- and so does ``García Márquez, G.J.R.``, the accepted cost of the line, since initials are conventionally written apart (``García Márquez, G. J. R.``), as separate words this rule does not read (#563). Case is irrelevant here: the periods are the signal, so ``john smith x.y.z.`` reads the same way. Words the vocabulary does know are untouched (``M.A.``, ``Ph.D.``, ``A.B.C.``), a single trailing period is still not this shape (``John Smith Xyz.`` keeps last ``Xyz.``), and a dotted run at the FRONT of a name is untouched (``J.R.R. Tolkien``). One accident retires with it: a dotted word whose only vocabulary matches were SINGLE ASCII CHARACTERS -- the roman numerals the suffix list holds, and the lone digit ``2`` -- was reading as a generational suffix, so ``Jack X.Y.I.`` gives last ``X.Y.I.`` again, as 1.4.0 read it, while ``Msc.Ed.``, ``JD.CPA`` and ``Lt.Gov.`` are unchanged. The digit is why a dotted VERSION STRING moves with them and moves SILENTLY: ``John Smith 1.4.2`` gives last ``1.4.2`` where 2.3 gave suffix ``1.4.2``, and ``John Smith, 1.4.2`` gives first ``1.4.2``, last ``John Smith``. Such a token reports nothing at any policy -- it is no acronym either, the shape reading wanting every chunk alphabetic -- and a version string read as a credential was the same accident this retirement removes. That retirement is NOT behind this switch and stands either way -- setting it to ``False`` reads an unlisted dotted word as name material by position instead (``John Smith X.Y.Z.`` keeps last ``X.Y.Z.``), the pre-2.4 reading for THAT half alone. See the ``S2`` and ``suffix-acronym-collisions`` entries of ``docs/design/decisions.md`` (closes #516)
2424

25-
- **New Policy field unlisted_caps_suffixes: an unlisted all-caps word reads as a credential after a comma by default, and elsewhere on request.** Its value is a ``CapsSuffixes``. The default, ``CapsSuffixes.AFTER_COMMA``, reads such a word in the part right after a comma behind two or more name words, alone or in a run with other credentials: ``HumanName("John Smith, XYZ")`` gives first ``John``, last ``Smith``, suffix ``XYZ``, where 1.4.0 through 2.3.0 gave first ``XYZ``, last ``John Smith``; ``John Smith, LEED AP`` and ``John Smith, PhD XYZ`` give suffix ``LEED AP`` and ``PhD XYZ`` the same way, and ``John Smith, RAI`` gives suffix ``RAI`` again, as it did before 2.3. The all-caps surname convention writes the capitals at the end of a name or before a comma (``Jean DUPONT``, ``DUPONT, Jean``) and never there. A word after a one-word surname stays the given name (``Smith, XYZ``), a two-letter word reads exactly as dotted initials do (``García Márquez, MJ`` and ``García Márquez, MJ PhD`` keep first ``MJ``), and the name has to contrast the capitals with a word of its own written in Title case, so a record that writes its surname in capitals keeps its given name whatever it writes in lowercase (``GISCARD d'ESTAING, VALÉRY``, ``LLOYD WEBBER, ANDREW PhD``), as does a name written wholly in lowercase. ``CapsSuffixes.EVERYWHERE`` also reads the end of a name, the given part's last word after a family comma and the word ending a maiden marker's clause: ``.parse("John Smith XYZ")`` gives suffix ``XYZ``, and ``Jean Pierre DUPONT`` gives last ``Pierre``, suffix ``DUPONT`` -- why it is not the default. ``CapsSuffixes.OFF`` reads none of them and reports nothing; it is the way to keep a given name written in capitals after a two-word surname, which the default reads as a credential (``García Márquez, GABRIEL`` gives suffix ``GABRIEL``). The field reaches the core parser only, through ``Parser(policy=Policy(unlisted_caps_suffixes=...))``; a ``HumanName`` tracks the parser's defaults, so the comma reading reaches it and the other two settings cannot be chosen from there. Neither this field nor ``unlisted_dotted_suffixes`` has a v1 ``Constants`` manager. See the ``S2`` and ``C1`` entries of ``docs/design/decisions.md`` (closes #516, closes #564)
25+
- **New Policy field unlisted_caps_suffixes: an unlisted all-caps word reads as a credential after a comma by default, and elsewhere on request.** Its value is a ``CapsSuffixes``. The default, ``CapsSuffixes.AFTER_COMMA``, reads such a word in the part right after a comma behind two or more name words, alone or in a run with other credentials: ``HumanName("John Smith, XYZ")`` gives first ``John``, last ``Smith``, suffix ``XYZ``, where 1.4.0 through 2.3.0 gave first ``XYZ``, last ``John Smith``; ``John Smith, LEED AP`` and ``John Smith, PhD XYZ`` give suffix ``LEED AP`` and ``PhD XYZ`` the same way, and ``John Smith, RAI`` gives suffix ``RAI`` again, as it did before 2.3. The all-caps surname convention writes the capitals at the end of a name or before a comma (``Jean DUPONT``, ``DUPONT, Jean``) and never there. A word after a one-word surname stays the given name (``Smith, XYZ``), a two-letter word reads exactly as dotted initials do (``García Márquez, MJ`` and ``García Márquez, MJ PhD`` keep first ``MJ``), and the name has to contrast the capitals with a word of its own holding a capital followed by a lowercase letter (``Smith``, ``DiCaprio``), so a record that writes its surname in capitals keeps its given name whatever it writes beside it (``GISCARD d'ESTAING, VALÉRY``, ``LLOYD McDONALD, RONALD``, ``LLOYD WEBBER, ANDREW PhD``), as does a name written wholly in lowercase. ``CapsSuffixes.EVERYWHERE`` also reads the end of a name, the given part's last word after a family comma and the word ending a maiden marker's clause: ``.parse("John Smith XYZ")`` gives suffix ``XYZ``, and ``Jean Pierre DUPONT`` gives last ``Pierre``, suffix ``DUPONT`` -- why it is not the default. ``CapsSuffixes.OFF`` reads none of them and reports nothing; it is the way to keep a given name written in capitals after a two-word surname, which the default reads as a credential (``García Márquez, GABRIEL`` gives suffix ``GABRIEL``). The field reaches the core parser only, through ``Parser(policy=Policy(unlisted_caps_suffixes=...))``; a ``HumanName`` tracks the parser's defaults, so the comma reading reaches it and the other two settings cannot be chosen from there. Neither this field nor ``unlisted_dotted_suffixes`` has a v1 ``Constants`` manager. See the ``S2`` and ``C1`` entries of ``docs/design/decisions.md`` (closes #516, closes #564)
2626

2727
- **The comma's own decision about an ambiguous credential is now reported.** ``parse("Smith, MA").ambiguities`` names ``suffix-or-name``, and so does every other decision at the ambiguous credential class -- before or after a comma, in either direction, with no new ``AmbiguityKind`` (the family-comma attachment fork already reported this way, e.g. ``parse("Berg, Jan vd")``). A flip of the comma in which no listed ambiguous acronym takes part is the exception and is made in silence: ``John Smith, X.Y.Z.`` and ``John Smith, PhD X.Y.`` report nothing, the only such word a reader takes for a name being a pair of initials, which the comma reads as the given name unless something beside it has already said otherwise. Two pairs speaking only for each other still make the credential run, and that flip reports: ``John Smith, X.Y. P.Q.`` gives last ``Smith``, suffix ``X.Y. P.Q.`` (#563). One report per decision: ``Smith, Ma`` reports that the word was kept as the given name just as ``Smith, MA`` reports that it was taken as a credential. The reading a SURNAME PARTICLE swallows is reported too, which no release before this one did: ``John van der Berg Ma`` gives last ``van der Berg Ma`` and names ``suffix-or-name``, where the chain took a word the credential reading had considered. ONE report goes away, because a comma segment the parser reads as a credential run is no longer called unrecognized: ``Steven Hardman, MD, DO, DDS`` no longer reports ``comma-structure``, on its written case. That is the whole of the losses over the differential corpora -- ``John Smith, MD, R.A.I.`` is quieted on its shape by the same change, but it never reported at 2.3.0 either, having only carried the flag inside this release's own development. The other movement an upgrader sees is a SWAP rather than a loss: ``Jack X.Y.I.`` reported ``given-or-family`` at 2.3.0 and reports ``suffix-or-name`` here, the dotted retirement above having handed it to the ambiguous class. Everything else at this class is a GAIN, which is what the rest of this bullet describes. Two slots this bullet left silent no longer are, and the two bullets below close them: a credential trailing the GIVEN part of a family-comma listing now reads as a credential and reports either way, and so does one ending a maiden marker's clause. See the ``S2`` and ``C1`` entries of ``docs/design/decisions.md``
2828

‎nameparser/_pipeline/_segment.py‎

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@
5757
ambiguous_class_candidate, ambiguous_class_member, ambiguous_lean,
5858
caps_shape_candidate, is_one_case, is_paired_initials,
5959
claimed_as_non_name, is_single_letter_numeral, is_wholly_suffix,
60+
written_as_a_name,
6061
name_word_count,
6162
surname_unit_count,
6263
run_word_fold,
@@ -126,21 +127,22 @@ def texts(seg: tuple[int, ...]) -> list[str]:
126127
# #564 (Derek): the contrast the caps shape needs is the NAME's,
127128
# and the name carries it only if one of its own words before the
128129
# comma (no maiden clause, no delimited content, as `own_words`
129-
# defines them for `case_class` above) is written in Title case --
130-
# S2's "written the way a name is written" -- and is not claimed as
131-
# a title, particle, connective, credential or generation
130+
# defines them for `case_class` above) is written the way a name is
131+
# written in mixed case -- a capital directly followed by a
132+
# lowercase letter (`written_as_a_name`) -- and is not claimed as a
133+
# title, particle, connective, credential or generation
132134
# (`claimed_as_non_name`); a word with a period is an abbreviation
133-
# or an initial, not one, and nor is a lone capital ('de GAULLE C'),
134-
# which `istitle()` alone accepts -- Title case needs a lowercase
135-
# letter behind the capital. A lowercase-only word never counts, so a
136-
# record that writes its surname in capitals keeps its given name
137-
# whatever else it writes in lowercase ('LLOYD ap RHYS', 'HAFEZ
138-
# al-ASSAD', "GISCARD d'ESTAING", 'LLOYD WEBBER née Smith'), and
139-
# so does a mixed-case credential ('LLOYD WEBBER, ANDREW PhD').
135+
# or an initial, not one. A lone capital ('de GAULLE C') or a
136+
# lowercase-only word has no such pair, and nor has a lowercase
137+
# prefix glued to a capitalized surname, so a record that writes
138+
# its surname in capitals keeps its given name whatever else it
139+
# writes ('LLOYD ap RHYS', 'HAFEZ al-ASSAD', "GISCARD d'ESTAING",
140+
# 'LLOYD McDONALD', 'LLOYD WEBBER née Smith'), and so does a
141+
# mixed-case credential ('LLOYD WEBBER, ANDREW PhD').
140142
# Asked only once a caps word is in hand: a part with no lowercase
141143
# at all is settled in one C-level comparison; past that the walk
142-
# costs about three frames a word before the comma, a fold and a
143-
# wordlist test only where the Title-case test in C passes.
144+
# costs a few frames a word before the comma, a fold and a
145+
# wordlist test only where the case test passes.
144146
def name_contrast() -> bool:
145147
before = "".join([state.tokens[i].text for i in groups[0]])
146148
if before == before.upper():
@@ -153,8 +155,7 @@ def name_contrast() -> bool:
153155
lex = state.lexicon
154156
return any(
155157
i < clause_at and (tok := state.tokens[i]).role is None
156-
and tok.text.istitle() and not tok.text.isupper()
157-
and "." not in tok.text
158+
and "." not in tok.text and written_as_a_name(tok.text)
158159
and not claimed_as_non_name(_normalize(tok.text), lex)
159160
for i in groups[0])
160161

‎nameparser/_pipeline/_vocab.py‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -720,6 +720,22 @@ def in_any_wordlist(n: str, lexicon: Lexicon) -> bool:
720720
if f not in ("surnames", "bound_given_names"))
721721

722722

723+
def written_as_a_name(text: str) -> bool:
724+
"""Whether TEXT is written the way a name is written in mixed case:
725+
a capital directly followed by a lowercase letter ('Smith',
726+
'DiCaprio', 'IJzerman', 'al-Rashid', "d'Estaing") -- #564's test
727+
for the name's case contrast (Derek). A word written wholly in one
728+
case has no such pair, and nor does a lowercase prefix glued to a
729+
surname written in capitals ("d'ESTAING", 'al-ASSAD'). A leading
730+
'Mc'/'Mac' is skipped where a capital follows it, so 'McDONALD'
731+
has no pair while 'McDonald' keeps 'Do' and 'Mack' its own 'Ma'."""
732+
if text.startswith("Mc") and text[2:3].isupper():
733+
text = text[2:]
734+
elif text.startswith("Mac") and text[3:4].isupper():
735+
text = text[3:]
736+
return any(a.isupper() and b.islower() for a, b in zip(text, text[1:]))
737+
738+
723739
def claimed_as_non_name(n: str, lexicon: Lexicon) -> bool:
724740
"""Whether a wordlist claims the folded word `n` as a title,
725741
particle, connective, credential, generation, maiden marker or

‎nameparser/_policy.py‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -709,8 +709,9 @@ class Policy:
709709
unlisted_dotted_suffixes: bool = True
710710
#: Where an UNLISTED all-caps word of two or more letters, with no
711711
#: period in it, reads as a credential (:class:`CapsSuffixes`). The
712-
#: name must contrast it: a word of the name written in Title case
713-
#: ("Smith") that the vocabulary does not claim as a title, particle
712+
#: name must contrast it: a word of the name with a capital followed
713+
#: by a lowercase letter ("Smith", "DiCaprio") that the vocabulary
714+
#: does not claim as a title, particle
714715
#: or credential -- a record written wholly in capitals or wholly in
715716
#: lowercase keeps every word a name word. A listed member keeps its
716717
#: own case lean in every setting ("Jack MA" gives suffix ``MA``),

0 commit comments

Comments
 (0)