Stop the chain before a trailing title after a family comma (#623)

docs/design/rules.md

@@ -333,14 +333,19 @@ H5. Rationale: a word abbreviated with a period at the END of a name
     particle chain is made, so a particle chain (P2) stops in front
     of a trailing title rather than taking it into the family name.
     A title word a connective joined into a longer unit is no single
-    word for the chain to take. After a family comma the given part
-    is read once all its words are joined, so there a particle chain
-    takes a trailing title as before. A maiden clause ends where the run the end of the name
+    word for the chain to take. After a family comma the given part's
+    chain stops in front of the trailing run of suffixes and titles,
+    read as the comma-less name reads it, so there too a trailing
+    title is the title and P6's attachment reads the particle in
+    front of it as though the title were absent. A maiden clause ends where the run the end of the name
     reads without it begins (M2), so where a trailing rule reads the
     part a trailing title ends the clause and is a title.
       "John van der Berg Prof."   →  title="Prof."
       "John van der Berg Prof."   →  family="van der Berg"
       "Mary Smith née Jones Prof." →  title="Prof."
+      "Smith, John van der Berg Prof." →  title="Prof."
+      "Smith, John van der Berg Prof." →  middle="van der Berg"
+      "Smith, Jan van Prof."      →  family="van Smith"
     Accepted: what the chain leaves is also what counts as a name
     word to spare (P5). A trailing title word is not one, so a bound
     given-name word behind one joins exactly as it joins with the
@@ -354,7 +359,7 @@ H5. Rationale: a word abbreviated with a period at the END of a name
     gap in the segment path rather than a boundary of this rule, and
     tracked as part of #316. Pinned by a case row rather than by an
     example here, so recording the gap does not make it normative.
-    history: decisions.md#H5 · interacts: H1, H2, H3, H4, M2, P2, P5, S2, C1 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_pieces.py
+    history: decisions.md#H5 · interacts: H1, H2, H3, H4, M2, P2, P5, P6, S2, C1 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_group.py, nameparser/_pipeline/_pieces.py
 
 ## Particles & surname prefixes (P)
 
@@ -976,7 +981,7 @@ P6. Rationale: a particle ending the name has nothing to link
     negative-control sweep pinning the disagreeing set the precedence
     bullet above names. A change that breaks one side of that pair
     should expect that test, not this file, to say so first.
-    history: decisions.md#P6 · interacts: A1, C1, P1, S2, P5, M2 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py
+    history: decisions.md#P6 · interacts: A1, C1, H5, P1, S2, P5, M2 · implemented: nameparser/_pipeline/_assign.py, nameparser/_pipeline/_pieces.py, nameparser/_pipeline/_post_rules.py
 
 P7. Rationale: a one-letter particle is spelled with the same letter
     as an initial, and the period is what tells them apart. An

docs/release_log.rst

@@ -22,7 +22,7 @@ Release Log
 
     - **Fix a credential run after a comma losing the name in front of it when two surname particles stand side by side in the run.** ``HumanName("John Smith, PhD DO DO")`` gives first ``John``, last ``Smith``, suffix ``PhD DO DO``, where 2.2 and 2.3 gave first ``PhD``, last ``DO DO John Smith`` and 2.0 and 2.1 gave title ``PhD``, first ``DO DO`` -- 1.4.0's reading, restored. ``DO``, ``MC`` and ``VD`` are credentials and surname particles at once, and two particles next to each other had joined into one particle run that took the credentials apart; the part after the comma is now read as credentials before any particle can join it. ``John Smith, PhD vd DO`` gives suffix ``PhD vd DO`` the same way, and ``John Smith, MD DO DO`` gives suffix ``MD DO DO`` where 2.3 gave title ``MD``, first ``DO``, middle ``DO``. A single ``DO`` is still left to its capitals (``John Smith, PhD DO`` gives suffix ``PhD DO``), and with one name word before the comma a part of credentials is all suffixes (``Smith, PhD DO DO`` gives last ``Smith``, suffix ``PhD DO DO``, and ``Smith, MA DO DO`` last ``Smith``, suffix ``MA DO DO``, where 2.3 gave first ``PhD`` and first ``MA``, last ``DO DO Smith``). See the #562 entry under ``C1`` in ``docs/design/decisions.md`` (closes #562)
 
-    - **Fix a trailing title being taken into a surname that begins with a particle.** ``HumanName("John van der Berg Prof.")`` gives title ``Prof.``, last ``van der Berg``, where every release from 1.4.0 through 2.3.0 gave last ``van der Berg Prof.``. The suffixes and titles at the end of a name are now read before the surname's particles join the words after them, so the particles no longer take a title in. Behind a family comma the given part is read as before (``HumanName("Smith, John van der Berg Prof.")`` keeps middle ``van der Berg Prof.``). The same reading order brings back 1.4.0's ``HumanName("abdul J. V")``: first ``abdul J.``, last ``V``, where 2.2 and 2.3 gave first ``abdul``, middle ``J.``. See the #614 entry under ``S2`` in ``docs/design/decisions.md`` (closes #614)
+    - **Fix a trailing title being taken into a surname that begins with a particle.** ``HumanName("John van der Berg Prof.")`` gives title ``Prof.``, last ``van der Berg``, where every release from 1.4.0 through 2.3.0 gave last ``van der Berg Prof.``. The suffixes and titles at the end of a name are now read before the surname's particles join the words after them, so the particles no longer take a title in. The given part behind a family comma reads the same way: ``HumanName("Smith, John van der Berg Prof.")`` gives title ``Prof.``, middle ``van der Berg``, and ``Smith, Jan van Prof.`` gives title ``Prof.``, last ``van Smith``, as ``Smith, Jan van`` gives last ``van Smith``, where 1.4.0 through 2.3.0 gave middle ``van der Berg Prof.`` and ``van Prof.``. The same reading order brings back 1.4.0's ``HumanName("abdul J. V")``: first ``abdul J.``, last ``V``, where 2.2 and 2.3 gave first ``abdul``, middle ``J.``. See the #614 entry under ``S2`` and the #623 entry under ``H5`` in ``docs/design/decisions.md`` (closes #614, #623)
 
     - **New Policy field unlisted_dotted_suffixes, on by default: a dotted acronym nobody has listed is read by position.** ``HumanName("John Smith X.Y.Z.")`` gives suffix ``X.Y.Z.`` where every release gave last ``X.Y.Z.``, while ``Jack X.Y.Z.`` keeps its surname, the same words-to-spare rule a listed acronym takes -- and both readings are reported. After a comma the count is of the words before it, and two dotted single letters are the exception: they are how a person's initials are written, and two words before a comma may be one surname, so ``García Márquez, G.J.`` keeps first ``G.J.`` and last ``García Márquez`` and reports the fork, unless an unambiguous post-nominal in front of the initials that is not also a title, or another unlisted dotted word beside them, says otherwise (``John Smith, PhD X.Y.`` gives suffix ``PhD X.Y.``, while ``García Márquez, Ms G.J.`` keeps title ``Ms``, first ``G.J.``). Three letters or more read by the count, so ``John Smith, X.Y.Z.`` gives suffix ``X.Y.Z.`` -- and so does ``García Márquez, G.J.R.``, the accepted cost of the line, since initials are conventionally written apart (``García Márquez, G. J. R.``), as separate words this rule does not read (#563). Case is irrelevant here: the periods are the signal, so ``john smith x.y.z.`` reads the same way. Words the vocabulary does know are untouched (``M.A.``, ``Ph.D.``, ``A.B.C.``), a single trailing period is still not this shape (``John Smith Xyz.`` keeps last ``Xyz.``), and a dotted run at the FRONT of a name is untouched (``J.R.R. Tolkien``). One accident retires with it: a dotted word whose only vocabulary matches were SINGLE ASCII CHARACTERS -- the roman numerals the suffix list holds, and the lone digit ``2`` -- was reading as a generational suffix, so ``Jack X.Y.I.`` gives last ``X.Y.I.`` again, as 1.4.0 read it, while ``Msc.Ed.``, ``JD.CPA`` and ``Lt.Gov.`` are unchanged. The digit is why a dotted VERSION STRING moves with them and moves SILENTLY: ``John Smith 1.4.2`` gives last ``1.4.2`` where 2.3 gave suffix ``1.4.2``, and ``John Smith, 1.4.2`` gives first ``1.4.2``, last ``John Smith``. Such a token reports nothing at any policy -- it is no acronym either, the shape reading wanting every chunk alphabetic -- and a version string read as a credential was the same accident this retirement removes. That retirement is NOT behind this switch and stands either way -- setting it to ``False`` reads an unlisted dotted word as name material by position instead (``John Smith X.Y.Z.`` keeps last ``X.Y.Z.``), the pre-2.4 reading for THAT half alone. See the ``S2`` and ``suffix-acronym-collisions`` entries of ``docs/design/decisions.md`` (closes #516)
 

nameparser/_pipeline/_group.py

@@ -66,7 +66,8 @@
     Peel, TailRead, chain_lead, chain_run_end, is_conj_piece, is_leading_title, is_prefix_piece,
     is_suffix_piece, is_title_piece, join_connectives, joined_tags,
     leading_titles, merge_particle_runs, merge_pieces, peel_walk,
-    read_trailing_run, tail_reading, trailing_candidates, trailing_start,
+    read_trailing_run, tail_reading, trailing_candidates,
+    trailing_run_start, trailing_start,
     trailing_start_past_titles,
 )
 from nameparser._pipeline._state import (
@@ -805,8 +806,10 @@ def _group_segment(seg: tuple[int, ...], additional: int,
         # suffix" -- on a main segment the run read above decides that
         # and is split off already; elsewhere group does not decide
         # it, and stops before whatever
-        # trailing_start says the run is, so the chain ends where assign's
-        # peel begins (#424). The maiden take reads the title-aware
+        # trailing_start says the run is (the given part after a family
+        # comma, `trailing_run_start`, its trailing titles included:
+        # #623, below), so the chain ends where assign's peel begins
+        # (#424). The maiden take reads the title-aware
         # `tail_reading` over the clause-free view instead (#601), so the
         # clause ends where assign's peel and title chain together begin.
         # rules.md#P2: "a particle joins the words after it into one name
@@ -903,10 +906,32 @@ def _group_segment(seg: tuple[int, ...], additional: int,
         # asks again after its merges whether the acronym still has
         # the pieces the fork counted (below).
         # the run read above is already split off (#614), so the chain
-        # stops at the end of what is left
-        tail = (0 if read is not None
-                else len(pieces) - trailing_start(name_start, pieces, ptags,
-                                                  tokens, one_case=one_case))
+        # stops at the end of what is left.
+        # rules.md#H5: "the title is TRANSPARENT to the suffix reading"
+        # -- in the given part after a family comma too, where assign's
+        # walk reads the trailing title run, so the chain stops before
+        # the run of suffixes and titles `tail_reading` takes there
+        # rather than before the suffixes alone, and does not carry a
+        # trailing title into the particle's name ('Smith, John van der
+        # Berg Prof.' reads title 'Prof.' as 'Smith, John Prof.' does,
+        # #623). The bound matters only where a particle stands past the
+        # chain's lead, so it is not asked otherwise: the chain then
+        # merges nothing (the test is `is_prefix_piece`'s, inline).
+        chains = False
+        if read is None:
+            for k in range(leading + 1, len(pieces)):
+                if ("prefix" in ptags[k]
+                        or len(pieces[k]) == 1
+                        and "particle" in tokens[pieces[k][0]].tags):
+                    chains = True
+                    break
+        given_slot = site is ClauseSite.GIVEN_SLOT
+        tail = (0 if not chains
+                else len(pieces) - (
+                    trailing_run_start(name_start, pieces, ptags, tokens,
+                                       one_case) if given_slot
+                    else trailing_start(name_start, pieces, ptags, tokens,
+                                        one_case=one_case)))
         # the members the chain reports below, which the read's picks
         # then give up (#614)
         took: set[tuple[int, ...]] = set()
@@ -1018,7 +1043,9 @@ def chain(tail: int) -> None:
                 #
                 # The DECLINED reading is what this reports, because
                 # this branch only runs over pieces the peel left
-                # standing: `tail` is the peel's own answer, the inner
+                # standing: `tail` is the peel's own answer (with the
+                # trailing titles in the given part after a family
+                # comma, #623), the inner
                 # scan stops at `len(pieces) - tail`, and a piece the
                 # peel TOOK is behind that bound. Where the chain runs
                 # again with a smaller tail (the re-peel below), the
@@ -1077,10 +1104,12 @@ def chain(tail: int) -> None:
                 k += 1
 
         # Where the run was read above (#614) it is split off and
-        # `tail` is 0, so none of what follows runs; it is the path for
-        # the segments that read no run above -- a family comma's
+        # `tail` is 0, so none of what follows runs, as it does not
+        # where no particle stands past the lead (#623); it is the path
+        # for the segments that read no run above -- a family comma's
         # parts, a part past the second comma, a run with a name piece
-        # inside it.
+        # inside it -- and the given part reaches it still
+        # ('Smith, Freiherr van Berg X.Y.Z.', the row that holds it).
         #
         # The peel was read over the pieces as they stand, and the
         # chain's own merges can change what it counts: behind a word
@@ -1102,9 +1131,12 @@ def chain(tail: int) -> None:
             kept = ([list(q) for q in pieces], [set(t) for t in ptags],
                     len(ambiguities))
             chain(tail)
-            left = len(pieces) - trailing_start(
-                leading_titles(pieces, ptags, tokens), pieces, ptags,
-                tokens, one_case=one_case)
+            start = leading_titles(pieces, ptags, tokens)
+            left = len(pieces) - (
+                trailing_run_start(start, pieces, ptags, tokens, one_case)
+                if given_slot
+                else trailing_start(start, pieces, ptags, tokens,
+                                    one_case=one_case))
             if left < tail:
                 pieces[:], ptags[:] = kept[0], kept[1]
                 del ambiguities[kept[2]:]

nameparser/_pipeline/_pieces.py

@@ -953,7 +953,9 @@ def trailing_start(start: int, pieces: Sequence[Sequence[int]],
     as they stand from `start`: the index of the first piece the S2
     peel takes, or len(pieces) when it takes none (#424). What P2's
     chain stops before where S2's run was not read once ahead of it
-    (#614: a family comma's parts, a run holding a name piece), and
+    (#614: the part before a family comma, a part past the second
+    comma, a run holding a name piece; the given part after a family
+    comma stops at `trailing_run_start` since #623), and
     what M2's walk stops before where no trailing rule reads the
     clause -- each had asked "is this a
     suffix?" with the suffix-piece test, which vetoes a bare 'V' as
@@ -1896,6 +1898,27 @@ def read_trailing_run(pieces: Sequence[Sequence[int]],
                     frozenset(run_titles), peel), start
 
 
+def trailing_run_start(start: int, pieces: Sequence[Sequence[int]],
+                       ptags: Sequence[Set[str]],
+                       tokens: Sequence[WorkToken],
+                       one_case: bool | None) -> int:
+    """Where the run `tail_reading` takes over `pieces[start:]` begins,
+    its trailing titles included: the first index of a piece the peel
+    or the H5 chain took, `len(pieces)` where they take none. The
+    bound for a reading that takes the trailing titles as well as the
+    suffixes, as assign's walk does in the given part after a family
+    comma (#623), where `trailing_start` sees only the suffixes and
+    `trailing_start_past_titles` leaves the titles inside the bound."""
+    rest, titled, peel = tail_reading(peel_walk(start, ptags), pieces,
+                                      ptags, tokens, one_case)
+    first = len(pieces)
+    for k in rest[peel.names:]:
+        first = min(first, k)
+    for k in titled:
+        first = min(first, k)
+    return first
+
+
 # rules.md#H5: "the title is TRANSPARENT to the suffix reading: where
 # two or more name words stand, what stands once the chain is taken
 # reads exactly as it would read written without the title, plus the