You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 0aefefe
Browse filesBrowse the repository at this point in the historyBrowse files
Fix what the review of the fix commit found (#596)
- 김゙민준 moves in its report at 2.0.0 too, not only 2.1.0/2.2.0: the
#449 rule joins the 2.0.0 ledger (seven #449 rules there now), and
decisions.md's MEASURED sentence says so; 1.4.0 has no report and
is parity.
- The hangul row is tolerated: rules.md's W Background leaves a mark
after a non-kana unpromised, and a contract row contradicted it. It
moves to the radar corpus; decisions.md says "watches", not "pins".
- Stale "follows a character" in the three fix(#596) ledger comments
and "every voicing mark" in _policy.py now say "after a kana".
- The 147 -> 153 native-script claim bumps carry dated comments, and
the new #449 rule has its _MUST_NOT_MATCH near-misses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/design/decisions.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -469,7 +469,7 @@ Declined:
469
469
- 2026-07-27 (script-scoped order amendment) — the family-first override is keyed to the SCRIPT of the written name, never to a guessed language: wholly-Han, wholly-Hangul, and kana-licensed Japanese read family-first because zh, ko and ja all write family-first in native script; wholly-katakana names are predominantly transcriptions and keep the declared order. Latin transliterations are never touched.
470
470
- 2026-07-29 #272 — the kana license: Han∪kana with at least one kana cannot be Chinese and is not a transcription, so 高橋みなみ reads family-first though it is written in two scripts.
471
471
- 2026-10-03 #594 — HALFWIDTH KATAKANA IS KATAKANA. The script table classifies the halfwidth kana block U+FF65–FF9F as `Script.KATAKANA`, so `山田 タロウ` reads family 山田 as `山田 エミ` does; every release through 2.3.0 left the block unclassified and read it given 山田, family タロウ, the positional reading. Legacy data written for JIS X 0201 systems, which had no kanji, spells names this way — bank, payroll and CSV exports. DECIDED by widening the table rather than by folding: nothing rewrites the text before parsing (rules.md's T Background), so normalization at tokenize was never available, and an NFKC fold at classification — the other place the issue offered — reaches far past the kana (`JOHN` to `JOHN`, `㈱` to `(株)`), and it would move a wholly-katakana name across W4's line: a voicing mark with no precomposed form to join (`ア゙`, `ン゙`) folds to the COMBINING U+3099, which sits in the HIRAGANA block, so the folded token reads as kana-licensed Japanese and `ア゙イ タロウ` would read family-first where the unfolded one keeps the declared order (measured 2026-10-03 by parsing the NFKC spelling `ア゙イ タロウ`, which reads family `ア゙イ`; a two-input sweep of halfwidth names against their NFC(NFKC) spellings differed only where such a mark stood; RECOMPUTE: parse each halfwidth name and `unicodedata.normalize("NFC", unicodedata.normalize("NFKC", name))` and compare the fields). The interpunct flank guard is NOT the hazard, though an earlier wording of this bullet said it was: it reads raw text and asks only whether a character is classified at all, which hiragana and katakana both are. The same measurement shows a pre-existing gap the fold would have widened rather than created: full-width katakana typed with an uncomposable combining U+3099 already takes the kana license today. A block range is how the table already decides (U+30FB and the voicing marks are in by block), and it takes the voicing marks U+FF9E/U+FF9F in because they are spacing characters following their base. The span starts at U+FF65, the halfwidth nakaguro, which tokenize already treats as a separator (rules.md#T2), and stops short of U+FF61–FF64, whose U+FF61 is a full stop. Everything keyed on the table follows: the kana license, the second-East-Asian-word count that keeps a segmenter from being asked (`高橋一郎 タロウ`), the 间隔号's flank guard (`タロウ·ヤマダ` divides), and `_NO_INITIALS`, which closes the limit #cjk-full-stops recorded (`タナカ. John` read title `タナカ.`). THE RATIONALE FOR WHOLLY-KATAKANA NAMES IS AMENDED, NOT THE BEHAVIOR: a wholly-halfwidth name keeps the declared order exactly as a wholly-katakana one does (`ヤマダ タロウ` reads given ヤマダ), but the 2026-07-27 reason — that such names are predominantly transcriptions — is weaker here, since a system with no kanji wrote Japanese names in katakana too, family-first. What stands is the narrower claim: the script cannot tell a Japanese reading from a transcription, so it settles no order, and a caller who knows their katakana is Japanese adds `(Script.KATAKANA, FAMILY_FIRST)` to `script_orders` (docs/usage.rst's Boundaries section carries the recipe). `name_order=FAMILY_FIRST` works too, at the cost of reversing Latin names as well. MEASURED 2026-10-03: the differential gate saw nothing before this change because no corpus line held halfwidth kana; the eight case rows #594 added are its whole population, five of them contract movers (`山田 タロウ`, `山田 ダイスケ`, `山田 ペーター`, the unspaced `山田タロウ`, `タロウ·ヤマダ`), two tolerated radar movers (`タナカ. John`, `マイケル.`) and one parity row (`ヤマダ タロウ`). RECOMPUTE: parse the eight on this tree and on 2.3.0 from PyPI and read `title` beside the name fields.
472
-
- 2026-10-03 #596 — A VOICING MARK TAKES THE SCRIPT OF THE KANA BEFORE IT. The four kana voicing marks U+3099–309C are in the hiragana BLOCK, which is how the script table decides, but none belongs to either syllabary: a mark voices the kana it follows. NFC composes one only where a precomposed voiced kana exists (カ + U+3099 is ガ), so `ア゙イ`, `ン゙` and every spacing ゛/゜ kept a hiragana-block character inside a katakana word, the word took the kana license, and a wholly-katakana name read family-first — the gap the #594 entry above measured, present since 2.1.0. DECIDED by deleting, from the classification copy only, every mark standing after a KANA, so the word reads as its base's script; token text is untouched. A mark after anything else — hangul, Han, Latin, or nothing — voices no kana and keeps the table's answer, as before. That scope is a review finding, not a nicety: the first cut deleted every mark with ANY character before it, which handed `김゙민준` the hangul script, and the surname division, which cuts the raw text, then split the mark off its base — given 김, family `゙민준`, unreported, where 2.3.0 left the word whole and reported given-or-family; `parser_for(locales.ZH)` cut `王゙小明` the same way. A parity case row pins `김゙민준`. Declined: taking the marks out of the hiragana range instead. Measured 2026-10-03 on patched copies of this tree, it fixes the katakana names as well but leaves a word holding a mark in NO script, so `あ゙い たろう` loses family-first, `山田 ア゙イ` loses the kana license and reads family `ア゙イ`, and the unspaced `山田ア゙イ` reads given; the two parity case rows #596 added pin those readings. Declined too: narrowing to the two combining marks, which leaves `ア゛イ タロウ` family-first. NFD-typed text was the issue's worry and is not affected by either option, since classification composes first: NFD `やまだ がくと`, `ジョン ガイ` and `김 민준` parse identically under all of them. A mark with nothing before it keeps the table's answer, so `゙アイ タロウ` still reads family-first; that is degenerate input, recorded in rules.md's W Background as unpromised rather than given a branch. ONE ARGUMENT IN THE #594 ENTRY ABOVE NO LONGER HOLDS: an NFKC fold would no longer move a wholly-katakana name across W4's line, since the combining mark it produces (`ア゙` to `ア゙`) now reads as the katakana it follows — #594's own RECOMPUTE, run 2026-10-03 on this tree over `ア゙イ タロウ`, `ン゙ タロウ`, `山田 ア゙イ`, `マイケル ア゙イ`, `ア゚イ タロウ`, `山田 タロウ` and `ヤマダ タロウ` under all three orders, finds no role differing. The range decision stands on its other ground, the fold's reach past the kana (`JOHN`, `㈱`). The interpunct clause of rules.md#W4 was corrected in the same change: a name the 间隔号 divides keeps the DECLARED order, not its source order — under `name_order=FAMILY_FIRST` `威廉·莎士比亚` reads family 威廉 — and an example pins it. MEASURED 2026-10-03: no corpus line held an uncomposable mark before this change, so the case rows #596 added are the whole population. At 2.1.0, 2.2.0 and 2.3.0 the three `fix(#596)` rows move; at 2.1.0 and 2.2.0 `김゙민준` moves in its report alone, #449's given-or-family being new in 2.3. At 1.4.0 and 2.0.0, which had no kana license, the three movers are parity, while `あ゙い たろう` and `山田 ア゙イ` move as every native-script CJK name does there, under the existing `fix(#271/#272/#298)` rule. RECOMPUTE: parse `ア゙イ タロウ`, `ア゛イ タロウ`, `マイケル ア゙イ`, `あ゙い たろう`, `山田 ア゙イ` and `김゙민준` here and on each baseline from PyPI.
472
+
- 2026-10-03 #596 — A VOICING MARK TAKES THE SCRIPT OF THE KANA BEFORE IT. The four kana voicing marks U+3099–309C are in the hiragana BLOCK, which is how the script table decides, but none belongs to either syllabary: a mark voices the kana it follows. NFC composes one only where a precomposed voiced kana exists (カ + U+3099 is ガ), so `ア゙イ`, `ン゙` and every spacing ゛/゜ kept a hiragana-block character inside a katakana word, the word took the kana license, and a wholly-katakana name read family-first — the gap the #594 entry above measured, present since 2.1.0. DECIDED by deleting, from the classification copy only, every mark standing after a KANA, so the word reads as its base's script; token text is untouched. A mark after anything else — hangul, Han, Latin, or nothing — voices no kana and keeps the table's answer, as before. That scope is a review finding, not a nicety: the first cut deleted every mark with ANY character before it, which handed `김゙민준` the hangul script, and the surname division, which cuts the raw text, then split the mark off its base — given 김, family `゙민준`, unreported, where 2.3.0 left the word whole and reported given-or-family; `parser_for(locales.ZH)` cut `王゙小明` the same way. A case row watches `김゙민준` — marked tolerated, the reading of a mark after a non-kana being unpromised (rules.md's W Background) — so the radar tier, not the contract, sees it move. Declined: taking the marks out of the hiragana range instead. Measured 2026-10-03 on patched copies of this tree, it fixes the katakana names as well but leaves a word holding a mark in NO script, so `あ゙い たろう` loses family-first, `山田 ア゙イ` loses the kana license and reads family `ア゙イ`, and the unspaced `山田ア゙イ` reads given; the two parity case rows #596 added pin those readings. Declined too: narrowing to the two combining marks, which leaves `ア゛イ タロウ` family-first. NFD-typed text was the issue's worry and is not affected by either option, since classification composes first: NFD `やまだ がくと`, `ジョン ガイ` and `김 민준` parse identically under all of them. A mark with nothing before it keeps the table's answer, so `゙アイ タロウ` still reads family-first; that is degenerate input, recorded in rules.md's W Background as unpromised rather than given a branch. ONE ARGUMENT IN THE #594 ENTRY ABOVE NO LONGER HOLDS: an NFKC fold would no longer move a wholly-katakana name across W4's line, since the combining mark it produces (`ア゙` to `ア゙`) now reads as the katakana it follows — #594's own RECOMPUTE, run 2026-10-03 on this tree over `ア゙イ タロウ`, `ン゙ タロウ`, `山田 ア゙イ`, `マイケル ア゙イ`, `ア゚イ タロウ`, `山田 タロウ` and `ヤマダ タロウ` under all three orders, finds no role differing. The range decision stands on its other ground, the fold's reach past the kana (`JOHN`, `㈱`). The interpunct clause of rules.md#W4 was corrected in the same change: a name the 间隔号 divides keeps the DECLARED order, not its source order — under `name_order=FAMILY_FIRST` `威廉·莎士比亚` reads family 威廉 — and an example pins it. MEASURED 2026-10-03: no corpus line held an uncomposable mark before this change, so the case rows #596 added are the whole population. At 2.1.0, 2.2.0 and 2.3.0 the three `fix(#596)` rows move; at 2.0.0, 2.1.0 and 2.2.0 `김゙민준` moves in its report alone, #449's given-or-family being new in 2.3, and is parity at 1.4.0, which has no report. At 1.4.0 and 2.0.0, which had no kana license, the three movers are parity, while `あ゙い たろう` and `山田 ア゙イ` move as every native-script CJK name does there, under the existing `fix(#271/#272/#298)` rule. RECOMPUTE: parse `ア゙イ タロウ`, `ア゛イ タロウ`, `マイケル ア゙イ`, `あ゙い たろう`, `山田 ア゙イ` and `김゙민준` here and on each baseline from PyPI.
Copy file name to clipboardExpand all lines: tools/differential/expected_since_2.0.0.toml
+12-1Lines changed: 12 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -91,7 +91,7 @@ issue = "fix(#436/#437) a space-separated post-nominal run renders with spaces,
91
91
name_regex = "^(?:JOHN DOE PHD MD|Jane Doe Jr\\. Ma|Jane Doe nee Puig i III|Jane Doe nee Smith PhD MA|John Doe MD PhD|John Smith MD PhD|John Smith Mc V|John Smith PhD Ed Ma|John Smith PhD Ma|Josep Lluis Carod i III|Josep Lluis Carod i V|Kenneth Clarke QC MP|Rovira, Josep Carod i Jr\\.|Smith, John PhD I\\.|The Rt Hon Kenneth Clarke QC MP, HMG|Washington Jr\\. MD, Franklin|abdul Smith Jr Ma|abdul Smith Jr V)$"
92
92
fields = ["suffix"]
93
93
94
-
# The six #449 rules go SECOND, not first: the rule above
94
+
# The seven #449 rules go SECOND, not first: the rule above
95
95
# declares its own first-in-file position with a measurement,
96
96
# and `fields = ["suffix"]` and `fields = ["_ambiguities"]`
97
97
# are disjoint, so nothing orders the two against each other.
@@ -187,6 +187,17 @@ issue = "feat(#449) a glued Korean honorific leaves a Latin name word the script
187
187
name_regex = "^Anderson선생님$"
188
188
fields = ["_ambiguities", "given", "suffix"]
189
189
190
+
[[change]]
191
+
issue = "feat(#449) a hangul word holding a stray voicing mark has no script, so the convention decides it"
192
+
# '김゙민준' (김 + U+3099), a tolerated case row #596 added (2026-10-03).
193
+
# A kana voicing mark after hangul voices no kana, so it keeps its
194
+
# hiragana block and the token is mixed hangul and hiragana: no
195
+
# script, no surname division, and O5's convention places the one name
196
+
# word left. The roles are this baseline's; the report is the whole
197
+
# diff. Literal-anchored, one corpus name, `_ambiguities` alone.
198
+
name_regex = "^김\\u3099민준$"
199
+
fields = ["_ambiguities"]
200
+
190
201
# The two #491 rules sit beside #449's, same position rule: both
191
202
# declare `fields = ["_ambiguities"]`, so they are disjoint from the
192
203
# `suffix` rule that owns first place and ahead of every rule below
0 commit comments