[ilien-dev]

Version 1.4.0 measures the skill against the models people use in November 2026 and adds four registers. The checklist and the meter change. The rewrite instructions do not, because the three new ones tried here did not clear the noise on held-out titles.

New assistant texts

The September texts in eval/ai/ have no record of which model version wrote them. The same titles were written again by Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 and GPT-6.1 for training, and by Opus 5.5, Sonnet 5.5, Fable 5.2, GPT-6.1 and GPT-6.1 Terra on the held-out titles: 282 texts in blog, fiction and Spanish. eval/gensamples.py writes them and MANIFEST.tsv records model, date and prompt for each.

On those texts the checklist catches less of each new Claude model. It failed every Sonnet blog post from September and now passes 79% of the Sonnet 5.5 ones clean, 79% of Opus 5.5 and 93% of Fable 5.2. Dashes are no longer a Claude signal: 7% to 29% of the new Claude posts have one, against 22% of held-out human posts.

What changed in the meter and the checklist

• The band files take their AI medians and directions from both months. The human edges are the same. In blog six directions that no longer hold are dropped, the dash among them. In Spanish, human posts failing C1 go from 7% to 5%.
• Four words pass the lexicon rule on the November posts: practical, quietly, deliberate, preserve. B1 now fails 14% of held-out Opus 5.5 posts (7% before), 29% of Sonnet 5.5 (14%) and 36% of GPT-6.1 (21%), and still none of the 40 held-out human posts.
• Spanish gets E4, a TELL for bold labels on list items: 14% of held-out human posts, 40% to 70% of the Claude ones, none of GPT's.

After these, 71% of Opus 5.5 blog posts pass clean, 57% of Sonnet 5.5 and 93% of Fable 5.2. SKILL.md now says that a clean checklist on a Claude draft says little.

Rewrites made with the new files were judged like the ones before, on the development titles, the held-out ones and the Spanish set (differences from -2.3 to +0.5 points, none significant).

Four new registers

scripts/build-corpus-work.sh fetches human texts from before ChatGPT, and the November models wrote theirs from the same titles.

band file	human texts	held-out humans clean	GPT-6.1 clean	Claude models clean
bands-pr.json	170 merged pull request descriptions, 2019 to 2021	98%	43%	67% to 89%
bands-commit.json	132 commit messages with a long body, 17 projects	97%	0%	67% to 100%
bands-readme.json	170 READMEs last pushed before June 2021	100%	0%	78% to 89%
bands-email.json	170 Enron emails (AESLC)	98%	12%	67% to 89%

The held-out assistant groups have 5 to 9 texts per model. Only texts of about 130 words and up are in the corpora, so the two-line commit message is not measured. The Enron subject lines and mailbox names are not in the repository; the email texts are numbered. The rewrite rules were not run end to end in these registers.

What was measured and left out

All of it is in references/numbers.md and in the ledger (20 new entries).

• Taking the order and the ending of a post from the writer's notes (after SlopShape): -1.5 and -3.4 points over 38 titles, not significant on the held-out ones, and overshoot went from 1.83 to 2.03 features per text.
• An instruction to undo the rewriter's overshoot raised it to 2.19 (p = 0.008).
• A rule to leave a text alone when it already measures human: no measurable gain.
• Matters in the lexicon and "the real X" as a TELL each made one held-out human post fail.
• Graphite's Opus 5.5 phrases, the 2026 contrast forms and the Spanish spelling calques are too rare in these texts or too common in human ones.

For the next round

The end-to-end harness can run on a cheaper panel: JUDGE=claude-sonnet-5-5 GPT_MODEL=gpt-6.1-terra. Both judges order human posts, rewrites and drafts as Opus does (r = 0.96 for Sonnet). report.py --noise prints how far one version moves by itself. Between two reps the set mean moved by up to 3.1 points, so a smaller difference between versions is not a result.

The local human corpora were rebuilt since September (159 blog posts measure, against 167), and the November figures are on those.