Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
{
"name": "quiron",
"source": "./",
"version": "1.3.0",
"version": "1.4.0",
"description": "AI humanizer skill: removes AI slop and the signs of AI writing from prose, then measures the rewrite against human baselines.",
"author": {
"name": "ilien",
Expand Down
262 changes: 261 additions & 1 deletion .claude/skills/upstream-sync/tried.json
Original file line number Diff line number Diff line change
Expand Up @@ -689,6 +689,266 @@
"result": "see numbers.md",
"reason": "see numbers.md",
"retry_if": "new evidence"
},
{
"id": "samples-2026-11",
"date": "2026-11-12",
"source": "research November 2026 (Graphite AI Tells: 55-72% of tells are version-specific)",
"kind": "method",
"target": "eval/ai",
"what": "the same titles written again by Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.2, GPT-6.1 and GPT-6.1 Terra, with a manifest",
"outcome": "measured",
"measured": "evaluate.py on the three registers, old and new groups side by side",
"result": "checklist clean, blog: Sonnet 0% (Sept) -> Sonnet 5.5 79%, Opus 5.5 79%, Fable 5.2 93%, GPT-6.1 7%; A8 dashes Sonnet 100% -> Sonnet 5.5 14%, humans 22%",
"reason": "recorded in numbers.md; the checklist sees less of each new Claude model",
"retry_if": "rerun with each new model generation"
},
{
"id": "cheap-judge-panel",
"date": "2026-11-12",
"source": "this repo, eval/e2e",
"kind": "method",
"target": "eval/e2e/harness.py",
"what": "Sonnet 5.5 and GPT-6.1 Terra as blind judges in place of Opus 5.5 and GPT-5.6",
"outcome": "measured",
"measured": "58 texts already judged by Opus: 18 human, 20 rewrites, 20 drafts",
"result": "Sonnet 4.8 / 12.4 / 79.2, Terra 6.3 / 46.1 / 91.9, Opus 5.9 / 32.2 / 87.8; r with Opus 0.96 overall, 0.67 and 0.55 on rewrites",
"reason": "JUDGE and GPT_MODEL in harness.py; documented in eval/e2e/README.md",
"retry_if": "-"
},
{
"id": "judge-noise",
"date": "2026-11-12",
"source": "this repo, eval/e2e",
"kind": "method",
"target": "eval/e2e/report.py --noise",
"what": "rep-against-rep and judge-against-judge spread of one snapshot",
"outcome": "measured",
"measured": "cached rewrites of base, c18, pf, noskill, sbase",
"result": "set mean moves up to 3.1 points between reps (Opus panel), 2.1 (Sonnet panel); per-title sd 5 to 14; judge-judge sd 3 to 10",
"reason": "a difference under about 3 points between versions is not a result",
"retry_if": "-"
},
{
"id": "bands-both-months",
"date": "2026-11-12",
"source": "this repo, eval/ai/*/train-2026-11",
"kind": "feature",
"target": "bands.json, bands-es.json, bands-fiction.json, AI_REF",
"what": "AI medians and directions from September and November training texts together; human edges kept",
"outcome": "shipped",
"measured": "evaluate.py, held-out split, ai-train both months",
"result": "blog: humans 100% clean both, AI-side mean on humans 0.93 -> 0.75, six directions dropped (ly adverbs, questions, repeated openers, semicolons, em dashes, participles); es: humans failing C1 7% -> 5%, clean 91% -> 93%; fiction: Opus C1 10% -> 30%, Sonnet 5.5 0% -> 20%, humans clean 97% both",
"reason": "fewer humans on the AI side, no loss on the November models",
"retry_if": "-"
},
{
"id": "lexicon-november-2026",
"date": "2026-11-12",
"source": "Graphite AI Tells (CC BY 4.0), Wikipedia Signs of AI writing mid-2026 list, eval/lexicon-pool-blog.tsv",
"kind": "lexicon",
"target": "ai-lean.txt",
"what": "practical, quietly, deliberate/deliberately, preserve",
"outcome": "shipped",
"measured": "evaluate.py, held-out split, ai-train both months",
"result": "B1 FAIL Opus 5.5 7% -> 14%, Sonnet 5.5 14% -> 29%, GPT-6.1 21% -> 36%, Terra 14% -> 29%, humans 0%; lexicon AUC Opus 5.5 0.60 -> 0.72, GPT-6.1 0.72 -> 0.89; B1 fires on 0 of 188 cached rewrites",
"reason": "each passed the per-entry rule on train-2026-11 (<=4% human, >=10% AI)",
"retry_if": "-"
},
{
"id": "lexicon-matters",
"date": "2026-11-12",
"source": "Graphite AI Tells; Wikipedia mid-2026 list",
"kind": "lexicon",
"target": "ai-lean.txt",
"what": "matters",
"outcome": "rejected",
"measured": "evaluate.py, held-out split, ai-train both months",
"result": "3% of human training posts, 38% of train-2026-11, but 5% of held-out humans; with it one held-out human post fails T1",
"reason": "flags a human",
"retry_if": "a form narrower than the bare word"
},
{
"id": "lexicon-technical-nouns-2026",
"date": "2026-11-12",
"source": "eval/lexicon-pool-blog.tsv on train-2026-11",
"kind": "lexicon",
"target": "ai-lean.txt",
"what": "handling, infrastructure, integration, aim, tone",
"outcome": "rejected",
"measured": "evaluate.py, held-out split, ai-train both months",
"result": "pass the per-entry rule (2-3% human, 10-19% AI)",
"reason": "field vocabulary, removed before for the same reason; aim and tone in 0-14% of held-out AI groups",
"retry_if": "-"
},
{
"id": "lexicon-es-fiction-2026",
"date": "2026-11-12",
"source": "eval/lexicon-pool-es.tsv, eval/lexicon-pool-fiction.tsv on train-2026-11",
"kind": "lexicon",
"target": "ai-lean-es.txt, ai-lean-fiction.txt",
"what": "the per-entry rule again on the November texts",
"outcome": "rejected",
"measured": "evaluate.py, held-out split, ai-train both months",
"result": "fiction: no entry passes; es: reciente and combinacion pass on training, 11% and 6% of held-out humans",
"reason": "nothing admissible",
"retry_if": "a larger Spanish pool"
},
{
"id": "e4-bold-labels-es",
"date": "2026-11-12",
"source": "Giammatteo & Carranza 2026 (Exlibris); A18",
"kind": "check",
"target": "CHECKS_ES E4",
"what": "a list item that opens with a bold label, as a Spanish TELL",
"outcome": "shipped",
"measured": "evaluate.py, Spanish held-out split",
"result": "humans 14%, Opus 5.5 60%, Sonnet 5.5 40%, Fable 5.2 70%, GPT 0%; checklist AUC Opus 5.5 0.70 -> 0.82, Fable 5.2 0.71 -> 0.88, GPT-6.1 0.39 -> 0.33; humans clean 93% both",
"reason": "AUC up on every Claude group, no new human FAIL",
"retry_if": "-"
},
{
"id": "a3c-real-actual",
"date": "2026-11-12",
"source": "reader reports October 2026 (r/ClaudeAI, Hacker News): real / actual as emphasis",
"kind": "check",
"target": "new TELL A3c",
"what": "the|a|any|no + real|actual + noun",
"outcome": "rejected",
"measured": "evaluate.py, held-out split, ai-train both months",
"result": "humans 15%, Opus 5.5 43%, Sonnet 5.5 50%, Fable 5.2 50%, GPT-6.1 29%; checklist AUC Opus 5.5 0.71 -> 0.74; one held-out human post now fails T1 (clean 100% -> 97%); 12% of the rewrites, under the human rate",
"reason": "flags a human",
"retry_if": "as a rate, not a TELL"
},
{
"id": "graphite-opus55-phrases",
"date": "2026-11-12",
"source": "Graphite AI Tells, Opus 5.5 update (zero-human trigrams)",
"kind": "check",
"target": "A12 or claude-lean.txt",
"what": "this matters because, taken together these, the takeaway is, just as importantly, helps to understand, in plain terms, bringing it all",
"outcome": "rejected",
"measured": "share of texts, 160 human posts and every eval/ai/blog group",
"result": "0% of humans and 0% to 14% of any group, 2% of train-2026-11",
"reason": "they belong to the article prompt Graphite used, not to posts written from a title",
"retry_if": "samples from an article-from-brief prompt"
},
{
"id": "contrast-2026-forms",
"date": "2026-11-12",
"source": "Graphite AI Tells; reader reports",
"kind": "check",
"target": "audit.py --review",
"what": "not simply/merely, rather than merely/simply/relying/treating, less like X more like Y, is more than a X, without sacrificing/losing",
"outcome": "rejected",
"measured": "share of texts, same groups",
"result": "each in 0% to 7% of November groups and 0% to 2% of humans (rather than merely: 29% of September Sonnet only)",
"reason": "too rare in the current models to act on",
"retry_if": "-"
},
{
"id": "es-orthotypography",
"date": "2026-11-12",
"source": "Giammatteo & Carranza 2026 (Exlibris)",
"kind": "check",
"target": "CHECKS_ES",
"what": "Title Case headings, capital after the colon of a list label, colon after a bare verb, comma before the final y; contrast variants (no solo... sino, no se limita a, semicolon form); ahi radica",
"outcome": "rejected",
"measured": "share of texts, 245 human posts and every eval/ai/es group",
"result": "Title Case 5% human, 0% AI; capital after colon 23% human, 0-60% AI; colon after verb 19% human, 0-20% AI; comma before y 15% human, 0-50% AI; contrast variants 0-6% human, 0-10% November AI; ahi radica in no text",
"reason": "human rates too high or the pattern absent from the models",
"retry_if": "-"
},
{
"id": "order-and-ending-from-notes",
"date": "2026-11-12",
"source": "SlopShape arXiv:2609.15369 (ending restates thesis, synthesis stage, thesis before first section)",
"kind": "various",
"target": "SKILL.md, build the post from the notes",
"what": "before writing, take the order, the ending, the dates and links from the notes; add no stage the notes lack",
"outcome": "rejected",
"measured": "eval/e2e, Sonnet 5.5 rewriter, 2 Sonnet 5.5 judges + 1 GPT-6.1 Terra judge, dev (20 titles) and test (18), 2 rewrites per title",
"result": "dev: Sonnet 15.0 -> 13.1 (worse 5/20, p=0.018), Terra 50.0 -> 46.3 (p=0.098); test: 13.2 -> 12.3 (p=0.64), 49.6 -> 46.7 (p=0.31); pooled 38 titles -1.5 (p=0.061) and -3.4 (p=0.048); overshot 1.83 -> 2.03 (worse on 18 titles, better on 9, p=0.19); factdiff fails 3 -> 5 of 76",
"reason": "gain inside the rep noise, not significant on test, overshot not rarer",
"retry_if": "a rewriter that restructures without overshooting"
},
{
"id": "undo-overshoot",
"date": "2026-11-12",
"source": "this repo: rewrites overshoot on ly adverbs, word length, long words, nominalizations",
"kind": "various",
"target": "SKILL.md Workflow step 4",
"what": "on top of order-and-ending: read the overshot rows and put back the exact word or adverb taken out",
"outcome": "rejected",
"measured": "eval/e2e, Sonnet 5.5 rewriter, 2 Sonnet 5.5 judges + 1 GPT-6.1 Terra judge, dev (20 titles) and test (18), 2 rewrites per title",
"result": "overshot 1.83 -> 2.19, worse on 19 of 38 titles, better on 6 (p=0.008); dev Terra -11.5 (p=0.001) not repeated on test (Sonnet +0.0, Terra -2.1)",
"reason": "the instruction raised what it was meant to lower",
"retry_if": "-"
},
{
"id": "do-no-harm-rule",
"date": "2026-11-12",
"source": "Shan, Lee & Hao arXiv:2608.27855; ledger human-as-input",
"kind": "various",
"target": "SKILL.md",
"what": "measure the draft first; with no notes and no FAIL, change only a ruled-against TELL",
"outcome": "rejected",
"measured": "eval/e2e hum (16 human posts, no notes) and devnn (20 AI drafts, no notes), Sonnet 5.5 rewriter, same panel, 1 rewrite each",
"result": "human posts: originals 5.5 / 6.4, current skill 6.5 / 8.2, with the rule 5.7 / 6.9 (Sonnet -0.8 p=0.15, Terra -1.3 p=0.31, worse on 2 of 16); AI drafts: 67.0 -> 65.2 (p=0.34), 81.7 -> 82.5 (p=0.60)",
"reason": "no measurable gain; with Sonnet 5.5 the loop costs a human text about one point",
"retry_if": "a larger human set"
},
{
"id": "factdiff-quotes",
"date": "2026-11-12",
"source": "Wang, Zhu & Chen 2026 (humanizer audit: at most 4 of 15 quotes kept verbatim)",
"kind": "method",
"target": "factdiff.py",
"what": "list quoted spans of four words or more that are in no source",
"outcome": "rejected",
"measured": "76 cached Sonnet rewrites against draft and notes",
"result": "43 quotes, 8 flagged, 6 of them the text between two quoted spans",
"reason": "mostly noise",
"retry_if": "a quote parser that pairs marks"
},
{
"id": "shared-openings",
"date": "2026-11-12",
"source": "SlopBench arXiv:2609.33905 (same-prompt opener repetition)",
"kind": "method",
"target": "eval/e2e/report.py",
"what": "share of rewrites of different titles that open on the same words",
"outcome": "not-measurable",
"measured": "40 cached rewrites, 40 no-skill rewrites, 160 human posts",
"result": "most common three-word opening: 4 of 40 rewrites, 2 of 40 no-skill, 8 of 160 human",
"reason": "different titles share no opening; the measure needs several outputs of one prompt",
"retry_if": "a set with repeated prompts"
},
{
"id": "package-1.4.0-e2e",
"date": "2026-11-12",
"source": "this repo",
"kind": "method",
"target": "eval/e2e",
"what": "bands from both months, four lexicon words and E4 together, against the version before them",
"outcome": "measured",
"measured": "eval/e2e, Sonnet 5.5 rewriter, 2 Sonnet 5.5 judges + 1 GPT-6.1 Terra judge, dev and test (2 rewrites per title) and es (1)",
"result": "dev Sonnet +0.5 (p=0.72), Terra -2.3 (p=0.43); test +0.1 (p=0.67), -1.7 (p=0.74); es +0.5 (p=0.55), -0.8 (p=0.80); overshot +0.16 per text (p=0.15)",
"reason": "rewrites judged the same; the changes act on detection",
"retry_if": "-"
},
{
"id": "work-registers",
"date": "2026-11-12",
"source": "reader reports 2026 on PR descriptions, commit messages and READMEs; ledger entries blocked on missing samples",
"kind": "feature",
"target": "bands-pr.json, bands-commit.json, bands-readme.json, bands-email.json",
"what": "four band files from pre-ChatGPT human texts (GitHub API, AESLC) and November 2026 assistant texts on the same titles",
"outcome": "shipped",
"measured": "evaluate.py, held-out split, per register",
"result": "held-out humans clean 98% pr, 97% commit, 100% readme, 98% email; GPT-6.1 clean 43%, 0%, 0%, 12%; Claude models clean 67% to 100%; 5 to 9 held-out assistant texts per model",
"reason": "new registers measured; humans within the 5% ceiling",
"retry_if": "larger held-out groups; samples written from a diff; an end-to-end set for these registers"
}
]
}
}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ __pycache__/
.*.quiron.json
.upstream/
.superpowers/
eval/titles/email-*.tsv
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ type: software
authors:
- alias: ilien
website: "https://github.com/ilien-dev"
version: 1.3.0
date-released: "2026-11-04"
version: 1.4.0
date-released: "2026-11-12"
license: AGPL-3.0-only
repository-code: "https://github.com/ilien-dev/quiron"
url: "https://quiron.ilien.dev/"
Expand Down
15 changes: 9 additions & 6 deletions README.es.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ El plugin guarda el skill bajo su propio nombre, así que lo llamas con `/quiron

Pídele a un modelo un post de blog y te llega con una forma que se aprende a reconocer:
listas de tres, el doble de encabezados de los que pondría una persona, una sección de
resumen al final. En español hay dos más: el "no es X: es Y"
y el cierre con "## Conclusión". Quirón le da al modelo una lista de 33 de estas señales
resumen al final. En español hay tres más: el "no es X: es Y",
el cierre con "## Conclusión" y las etiquetas en negrita al inicio de cada elemento de una lista. Quirón le da al modelo una lista de 33 de estas señales
de escritura con IA y un medidor que dice con números si el arreglo funcionó o se pasó de
la raya.

Expand All @@ -101,7 +101,7 @@ $ python3 scripts/aimeter.py eval/ai/blog/e2e-raw/aspittel-782713.md
feature this human band verdict
long words (7+) /1k 353.83 159.68 - 250.46 above band, AI side
nominalizations /1k 55.21 7.29 - 32.65 above band, AI side
em dashes /1k 3.76 0.00 - 2.74 above band, AI side
mean word length 5.46 4.28 - 4.86 above band, AI side
lists of three /1k 11.29 0.00 - 5.51 above band, AI side
plain words /1k 33.88 43.19 - 95.24 below band, AI side
...
Expand Down Expand Up @@ -134,11 +134,13 @@ este repo. Nada entra porque "se lee mejor".

- **Bases humanas.** Las bandas salen de 167 posts de dev.to y 157 cuentos de
WritingPrompts escritos antes de que existiera ChatGPT, y de 241 posts de dev.to en
español de 118 autores. Una cuarta parte de cada conjunto se reserva y nunca se usa para
español de 118 autores. Desde noviembre de 2026 hay también bandas para descripciones
de pull request, mensajes de commit, README y correo de trabajo, en inglés. Una cuarta parte de cada conjunto se reserva y nunca se usa para
elegir nada.
- **Texto de IA de 2026 con los mismos títulos.** Claude Opus, Sonnet, Haiku y GPT
escribieron los textos de comparación que hay en `eval/ai/`, a partir de la petición
simple que escribiría cualquier usuario.
simple que escribiría cualquier usuario. Se escribieron en septiembre de 2026 y otra
vez en noviembre, con los modelos de ese momento.
- **Dos formas de fallar.** Un texto puede quedar del lado de la IA o *pasarse* al otro
lado de la banda humana. Pasarse también delata: los modelos a los que se les pide
"escribir como humano" acaban más cortados y más planos que cualquier persona. El
Expand Down Expand Up @@ -205,7 +207,8 @@ herramientas leen probabilidades de tokens, y Quirón no las toca. Trabaja únic
lo que realmente nota alguien al leer.

**¿Con qué modelos funciona?** Con cualquier modelo detrás de un agente que cargue skills.
Se probó con texto de tres modelos de Claude y uno de GPT.
Se probó con texto de diez modelos de Claude y GPT, cuatro de septiembre de 2026 y seis
de noviembre.

**¿Se inventa cosas para sonar humano?** Sus instrucciones se lo prohíben. Una nueva
versión solo puede usar datos de tu texto o de tus notas. `scripts/factdiff.py` lista los
Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ $ python3 scripts/aimeter.py eval/ai/blog/e2e-raw/aspittel-782713.md
feature this human band verdict
long words (7+) /1k 353.83 159.68 - 250.46 above band, AI side
nominalizations /1k 55.21 7.29 - 32.65 above band, AI side
em dashes /1k 3.76 0.00 - 2.74 above band, AI side
mean word length 5.46 4.28 - 4.86 above band, AI side
lists of three /1k 11.29 0.00 - 5.51 above band, AI side
plain words /1k 33.88 43.19 - 95.24 below band, AI side
...
Expand Down Expand Up @@ -129,10 +129,13 @@ Every rule and number traces to a published study or to a run of the scripts in
repo. Nothing ships on "this reads better".

- **Human baselines.** The bands come from 167 dev.to posts and 157 WritingPrompts
stories, all written before ChatGPT existed, plus a Spanish set. A quarter of each is
stories, all written before ChatGPT existed, plus a Spanish set. Since November 2026
there are also bands for pull request descriptions, commit messages, READMEs and work
email. A quarter of each is
held out and never used to pick anything.
- **2026 AI text on the same titles.** Claude Opus, Sonnet, Haiku and GPT wrote the
comparison texts in `eval/ai/`, from the plain prompt a user would type.
comparison texts in `eval/ai/`, from the plain prompt a user would type. They were
written in September 2026 and again in November with the models current then.
- **Two failure modes.** Text can sit on the AI side of a band or *overshoot* past the
human side. Overshoot is a tell too: models told to "write like a human" go choppier
and plainer than any person does. The meter flags both.
Expand Down Expand Up @@ -193,7 +196,8 @@ token probabilities, and Quirón doesn't touch them. It works only on what a hum
actually notices.

**Which models does it work with?** Any model behind an agent that loads skills. It was
tested on text from three Claude models and one GPT model.
tested on text from ten Claude and GPT models, four from September 2026 and six from
November.

**Does it make things up to sound human?** Its instructions forbid it. A rewrite may use only
facts from your text or your notes. `scripts/factdiff.py` lists the numbers and links in
Expand Down
Loading
Loading