Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
{
"name": "quiron",
"source": "./",
"version": "1.0.0",
"version": "1.1.0",
"description": "AI humanizer skill: removes AI slop and the signs of AI writing from prose, then measures the rewrite against human baselines.",
"author": {
"name": "ilien",
Expand Down
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ type: software
authors:
- alias: ilien
website: "https://github.com/ilien-dev"
version: 1.0.0
version: 1.1.0
date-released: "2026-09-23"
license: AGPL-3.0-only
repository-code: "https://github.com/ilien-dev/quiron"
Expand Down
33 changes: 31 additions & 2 deletions SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,8 @@ Quirón progress:
- [ ] 4. Meter: AI side fixed, then overshot
- [ ] 5. Checklist: no FAIL, every READ and TELL ruled on
- [ ] 6. check.sh converged (two clean runs in a row, with --source when rewriting)
- [ ] 7. Final message follows the step 6 template (never "ready to publish")
- [ ] 7. Review items asked one at a time, if the writer is in the conversation
- [ ] 8. Final message follows the step 7 template (never "ready to publish")
```

1. **Write or rewrite.** Apply the points above, the pattern index below, and the word
Expand All @@ -177,7 +178,8 @@ Quirón progress:
have a median of 20 of 23 features in band. The goal is a clean checklist, not 23 of 23.
5. **Converge** with `check.sh` (next section). When rewriting, pass the draft and any
notes with `--source`; a number, date or link the sources do not have fails the run.
6. **Report honestly**, with this template, in the user's language:
6. **Review with the writer** (section below): `audit.py --review`, one question at a time.
7. **Report honestly**, with this template, in the user's language:

```
Result: <N> of 23 features in the human band; checklist <F> FAIL, <T> TELL; converged: yes/no.
Expand Down Expand Up @@ -250,6 +252,33 @@ any Title Case heading and any rate outside the band; it failed every one of 42
human posts. The current rules pass 98% of them clean and still fail 86% of the Claude
Sonnet and 93% of the GPT posts written on the same titles.

## Review with the writer

`scripts/audit.py --review [--json] FILE` lists sentences a reader may take for AI that
the loop must not fix on its own. Today that is the negated setup: a short "X isn't Y."
opening a paragraph or right before the point ("What surprised me most wasn't a
number."). It is in 31% of assistant posts and 7% of human ones, but removing it inside
the loop made rewrites read more AI to blind judges, because many of these sentences are
ordinary ones (`references/numbers.md`). So the writer decides, sentence by sentence,
and you leave these sentences exactly as they are while writing, rewriting and looping.

Do this only when the writer is in the conversation, after `check.sh` converged; skip it
in batch or headless runs. Ask about one item at a time, never as a list, and wait for
the answer before the next. Each question carries:

- the sentence, quoted, and which paragraph it is in;
- why a reader may take it for AI: the reasons `--review` prints, in the writer's language;
- your suggested change, written out: a rewrite that states the point, or deleting the
sentence when the paragraph works without it;
- four choices: **Keep**, **Use the suggestion**, **Delete**, **Write my own**.

Use the host's question tool when it has one (in Claude Code, AskUserQuestion: the first
three as options with the suggestion in the preview, and its free-text answer as "Write
my own", used word for word); otherwise ask in plain text with the same four choices.
Apply each answer before asking the next. A suggestion adds no fact and is not another
negated setup. After the last answer, if the text changed, run `check.sh` once more and
report the result; never undo the writer's choice to clear a TELL.

## Pattern index

What `audit.py` checks. The state is what it reports. Full entries, with measured rates
Expand Down
33 changes: 33 additions & 0 deletions references/numbers.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ The bands, the AI directions, what the whole apparatus does on held-out texts, h
- Four counter-intuitive results
- What the apparatus does on held-out texts
- What the loop does to a text
- Checks measured and not shipped
- Calibrating for another register
- Part D: what this does not do

Expand Down Expand Up @@ -155,6 +156,38 @@ python3 scripts/evaluate.py --human CORPUS --ai-train eval/ai/blog/train \
opus=eval/ai/blog/test-opus sonnet=eval/ai/blog/test-sonnet gpt=eval/ai/blog/test-gpt
```

### Checks measured and not shipped

Each of these was measured with the same held-out split and left out, so it is not
proposed again without new evidence. Human corpus: 176 dev.to posts (44 held out), rebuilt
in September 2026.

- **Stock phrases** (2 to 4 words, chosen on the training split). Phrases in at most 4% of
human and at least 10% of assistant training posts: 41% of held-out human posts carry
three or more, so the list flags people. Phrases in no human training post: AUC 0.53 to
0.57 against held-out assistant posts, close to chance. Rewrites by the skill already sit
at the human level on both lists (AUC 0.45 to 0.50).
- **Length as verbosity.** Opus 5.5 and Sonnet 5 asked for a dev.to post with no length
wrote 1.41 and 1.27 times the words of the human post on the same title (AUC 0.67 and
0.59, 25 posts). The extra is code and headings: prose alone was shorter than the
human's (0.70 times), with shorter sentences. Headings are already a band; a length rule
has no human text to compare a single post against.
- **Negated setup (A1c)**: a sentence of 12 words or fewer that only denies something,
opening a paragraph or followed by the reframe ("What surprised me most wasn't a
number."). As a checklist item it looked right: 7% of held-out human posts, 31% of
assistant posts, held-out humans still 0% T1 FAIL, T1 FAIL on GPT posts from 7% to 36%.
End to end it made rewrites worse. 20 titles (8 fresh), 52 rewrites per version, two
fresh Opus judges each: mean P(AI) 43.6 without the check and 46.4 with it, higher on 13
of 20 titles (Wilcoxon p = 0.036); on the fresh titles 34.6 against 40.5. The countable
rates did not move. Many of the sentences it removed were ordinary ones from the
author's notes ("The page isn't finished yet."), and the judges' reasons for the
rewrites with the check were "too even" and "uniformly polished". It is kept as a review item
the writer rules on (`audit.py --review`), outside the loop, T1 and `check.sh`. With
that version of the skill, 40 more rewrites of the same 20 titles (two per title,
headless, so no review questions) gave mean P(AI) 43.7 against 43.6 without it (+1.3
per title, 95% CI -2.0 to +4.7, Wilcoxon p = 0.42), and the countable rates did not
move. Negated setups were left in 5 of 40 rewrites, against 11 of 52 before.

### Calibrating for another register

**Calibrating for another register.** Academic prose, news, email and other languages
Expand Down
9 changes: 7 additions & 2 deletions references/patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,17 @@ State in brackets is what `audit.py` reports.

**Watch for:** it's not X, it's Y; not just / not only / not merely X, but Y; X rather
than Y; the contrast split across two sentences ("This does not mean X. It means Y.");
the clipped negative tail ("…, no guessing", "That's a favor, not a liability.").
the clipped negative tail ("…, no guessing", "That's a favor, not a liability."); the
negated setup, a short sentence that only denies something, opening a paragraph or right
before the point ("What surprised me most wasn't a number.").
**Measured:** the family is in 38% of assistant training posts and 18% of human ones; "rather
than" alone was in 35% and 8% on the training split. "It's not X, it's Y" is in 5% of
human posts and 9% of assistant ones, so it cannot fail on sight. Across 22 models the
slop-score benchmark (Paech 2025) finds the construction at 2 to 13 times the human rate;
the rate, not a single use, is the tell.
the rate, not a single use, is the tell. The negated setup is in 7% of held-out human posts
and 31% of assistant ones, but fixing it inside the loop made rewrites read more AI to
blind judges, so it is a review item only: `audit.py --review` lists it and the writer
decides (SKILL.md, "Review with the writer"). Blog bands only.
**Why it fails:** the negative half names something nobody claimed, so the positive half
sounds larger. It adds weight without adding a claim.
**Instead:** state the point. Keep the contrast only when the negative half corrects a
Expand Down
51 changes: 51 additions & 0 deletions scripts/audit.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@
audit.py --json FILE
audit.py --sample PATH FILE a file or folder of the writer's own texts: a TELL that
fires there too is their habit and is not counted
audit.py --review [--json] FILE
sentences for the writer to decide on, one at a time; never
part of the checklist, T1 or check.sh, and the exit code is 0
"""
import json, os, re, subprocess, sys

Expand Down Expand Up @@ -196,6 +199,41 @@ def _thresholds():
c for c in CHECKS if c[0] in ("C1", "C2", "T1", "V1", "V2")]

SENT = re.compile(r"(?<=[.!?])\s+")

# Review items: patterns a reader notices that the rewrite loop must not act on alone.
# A1c (negated setup) is in 7% of held-out human posts and 31% of assistant posts, but
# fixing it inside the loop made rewrites read more AI to blind judges (numbers.md,
# "Checks measured and not shipped"), so only the writer decides, sentence by sentence.
NEGSET = re.compile(r"\b(?:isn't|wasn't|aren't|weren't|is not|was not|are not|were not)\b", re.I)
NEGSET_SKIP = re.compile(r"\b(?:but|because|if|when|unless|so)\b", re.I)
REFRAME = re.compile(r"(?:it's|it is|it was|that's|that is|this is|what |the (?:real|actual|point|answer)|"
r"instead|rather)", re.I)
NEGSET_WHY = [
"It denies something nobody said, so the sentence after it sounds bigger than it is.",
"It sits where the point should be: {where}.",
"A short 'X isn't Y.' sentence like this is in 31% of assistant blog posts and 7% of "
"human ones (held-out dev.to posts).",
]


def review(path):
"""Negated setups, with the reasons a reader may take them for AI. Blog bands only."""
if lexicon_name() != "ai-lean.txt":
return []
items = []
for n, para in enumerate(re.split(r"\n\s*\n", prose(open(path, encoding="utf-8").read()))):
ss = [s_.strip() for s_ in SENT.split(para.replace("\n", " ")) if s_.strip()]
for i, s_ in enumerate(ss):
nxt = ss[i + 1] if i + 1 < len(ss) else ""
opens, reframed = i == 0, bool(REFRAME.match(nxt))
if (len(s_.split()) <= 12 and s_.endswith(".") and NEGSET.search(s_)
and not NEGSET_SKIP.search(s_) and (opens or reframed)):
where = ("it opens the paragraph, before the real point" if opens
else "right before the sentence that makes the real point")
items.append({"id": "A1c", "name": "negated setup", "paragraph": n + 1,
"sentence": s_, "next": nxt,
"why": [w.format(where=where) for w in NEGSET_WHY]})
return items
MINOR = set("a an the and or but of to in on for with vs at by from is as into via".split())


Expand Down Expand Up @@ -348,6 +386,19 @@ def main():
missing = [p for p in paths if not os.path.isfile(p)]
if missing:
sys.exit(f"no such file: {', '.join(missing)}")
if "--review" in sys.argv:
for path in paths:
items = review(path)
if as_json:
print(json.dumps({"file": path, "items": items}, indent=1))
continue
print(f"\n{os.path.basename(path)}: {len(items)} to review")
for k, it in enumerate(items, 1):
print(f"\n{k}. {it['id']} {it['name']}, paragraph {it['paragraph']}")
print(f" \"{it['sentence']}\"")
for w in it["why"]:
print(f" - {w}")
return 0
fails = 0
for path in paths:
res = run(path, samples)
Expand Down
Loading