Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
{
"name": "quiron",
"source": "./",
"version": "1.2.0",
"version": "1.3.0",
"description": "AI humanizer skill: removes AI slop and the signs of AI writing from prose, then measures the rewrite against human baselines.",
"author": {
"name": "ilien",
Expand Down
32 changes: 16 additions & 16 deletions .claude/skills/upstream-sync/upstream.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,50 +3,50 @@
{
"repo": "sam-paech/slop-score",
"weight": "proven",
"reviewed": null,
"reviewed_on": null
"reviewed": "289264ab2a0df1358ef81ced3c259610ea709514",
"reviewed_on": "2026-11-04"
},
{
"repo": "sam-paech/antislop-sampler",
"weight": "proven",
"reviewed": null,
"reviewed_on": null
"reviewed": "0ae330e98fbe6f09351f2d1063a51956378a44b2",
"reviewed_on": "2026-11-04"
},
{
"repo": "theclaymethod/unslop",
"weight": "proven",
"reviewed": null,
"reviewed_on": null
"reviewed": "17ed39c9d0b522f44190ff0c6233867eadee192a",
"reviewed_on": "2026-11-04"
},
{
"repo": "harshaneel/humanize",
"weight": "proven",
"reviewed": null,
"reviewed_on": null
"reviewed": "9c3dec34dc8e0ed9d26de14a869900a714da8f9e",
"reviewed_on": "2026-11-04"
},
{
"repo": "blader/humanizer",
"weight": "popular",
"reviewed": null,
"reviewed_on": null
"reviewed": "0ef9fe5bc3572734fd3ae6c448fe00a30cd5dcf0",
"reviewed_on": "2026-11-04"
},
{
"repo": "hardikpandya/stop-slop",
"weight": "popular",
"reviewed": null,
"reviewed_on": null
"reviewed": "8da1f030185bdfe8471220585162991eaeb970e9",
"reviewed_on": "2026-11-04"
},
{
"repo": "petergyang/no-ai-slop",
"weight": "popular",
"reviewed": null,
"reviewed_on": null
"reviewed": "000650b156983f5159695b441477f4e63b25dc85",
"reviewed_on": "2026-11-04"
},
{
"repo": "conorbronsdon/avoid-ai-writing",
"weight": "popular",
"reviewed": null,
"reviewed_on": null
"reviewed": "ccc8fc486c3a1dd6cc00be6258a6db3b12e801d3",
"reviewed_on": "2026-11-04"
}
],
"deferred": []
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ type: software
authors:
- alias: ilien
website: "https://github.com/ilien-dev"
version: 1.2.0
date-released: "2026-09-24"
version: 1.3.0
date-released: "2026-11-04"
license: AGPL-3.0-only
repository-code: "https://github.com/ilien-dev/quiron"
url: "https://quiron.ilien.dev/"
Expand Down
32 changes: 32 additions & 0 deletions references/numbers.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,19 @@ against 0.39 USD) and left the meter where it was. The meter did not move (check
both, overshot 1.83 and 1.85 features per text), and rewrites with a figure or link in
no source fell from 4 of 60 to 1 of 60.

November 2026, two rewrites per title, two Opus judges and one GPT judge. The current
version against the same rewriter given the same draft and notes and one sentence of
instruction instead of the skill ("rewrite the text so it reads naturally, the way the
writer would write it; keep every fact from the notes and add none"): on the 20
development titles Opus 32.0 against 44.0 (-12.0, better on 18 of 20, p = 0.001), GPT 16.4
against 22.3 (p = 0.022); on the 18 held-out titles Opus 26.7 against 32.1 (better on 14
of 18, p = 0.011), GPT 14.0 against 16.2 (p = 0.025). The notes alone do not explain the
result. Given a human post as the draft instead (14 pre-2022 dev.to posts outside every
set, no notes), the rewrite read slightly more AI than the original: Opus 6.0 to 7.5
(worse on 8 of 14, p = 0.056), GPT 6.4 to 7.9 (p = 0.09). Two more posts were not
rewritten, because the skill stopped to ask for the notes. On a text a person wrote, the
loop costs a little.

Where the rules came from. The judges' 288 reasons for calling a 1.1.0 rewrite AI,
grouped: human cues present (70, counter-evidence), uniformly tidy prose (63), a line that
wraps a section up or a stock hand-off (58), safety and best-practice caveats nobody
Expand Down Expand Up @@ -286,6 +299,25 @@ in September 2026.
the same cost per rewrite (0.52 USD, 13 turns), so it bought nothing measurable.
- **Placeholders and date notes as checklist items**: in 12 and 10 of 176 human posts
(placeholders in code, "at the time of writing"), so they stay instructions only.
- **November 2026 upstream intake** (slop-score, Antislop, unslop, humanize, humanizer,
stop-slop, no-ai-slop, avoid-ai-writing), on the rebuilt corpus (160 dev.to posts, 40
held out; 157 stories; 245 Spanish posts). A cross-sentence contrast with any subject
("The hardest part wasn't X. It was Y.") raised the checklist AUC (0.860 to 0.873) and
left held-out humans unflagged, but end to end it made rewrites worse on the development
titles (Opus 32.0 to 34.0, p = 0.28; held-out 26.7 to 26.1). A feature for paragraphs
that open on a word from the intro separated AI posts (AI-side AUC 0.879 to 0.881 blog,
0.829 to 0.849 Spanish) and also made rewrites worse (Opus +1.8 dev, +2.4 held-out). Word
lists: of 2,779 upstream entries one passed for blog; the fiction survivors flagged one
more human story on held-out and on 117 fresh stories (FAIL 8.5% to 9.4%), and none of
2,529 Antislop phrases passed. No gain or more humans flagged: plain "not X, but Y"
(A1b on held-out humans 12.5% to 27.5%), "No X, no Y, no Z" chains (one held-out story),
wider A9, A12, A16, A17 and A21/A22 phrase lists (no hits, or more human hits), a Title
Case filter (A19 stopped catching AI), a proximity gate on A1, short self-answered
questions, fillers ("in terms of", in 14% to 20% of human posts), bare-noun bullet
lists, and features for staccato runs, callbacks, n-gram repetition, specific anchors,
bold, punctuation spread, function-word entropy and paragraph uniformity. A copy gate
against voice samples fired on 7 of 19 rewrites, all of them the author's own sign-off,
which the notes also held.

### Calibrating for another register

Expand Down
10 changes: 6 additions & 4 deletions references/patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,12 @@ carries a new fact. Merge a row of fragments into a sentence with a specific cla

### 3. Sayings that sound deep [TELL; READ for "X is the Y of Z"]

**Watch for:** the real question is, at its core, in reality, what really matters,
fundamentally, the deeper issue, the heart of the matter, X is the Y of Z, X becomes a
trap, the language of, the architecture of.
**Measured:** in 2% of posts on either side. Cheap to check, rare in 2026 output.
**Watch for:** the real question is, the real problem (cost, story) is, at its core, in
reality, what really matters, fundamentally, the deeper issue, the heart of the matter, X
is the Y of Z, X becomes a trap, the language of, the architecture of.
**Measured:** in 2% of posts on either side. Cheap to check, rare in 2026 output. Widening
"the real question is" to any noun (from unslop's phrase list) caught 2 of 14 held-out
Sonnet posts and no held-out human post.
**Instead:** replace the saying with the specific claim underneath it.

### 4. Staged run-up before the point [TELL]
Expand Down
4 changes: 4 additions & 0 deletions references/sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ Every study and dataset behind a number in this skill.
sentence-length burstiness study, the 49-feature stylometric fingerprint, the
human-vs-AI classification study). *Do AI Models Speak Human?* uses ten PMC passages
and *Modelometry* uses LMArena chat.
- Conor Bronsdon, *avoid-ai-writing* (github.com/conorbronsdon/avoid-ai-writing, commit
ccc8fc4): detector word tiers, the source of *surprisingly* in `ai-lean.txt`.
- *unslop* (github.com/theclaymethod/unslop, commit 17ed39c): matching curly apostrophes
as straight ones before the phrase checks, and "the real X is" in A3.
- Herbold et al. (2023) on structural uniformity in ChatGPT essays.
- Fan, Lewis & Dauphin (2018). *Hierarchical Neural Story Generation.* ACL 2018. The
WritingPrompts corpus the fiction bands are built from.
Expand Down
6 changes: 4 additions & 2 deletions references/word-choice.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,10 @@ skill's run, it appeared in at most 4% of human posts and in at least 10% of ass
posts. The 2026 survivors are words like *increasingly*, *significantly*, *consistently*,
*comprehensive*, *meaningful*, *merely*, *careful*, *remains*, *maintaining*: ordinary
words, used by models at many times the human rate. One of them is English. Three or more
distinct entries in one text fail B1, which none of the 42 held-out human posts did and
43% of the GPT posts did. Technical nouns that passed the same test (*architecture*,
distinct entries in one text fail B1, which none of the 40 held-out human posts did and
50% of the GPT posts did. In November 2026 the word lists of slop-score, Antislop and
avoid-ai-writing were scored the same way, word by word: of 2,779 entries not already
here, one passed, *surprisingly* (3% of human training posts, 17% of assistant ones). Technical nouns that passed the same test (*architecture*,
*infrastructure*, *integration*, *handling*, *complexity*) were taken out again: they are
the field's vocabulary, and rewriters had to cut real terms to clear the check.

Expand Down
6 changes: 4 additions & 2 deletions scripts/audit.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,8 @@ def _thresholds():
("A3", "sayings that sound deep", "tell",
r"\b(?:the real question is|at its core|in reality,|what really matters|"
r"fundamentally,|the deeper issue|the heart of the matter|becomes a trap|"
r"the (?:language|currency|architecture) of (?:the|our|modern)\b)", None),
r"the (?:language|currency|architecture) of (?:the|our|modern)\b|"
r"the real \w+ (?:is|isn't|was)\b)", None),
("A3b", "X is the Y of Z", "judge", r"\bis the \w+ of (?:the |a )?\w+\b", None),
("A4", "staged run-up", "tell",
r"\b(?:here's the thing|the thing is,|let's be honest|real talk|buckle up|"
Expand Down Expand Up @@ -245,7 +246,8 @@ def run(path, samples=()):
"""Every check on one text. A TELL that also fires in the writer's own sample
texts is the writer's habit, not a tell: it is reported and not counted."""
habits = {r[0] for p in samples for r in run(p) if r[2] == "TELL"}
raw = open(path, encoding="utf-8").read()
# the patterns are written with a straight apostrophe; GPT writes it’s and let’s
raw = open(path, encoding="utf-8").read().replace("’", "'").replace("‘", "'")
body = re.sub(r"```.*?```", "", raw, flags=re.S)
text = prose(raw)
sample_args = [x for p in samples for x in ("--sample", p)]
Expand Down
29 changes: 26 additions & 3 deletions scripts/evaluate.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,10 @@
Per feature: AUC of each AI group against the held-out humans, i.e. the chance
a random AI text sits further toward the AI side than a random human text.
0.5 is no signal. Per check: the share of texts in each group it FAILs, so a
check that fires on humans is visible at once.
check that fires on humans is visible at once, and the share where it reports
TELL or FAIL, which is how a TELL check shows what it catches. Checklist AUC:
the number of items found per text (TELL or FAIL, combined checks left out),
AI group against held-out humans.

With --ai-train DIR, a second and independent judge is trained: a naive Bayes
classifier over word frequencies, fitted on the calibration humans against those
Expand All @@ -39,6 +42,10 @@
import aimeter # noqa: E402


# checks that combine others, or that only a reader can settle: not counted as found items
COMBINED = {"C1", "C2", "T1", "V1", "V2"}


def texts(directory):
return sorted(os.path.join(directory, n) for n in os.listdir(directory)
if n.endswith((".md", ".txt")))
Expand Down Expand Up @@ -120,10 +127,12 @@ def main():
if ai_train:
judge = Judge(train_paths, texts(ai_train))

report = {"bands_from": len(train), "groups": {}, "auc": {}, "checks": {}}
report = {"bands_from": len(train), "groups": {}, "auc": {}, "checks": {}, "tells": {},
"checklist_auc": {}}
checklist = {}
for name, items in rows.items():
scores, ai_side, over, fails, zero = [], [], [], [], 0
check_fail = {}
check_fail, check_hit, found = {}, {}, []
for path, m in items:
v = {key: aimeter.verdict(key, m[key], bands[key]) for key in aimeter.AI_REF}
scores.append(sum(1 for x in v.values() if x == "ok"))
Expand All @@ -135,6 +144,10 @@ def main():
zero += not nf
for cid in nf:
check_fail[cid] = check_fail.get(cid, 0) + 1
hit = [r[0] for r in res if r[2] in ("TELL", "FAIL") and r[0] not in COMBINED]
found.append(len(hit))
for cid in hit:
check_hit[cid] = check_hit.get(cid, 0) + 1
n = len(items)
pj = [judge.p_ai(p) for p, _ in items] if judge else []
report["groups"][name] = {
Expand All @@ -149,6 +162,12 @@ def main():
"audit_clean_share": round(zero / n, 2) if n else None,
}
report["checks"][name] = {c: round(x / n, 2) for c, x in sorted(check_fail.items())}
report["tells"][name] = {c: round(x / n, 3) for c, x in sorted(check_hit.items())}
checklist[name] = found
for name, found in checklist.items():
if name != "human-heldout":
# how many checklist items fire, as a score: AI text further up than human text
report["checklist_auc"][name] = auc(found, checklist["human-heldout"], "hi")
for key in aimeter.AI_REF:
push = bands[key].get("push", aimeter.AI_REF[key][0])
if not push:
Expand All @@ -175,6 +194,10 @@ def main():
cids = sorted({c for g in report["checks"].values() for c in g})
for c in cids:
print(f"{c:18}" + "".join(f"{report['checks'][n].get(c, 0):>16}" for n in rows))
print(f"\n{'TELL or FAIL rate':18}" + "".join(f"{n:>16}" for n in rows))
for c in sorted({c for g in report["tells"].values() for c in g}):
print(f"{c:18}" + "".join(f"{report['tells'][n].get(c, 0):>16}" for n in rows))
print(f"\n{'checklist AUC':18}" + "".join(f"{report['checklist_auc'][n]!s:>16}" for n in names))


if __name__ == "__main__":
Expand Down
1 change: 1 addition & 0 deletions scripts/lexicons/ai-lean.txt
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ realm # legacy
remains # kobak2025
significantly # ai-lean(current),lexa-news-gpt4.1mini
subtle # lexa-sci-gemini3
surprisingly # avoid-ai-writing2026
tapestry # legacy
testament # legacy
thereby # legacy
Expand Down
Loading