Today's software is generated once and frozen. Tomorrow's software improves itself.
Point it at any API or codebase. It builds an agent-native CLI, grades it against reality, and refactors until the grade stops climbing — then tells you what it learned.
The "I'm going to the beach" workflow: give it a target and a goal, walk away, and come back to a measurably better tool plus a report on how it got there — every iteration recorded, every regression rolled back, every unsafe change blocked.
🎯 10+ things you can put in a loop — loop-anything-hub
Point the loop at a domain and it turns "generate once, hope it's good" into
"improve until an independent referee says it's good." The
loop-anything-hub is a
live catalog of these loops, auto-published from demos/ on every push — each card
headlines the loop outcome (grade trajectory + convergence + report), which a
plain CLI gallery can't.
Not aspiration — recorded runs, graded by the independent CLI-Judge referee and
refined by a free-tier LLM (no Anthropic quota), recorded via demo record (the
only path to a live_verified card):
| Loop | What it automated | Outcome | Proof |
|---|---|---|---|
| automate-your-job | a team-lead's daily standup digest, graded against a captured day of real activity | F → A |
PROOF |
| factcli | a CLI brought up to the agent-native (non-interactive) contract | F → A |
PROOF |
| one-person-industrial-engine | a 2-slice product fleet (API → dependent digest), both slices converged in dependency order with the upstream outcome routed downstream | F → A ×2 |
PROOF |
| software-arch | a local microservice made agent-native — an ops CLI graded on a 4-task non-interactive JSON contract, refactored until every command is clean | F → A |
PROOF |
Before vs after — what the loop actually changed. Each verified run turned a failing baseline into an A:
factcliwentF(0.0) → A(100.0)in 4 iterations (two refactors rolled back for no gain before the third stuck); the standup digest wentF → Aonceversion --jsonstopped crashing and the digest became structured JSON with the captured blocker surfaced;software-archwentF → Aonce its ops CLI stopped crashing/prompting and emitted valid JSON on every command. And when the loop can't reach the bar it stops and says so (thebiotech-discoveryrecipe plateaus atC → B → B,stopped) rather than fake a green. Full side-by-side with grades, trajectories, and the specific bugs fixed: docs/loop-anything-before-after.md.
| Domain | What "improving itself" looks like |
|---|---|
| 🔀 PR lifecycle | a loop that drives a PR green — tests, review comments, conflicts — until it merges |
| ⚖️ Legal | clause/contract tooling refined against a rubric of real redlines |
| 🧪 Clinical trials | eligibility/protocol tooling graded against real trial criteria |
| 🧬 Biotech | assay/pipeline CLIs improved against captured lab data |
| 📈 Quant | a strategy/backtest tool refined against held-out market data |
| 💼 VC | deal-screening tooling graded against a real diligence rubric |
| 🏛️ Software architecture | architecture-review CLIs climbing a quality rubric |
| 🎓 Education | tutoring/grading tools refined against answer keys + rubrics |
| ⚡ Smart grid | load/forecast tooling graded against real telemetry |
| 📦 Supply chain | routing/inventory CLIs improved against real logistics data |
← Your application here. Each row is a loop waiting to be built. Add one in a single PR — see
CONTRIBUTING-demos.md. Domains beyond the engine's current reach ship as loop recipes (the map, not fake demos); starter cards are badgedillustrativeuntil a live run is recorded.
loop-anything demo list # registered demos + recipes
loop-anything showcase --out showcase.html # the self-contained gallery
# Turn a real catalog CLI into a verified before/after proof (refine-only):
loop-anything demo proof <id> --catalog cli-anything --name <entry> \
--sha <full-40-char-commit-sha> --install-kind pip_git_subdir --dry-runProofs validate the refine loop, not the generate frontier.
demo proofadopts an already-generated catalog CLI as the baseline ("before"), runs the loop (judge →/ce-work→ re-judge →/ce-compound), and records alive_verifiedcard with the before/after grade, the per-dimension diff, iterations, and the regression tests it compounded.
Beyond running loops, this repo is a hub of three things: the knowledge, the
tooling, and the experts of the loop-engineering ecosystem. The ecosystem-hub
factory ingests each top-rated cited repo and emits, per repo, a knowledge
graph and grade-gated agentic tooling (skills / plugins / MCP servers) —
composing existing blocks (understand-anything / graphify for the KG, cli-anything build for tooling, the loop controller to grade-and-converge, SKILL.md + MCP to
distribute). The ingestion queue is data, SHA-pinned and license-gated:
docs/cited-repos.yaml. Design + the five axes
(high-quality / fast / cheap / up-to-date / future-proof) as concrete mechanisms:
docs/plans/2026-07-19-001-ecosystem-hub-factory.md.
First live output: a real, deterministic (no-LLM) knowledge graph for a cited
repo — HarnessX, 584 files → 6317 nodes / 21715 edges, correctly surfacing its
event-driven architecture. Evidence: docs/hub/kg/harnessx/;
regenerate any repo's graph with scripts/build_repo_kg.py. The tooling half now
has its quality gate: a deterministic ToolingJudge grades a generated skill on
structure / claim-fidelity / safety (proven to discriminate — skills/loop-anything
→ A, broken/unsafe → F; see docs/hub/tooling/JUDGE-PROOF.md).
And the tooling half now runs end to end: a KG-keyed ToolingSkillFactory
generates a skill from a repo's knowledge graph, which the loop grades and
converges — proven on HarnessX (generate → A(100) → CONVERGED;
docs/hub/tooling/FACTORY-PROOF.md).
Building agent-native tooling today is a one-shot act: generate a CLI, eyeball it, stop. Nothing closes the gap between "it exists" and "it's actually good."
- 🔁 Compounding, not one-shot — each iteration makes the tool measurably better, and the fix is captured as a regression test so a solved problem never comes back.
- 🌍 Grounded in reality, not vibes — quality comes from an independent referee grading the tool against real captured payloads, never from the model admiring its own code.
- 🛡️ Degradation-proof — a multi-signal convergence policy (plateau + regression rollback + budget) stops the recursive "flying turd" effect where agents slowly make things worse.
- 😴 Autonomous by design — kick it off, go to sleep, read the research report in the morning.
- 🔒 Safety is a hard gate — an unsafe tool can never ship, no matter how high it otherwise scores.
It treats the generated tool as a Heuristic System — code, rules, detectors, tests — and runs an agentic "nutrition pipeline" that evolves it continuously. Fast code-based heuristics execute (System 1); the slow LLM agent reflects and refines (System 2).
flowchart LR
T([🎯 target + goal]) --> R{route}
R -->|service / API| F1[CLI-Printing-Press]
R -->|codebase| F2[CLI-Anything]
F1 --> TOOL[(agent-native CLI)]
F2 --> TOOL
TOOL --> J[CLI-Judge\nreport.json]
J -->|grade < A & safe & budget left| W[/ce-work refactor/]
W --> TOOL
J -->|grade A| WIN([✅ converged])
J -->|safety fail| STOP([🛑 blocked, never ships])
W -.accepted fix.-> C[/ce-compound\nlearning + regression test/]
loop-engineering-anything is not another CLI generator. It is the controller, memory, and
convergence policy that wire four existing tools into a closed feedback loop:
| Stage | Tool | Role in the loop |
|---|---|---|
| 🏗️ route + generate | CLI-Printing-Press · CLI-Anything | build the agent-native CLI (service lane / codebase lane) |
| ⚖️ judge | CLI-Judge | grade it against reality — 5 dimensions, A–F, report.json |
| 🔧 refactor | /ce-work |
fix the lowest-scoring dimensions |
| 📚 compound | /ce-compound |
record the learning + a regression test |
These are installable dependencies, wrapped behind adapters — never forked. The orchestrator stays thin; the upstreams evolve on their own.
| Today's pain point | What loop-engineering-anything does |
|---|---|
| 🪦 Generated CLIs are frozen the moment they're built | Keeps refactoring until the grade converges |
| 🤷 "Looks fine to me" is the only quality bar | Grades against real payloads via an independent referee |
| 📉 Agents recursively degrade what they touch | Plateau detection + regression rollback + hard budget |
| 💸 Overnight runs balloon in cost | Iteration/token budget enforced by the convergence policy |
| ☠️ Unsafe code slips through on a high score | Safety failure is a terminal state — capped, blocked, never shipped |
| 🧠 The same bug gets re-fixed forever | Every accepted fix is compounded into a regression test |
|
Hand it a URL, HAR capture, or OpenAPI spec. It routes to the service lane and keeps improving the CLI against real responses. |
Hand it a local repo. It routes to the codebase lane and grinds the generated CLI up the grade scale. |
One command, unattended. Wake up to a converged tool, a full grade trajectory, and a research report. |
Every accepted fix is compounded into a learning the next run reuses. Those learnings
used to live only in a machine-local database (loopeng.db, gitignored) — training notes
locked in one gym. Now the corpus is a versioned artifact: export it as diff-able
JSONL, commit it, review it in a PR, import it on any machine — the learning-reuse
flywheel keeps compounding across laptops and teammates.
loop-anything learnings export --redact -o learnings/corpus.jsonl # commit this file
loop-anything learnings import learnings/corpus.jsonl # idempotent merge--redact strips target-specific tokens (URLs, paths, long ids) so a shared corpus
carries the transferable lesson, not your target's internals. Imports flow through the
same sanitize-on-write path as recorded learnings (MemoryStore.record_learning), so a
hostile line in a corpus file can't forge prompt structure — and re-importing the same
file inserts nothing.
A run used to exist only as CLI flags — unreviewable, un-diffable, and impossible to attach to the proof pack it produced. Now the ask is a file you commit next to the thing it converges:
loop-anything contract check loop.yaml # validate; print the compiled plan
loop-anything run --contract loop.yaml # run it
loop-anything contract evidence loop.yaml --run 12 # verify it proved what it promisedversion: 1
name: qms-agent-native
target: ./qms-kbp
goal: Make the factory QMS inspection workflow operable by AI agents.
budget: { target_grade: A, max_iterations: 8, plateau_patience: 2, token_budget: 250000 }
evidence:
required: [grade_trajectory, dimension_diff, regression_tests]Three rules keep it from becoming decoration (design, annotated example):
- It compiles, it doesn't extend. Every key becomes a
config.Budget/Lanethe loop already reads. No new controller state, no knob the engine ignores. - An unenforced declaration is an error, not a no-op. A typo'd
max_iteratons:— or a hopefulsafety:block nothing consumes — fails the parse. A governance field that quietly does nothing is the false-green this engine exists to prevent. - The gate only tightens.
require_human_confirm: falseis rejected. The contract is caller-authored, and a caller can never pre-confirm its own run.
evidence.required is checked against the run's real proof pack and exits non-zero on
a gap — so declaring evidence is a claim the run has to satisfy.
A webinar summary listed sixteen things a mature agent loop must have, and verified none of
them against a running system. That is a rubric wearing an essay's clothes, so it is encoded
as one: docs/rubrics/agent-loop-engineering.yml
holds the claims as data, and scripts/audit_loop_rubric.py
is the only thing that reads them, so the published score cannot drift from the claims it scores.
python3 scripts/audit_loop_rubric.py # score this engine
python3 scripts/audit_loop_rubric.py --gate # exit 1 unless every item passes**Conformance: 25/28 (89%)** verifiable claims implemented · 3 declared gaps · 0 unmeasured · gate **FAIL**
Evidence is observed — a symbol at a real file:line, or a pytest node that is actually
executed. No evidence means no. A probe that cannot run is unmeasured: excluded from the rate
and blocking, because "I could not look" is not "it works".
The first version of this rubric scored 23/23, and that was the bug. It was written by someone who had just spent an hour in this codebase, and it contained no question he expected to fail — an assessment that cannot fail its author is a mirror. That run is kept, honestly labelled a reconstruction, so the claim is reproducible rather than asserted:
python3 scripts/audit_loop_rubric.py --rubric docs/rubrics/archive/agent-loop-engineering.v1.yml
# 23/23 (100%) · gate PASS ← the score that measured nothingThree items stay red with a rationale recorded beside the mark and no mechanism to turn a
rationale into a pass (ReAct interleaving and manager-worker are deliberate divergences;
human takeover of a running loop is a real gap). Three further declared gaps — typed
failure taxonomy, branching search, span-level trace schema — carry probes that fail the
moment the gap silently closes, so the limitation list cannot rot.
Full output: docs/rubrics/AUDIT.md.
Per-run cost, wall time and iteration count were always recorded; nothing ever divided.
MemoryStore.success_rate(target=None) closes that, and two choices decide whether the number
is honest: a run still in flight is excluded from the denominator, and a run halted by the
safety gate counts as a failure — otherwise the metric would rise when the safety gate
fires. It returns None, never 0.0, when nothing has finished: not measured must not read
as tried and failed.
research/attention-budget/ applies the @skills protocol
(arXiv:2608.12610) to a real skill library and then measures the quantity that paper explicitly
does not measure. Top-1 routing against a 175-skill corpus: 100% at N=10 and N=40, 83% at
N=80, 44% at N=176 — the break falls below the paper's argued 100-slot bound. Limits, bias and
two self-corrections (a YAML block-scalar parse bug that made the first numbers wrong) are
recorded in that directory's README rather than quietly fixed.
| 🔁 Multi-signal convergence | Stops at the first of: target grade reached, plateau (no gain over N), iteration/token budget, or a safety block. No single "until Grade A" naïveté. |
| 🪞 Reflective refinement | Each refactor brief carries why the last attempt scored what it did — the referee's dimension-level feedback, which failures keep resisting, and whether the prior edit was kept or rolled back — so the refiner aims instead of re-guessing blind (the GEPA "actionable side information" idea, borrowed additively; judge-sourced only, so maker≠checker holds). |
| ♻️ Learning-reuse flywheel | Lessons compounded on past runs of a target are retrieved and fed into future runs' briefs (ranked by the grade gain they produced), so accumulated usage makes each run start less blind — and per-target trend queries make the compounding measurable (does it converge faster as history grows?). Reused lessons feed the refiner only, never the referee (maker≠checker), and are sanitized on the way into storage. |
| 💡 Spec-synthesis loop | The same loop runs one stage up: hand it a vague idea and a deterministic rubric spec-grader scores the draft spec (completeness, testability, cross-reference consistency, scope, grounding) while the spec-refiner improves it — idea → spec → grade → refine — so spec quality compounds with usage too. The grader reads only the spec (maker≠checker), and the stage reuses the same controller, learning-reuse, and ablation-proof machinery. The live idea→spec / spec-refiner hookup is wired (and the reuse-vs-blind ablation proof harness with it); the real graded runs open with quota. |
| 🛡️ Unbypassable safety gate | A safety-failing verdict is a terminal BLOCKED_SAFETY state — the change is rolled back and the tool can never ship. |
| ↩️ Regression rollback | A refactor that doesn't raise the grade is reverted to the prior git checkpoint; the better verdict is kept. |
| 📚 Compound-on-accept | /ce-compound fires only on a kept improvement — never on a transient gain that's later rolled back. |
| 🗄️ Local-first memory | Every run/iteration/grade/learning lands in SQLite, enabling cross-run queries (trend, plateau, recurring failures). |
| 🔌 Protocol-driven core | The controller depends only on Judge/Refiner/Compounder/Checkpoint protocols — so loop dynamics are proven against recorded verdicts, no live tool required. |
The popular "agent loop" anatomy (state → reason → act → observe → reflect → terminate) describes one agent improving its own answer. This is an outer loop that improves a generated tool and is opinionated about three things on purpose — they are design choices, not missing features:
- Outer-loop sovereignty — we referee the refiner's output, never its inner tokens. The inner agent loop is a swappable vendor; we are the meta-loop it plugs into.
- Single referee of record — quality comes only from the independent CLI-Judge verdict, never from the maker's self-report. That single authority is the maker≠checker moat.
- Gated human confirm — a converged result is a claim until a human confirms; the gate is on by default and its verdict is recorded for audit but never auto-ships.
Full rationale, with the failure mode each choice accepts: docs/solutions/outer-loop-non-gaps.md.
Relation to
cobusgreyling/loop-engineering— that 8.5k★ repo is the outer loop (schedule / triage / escalate a repo over time); this engine is the inner loop (converge one artifact to an independent grade). They compose, they don't compete — history says every dead improvement loop died at the grading step, which is exactly the step this engine enforces. Should-we / how-to integrate, with the survival-test evidence:docs/solutions/integrate-loop-engineering.md.
A single loop improves one target. A fleet coordinates many self-improving loops toward one goal — the orchestration layer above the loop:
- Dependency-ordered — items run in topological waves over the worktree fan-out; a cycle fails closed before any work starts.
- Feedback-routed — when an item converges, its outcome is routed into its dependents' briefs automatically (no human forwarding).
- Human-efficient — only high-judgment forks (a safety block, a gated result, a stuck worker) reach a human; everything else proceeds. You can re-brief a single worker without re-running the fleet.
loop-anything fleet run fleet.json --goal "ship the feature across these targets"
loop-anything fleet run fleet.json --dry-run # materialize only, don't execute
loop-anything fleet status <fleet_id>
loop-anything fleet report <fleet_id>
loop-anything fleet escalations <fleet_id>Each fleet item carries its own target (and optional goal/lane); fleet run
executes the DAG by default — generating, judging, and refining each item inside
its own git worktree, with the referee held immutable to the maker per item.
--dry-run keeps the old materialize-only behavior.
This is a loop engine coordinating our own loops — not an issue→PR→CI Agent
IDE. Boundary + rationale:
docs/solutions/fleet-orchestration-boundary.md.
flowchart TD
User([user / agent]) -->|"/loop-anything <target> --goal"| Skill[loop-anything skill / CLI]
Skill --> Runner[autonomous runner]
Runner --> Router{target router}
Router --> PP[Printing-Press adapter]
Router --> CA[CLI-Anything adapter]
PP --> Tool[(generated CLI)]
CA --> Tool
Tool --> Ctrl[loop controller]
Ctrl --> Judge[judge adapter → CLI-Judge]
Judge -->|verdict| Ctrl
Ctrl -->|refactor brief| Work[/ce-work/]
Work --> Tool
Ctrl -->|accepted fix| Compound[/ce-compound/]
Ctrl <--> Mem[(SQLite memory)]
Runner --> Report[research report]
loop-engineering-anything/
├── src/loopeng/
│ ├── cli.py # loop-anything entrypoint (run / contract / preflight / status / report / demo proof)
│ ├── config.py # budgets, convergence knobs, dependency table
│ ├── contracts/ # loop.yaml run contract — parse, compile to Budget/Lane, verify evidence
│ ├── preflight.py # per-mechanism dependency detection (+ refine-only gate)
│ ├── adopt.py # catalog tool adopter — venv-isolated, env-pruned, full-SHA pin
│ ├── proof.py # ProofPack builder + store-backed compounder
│ ├── router.py # target → lane classification
│ ├── memory/store.py # + success_rate(): cross-run outcome, safety blocks count as failures
│ ├── adapters/
│ │ ├── base.py # Verdict / GenerateResult + Judge/Refiner/Compounder/Checkpoint protocols
│ │ ├── safety.py # shell=False exec, metachar rejection, workspace jail
│ │ ├── printing_press.py # service lane factory shell
│ │ ├── cli_anything.py # codebase lane factory shell
│ │ └── judge.py # CLI-Judge wrapper + strict report.json parsing
│ ├── memory/ # SQLite store + trend/plateau/recurring queries
│ ├── loop/ # controller state machine, convergence, brief, compound, GitCheckpoint
│ └── autonomous/ # research report + autonomous runner
├── docs/rubrics/ # the Agent Loop Engineering rubric as data + its generated audit
├── research/attention-budget/ # the @skills residency audit + the trigger-reliability experiment
├── scripts/audit_loop_rubric.py # scores this engine against the rubric; --gate exits 1
├── skills/loop-anything/ # the /loop-anything agent skill
├── tests/ # 606 tests — loop dynamics validated against recorded verdicts
└── docs/plans/ # the implementation plan
git clone https://github.com/wjlgatech/loop-engineering-anything
cd loop-engineering-anything
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
loop-anything preflight # check the four dependencies
pytest -q # 286 passingPreflight tells you exactly what's wired:
[MISSING] CLI-Printing-Press (service/API lane) -- not on PATH (looked for: printing-press, cli-printing-press)
[MISSING] CLI-Anything (codebase lane) -- not on PATH (looked for: cli-anything, cli-hub)
[MISSING] CLI-Judge (referee) -- not on PATH (looked for: cli-judge)
[ok ] compound-engineering plugin (/ce-work, /ce-compound) -- plugin found
Drive a target (auto-routes by lane; gates on preflight; generates → judges → refines to Grade A):
loop-anything run https://api.example.com --goal "make this agent-native and keep improving it"
loop-anything run ./my-repo --goal "raise correctness and safety to Grade A"
loop-anything run ./my-repo --refiner chain # claude, then free-tier LLM on infra failure (default)
loop-anything run ./my-repo --judge-adapter path/to/adapter.py # override adapter auto-discovery
loop-anything status # recorded runs
loop-anything report <run_id> --json # the research reportThe CLI-Judge adapter is auto-located from the generated tool (override with
--judge-adapter); an adapter inside the tool itself is refused so the referee
stays immutable to the maker. A converged run is marked shippable only after a
human confirms it (--confirm); --scheduled --confirm is rejected so an
unattended run can't pre-confirm itself.
CLI-Judge's safety gate caps a tool's grade at C on any safety failure. The loop treats this
as a terminal BLOCKED_SAFETY state: the offending change is rolled back, the run halts, and
the tool is never shipped (R3) — regardless of how high it scored elsewhere. Autonomous runs
apply code only inside a workspace boundary, read credentials from the environment (never logged),
and checkpoint every iteration so any regression reverts cleanly.
Built from an 8-unit plan (docs/plans/). Loop dynamics are already validated against recorded
verdicts — so the hardest part (does the loop converge without degrading?) is proven before any
live run.
- U1 — Scaffold,
loop-anythingCLI,/loop-anythingskill, dependency preflight - U2 — SQLite memory layer (trend / plateau / recurring-failure queries)
- U3 — Target router (service vs. codebase lane)
- U6 — Loop controller core: state machine, convergence policy, safety hard-gate, regression rollback
- U4 — Factory adapter shells (Printing-Press / CLI-Anything)
- U5 — Judge adapter shell (strict
report.jsonsafety derivation) - U8 — Autonomous runner shell (preflight + credential + workspace guards)
- U7 — History Compression Engine (grade-neutral-or-better System-2 pass)
- P0 #1 mechanism — headless
/ce-work+/ce-compoundviaclaude -p - P0 #2 mechanism — grade-variance probe + noise-aware acceptance band
- Judge live-binding —
CLIJudgepinned to the realreport.json(safety_blocker,D1..D5dims,--out) and verified against an installedcli-judge - Real loop run —
LoopControllerdriven by the livecli-judgeto a correct terminal state - Factory live-binding — CLI-Anything generation is an agentic
claude -p "/cli-anything …"skill; CLI-Printing-Press needs a Go toolchain - Full agentic e2e — real generate (
/cli-anything) + real refine (/ce-work) on a live target
Both feasibility gates are resolved. P0 #2 is empirical: the live CLI-Judge is deterministic (variance probe spread
0.0), so grades are a safe control signal. P0 #1 is mechanical:/ce-work+/ce-compoundrun headlessly viaclaude -p. The loop has been run end-to-end against the real referee; the remaining frontier is the agentic generate/refine steps and the Go-based service lane — see the runbook.
| Tool | What it brings |
|---|---|
| CLI-Printing-Press | URL/HAR/OpenAPI → a CLI with a local SQLite mirror and domain archetypes |
| CLI-Anything | local software → an agent-native Click CLI with a SKILL.md |
| CLI-Judge | reality-grounded referee — 5 dimensions / 100 pts, hard safety gate |
| compound-engineering | the brain — /ce-work refactors, /ce-compound captures learnings |
PRs welcome. The repo enforces a simple bar: CI must be green (pytest on Python 3.11–3.13) and
behavior changes carry a CHANGELOG.md entry plus synced docs. See AGENTS.md for
architecture boundaries — chiefly wrap-don't-fork, the protocol-only controller, and the
unbypassable safety gate.
MIT — free to use, modify, and distribute.
A self-improving loop for the age of AI agents · grounded in reality · safe by construction
Built by driving CLI-Printing-Press · CLI-Anything · CLI-Judge · compound-engineering around a closed loop.
Thanks for visiting ✨