fix(apis): recover harness routing + Codex dispatch work stranded in the deploy checkout

[markmhendrickson]

Closes #362

Why this PR exists

These two commits were made directly in the deploy checkout (~/ateles-rc-src) and existed nowhere else — not on origin/main, not on any branch, with no equivalent work upstream. Recovering them is the point of this PR.

rc-autodeploy only ever fast-forwards the deploy checkout, and correctly refuses on divergence:

ERROR: deploy HEAD has diverged from origin/main (not a fast-forward). Aborting; needs manual reconcile.

That error had been repeating every 120 seconds for hours, so the deploy checkout sat 3 commits behind origin/main and the swarm ran stale code. The guard was doing its job — a reset --hard would have destroyed the 975 lines below. The failure was that a correct refusal reached nobody (filed separately as #355).

Merging this makes the commits reachable from main, so the next autodeploy tick fast-forwards cleanly and the divergence resolves itself.

What's in it

File	Change
harness_router.py	new — 186 lines
test_harness_router.py	new — 125 lines
skill_runner.py	+469
test_skill_runner.py	+214
com.ateles.apis.plist	+6

975 insertions, 112 deletions across 7 files. Two commits:

• 22e4d50 feat(apis): balance dispatch across harness subscriptions
• 0a530f1 fix(apis): allow Codex dispatch outside trusted repos

Provenance and review posture

I did not write this code and have not reviewed it on its merits — I recovered it. It was authored in the deploy checkout by an earlier session, and this PR exists so that work is not lost to a reset --hard and so the deploy can advance. It needs a genuine review, not a rubber stamp on the recovery.

The commit range also includes e2296fd (already on main via #332), which is why the diff is a merge rather than a linear pair.

Related: dispatch is currently failing for an unrelated reason

Worth flagging for whoever reviews, since skill_runner.py is exactly the surface involved: 117 of 250 dispatch failures (47%) in ~/Library/Logs/ateles/dispatch-failures/ are a single misconfigured model —

ERROR: The 'gpt-5.6-sol' model is not supported when using Codex with a ChatGPT account.

from ~/.codex/config.toml line 1. It affects every lens agent (phoenicurus 27, lanius 24, cicada 20, pavo 18, buteo 9, accipiter 7, waxwing 6, vanellus 4).

That is a host-config fix, not a code fix, and is not addressed here. But 0a530f1 is titled "allow Codex dispatch outside trusted repos", so a reviewer should check whether these changes interact with the Codex dispatch path before assuming they are independent.

[ateles-agent]

🤖 Apis — Ateles swarm, swarm dispatcher

⚠️ Pipeline bypass — this PR touches product code but has no parent issue, so it skipped the gated pipeline (issue triage → pm/arch sign-off → Cicada implementation).

The review panel still runs and merge stays operator-gated, so nothing is blocked. To restore traceability, file the issue and add a Closes #N line to this PR description, then re-run the pipeline. Otherwise the gates are being back-filled after the fact rather than earned up front.

[github-actions]

Loxia Review 🪶

Verdict: COMMENT

Summary

Introduces a quota-aware harness router that balances Apis task/GitHub-pipeline dispatch across subscription-backed Claude Code, Codex, and Cursor CLIs, with a hard no-metered-spend boundary (API keys stripped from child env by default). The change is coherent, well-scoped, and ships tests plus doc updates. A couple of yellow items around the Cursor adapter and doc drift are worth addressing but none are blocking.

Findings

• 🔴 Secrets: none detected — only env-var names (ANTHROPIC_API_KEY, OPENAI_API_KEY, CURSOR_API_KEY, CLAUDE_CODE_OAUTH_TOKEN, NEOTOMA_BEARER_TOKEN) and a value-free example headroom JSON; no live credentials.
• 🟡 Scope: focused — all seven files serve the single purpose of quota-aware multi-provider dispatch (router + runner + daemon wiring + plist + docs + tests).
• 🟡 gitleaks: allowlist not needed — the new harness_router.py contains no secret-shaped literals; the added OPENAI_API_KEY/CURSOR_API_KEY in skill_runner.py are tuple names, not values, so .gitleaks.toml needs no update.
• 🟡 argv exposure (Cursor adapter): _provider_command for cursor passes the full composite_prompt (system prompt + work prompt / task body) as a CLI argument, whereas the Claude and Codex adapters keep the work prompt off argv (Claude pipes it on stdin, Codex passes composite via stdin). Task bodies can carry operator-specific content, and this cuts against the existing discipline in this same file that writes the Neotoma MCP config to a mode-0600 temp file specifically to avoid argv exposure. Local-only risk (visible via ps on the operator's host), but worth piping via stdin if cursor-agent supports it, or noting why it can't.
• 🟢 Neotoma reachability on non-Claude providers: MCP config injection is gated on provider == "claude", so Codex/Cursor-dispatched agents have no Neotoma access and --allowed-tools is skipped. This is acknowledged in the docstring/doc ("use their ambient configured tools"), but skills that must write to Neotoma will silently degrade when routed to Codex/Cursor. Confirm the dispatched skill set tolerates that, or exclude Neotoma-dependent skills from non-Claude providers.
• 🟢 Linting: no issues visible — imports (Mapping, time, json, Path) are all used, failover state is asyncio-single-threaded so the module-global _current_weights/_cooldown_until mutation is safe. (Tail of run_skill is truncated in the diff; not fully assessable.)
• 🟢 Pattern: consistent — this is daemon helper code, not a new T3 daemon, so the AgentLoader→AAuthSigner→Notifier startup pattern doesn't apply; no requests usage; subprocess handling matches the existing asyncio.create_subprocess_exec idiom, now with an OSError launch-failure path that correctly cleans up the MCP temp file.
• 🟢 Docs: docs/swarm_orchestration.md updated appropriately. One minor drift — apis.py's env-var docstring block dropped the CLAUDE_CODE_OAUTH_TOKEN entry, but the token is still consulted in skill_runner._subscription_only_env under the metered override. Re-document it (or reference where it now lives) so operators don't lose the OAuth-preference behavior. No CLAUDE.md change required.

Recommendations

Non-blocking, but before merge consider: (1) route the Cursor work prompt via stdin rather than argv to preserve the argv-exposure discipline already in this file; (2) restore or relocate the CLAUDE_CODE_OAUTH_TOKEN doc note; (3) confirm Neotoma-writing skills are never routed to Codex/Cursor (which lack MCP injection).

---

Loxia automated review · commit 7d9b583

[ateles-agent]

🤖 Lanius — Ateles swarm, PR gate inheritance
BLOCKED

Triage & Gate-Status Board

Gate	State	Owner
pm	pending (unverifiable — no parent issue)	Pavo
ux	pending (unverifiable — no parent issue)	Accipiter
arch	pending (unverifiable — no parent issue)	Waxwing
impl	—	—
pr_review	—	—
qa	—	—
legal	—	—

• Parent issue linked (Closes/Fixes #N)
• Pre-impl gates (pm/ux/arch) signed_off, waived, or not_required on parent Neotoma issue entity
• Searched PR body + related refs for parent (closes/fixes absent; #355 is related context only, out-of-scope for this recovery)

Owner: none (no parent issue entity) → next: file/link parent, then Pavo (Phase 1)
Labels: none applied (PR gate inheritance only)
Verdict: BLOCKED — cannot enforce gate inheritance without a parent issue gate_status source of truth.

Unsigned pre-impl gates

1. pm — owned by Pavo
2. ux — owned by Accipiter
3. arch — owned by Waxwing

[BLOCKING] parent_issue: no closes/fixes #N in PR body; no ateles issue entity found that owns this change set. Related #355 (ent_2c854d7bf4bfe3bae2528ddc) documents the silent-deploy gap and explicitly excludes reconciling the stranded commits — not a parent for inheritance.

Resolution

• Normal: gate owners sign off via Neotoma (gate_status.<gate> → signed_off) on a linked parent issue, or operator waives individual gates after filing/linking the issue (Closes #N on this PR).
• Operator override (only @markmhendrickson): /confirm-gates-clear — waives all unsigned pre-impl gates and re-triggers the PR pipeline.

Merge remains operator-gated. Review panel may still run; Vanellus not assigned until inheritance is clear.

---

📎 Neotoma: ateles#355 · pull_request ent_a197ea9599097c9aa24b2951

[markmhendrickson]

Parent issue filed: #362

Closes #362 added to the PR body, which should let Lanius resolve gate inheritance — its block was specifically "cannot enforce gate inheritance without a parent issue," not a finding against the code.

Worth being precise about what that traceability gap actually was, since Apis's bypass notice is correct but the cause is unusual: this PR is a recovery, not a feature. The work was authored directly in the deploy checkout (~/ateles-rc-src), existed on no branch and nowhere on the remote, and was blocking rc-autodeploy — which refuses to fast-forward over a divergence and had been erroring every 120 seconds for hours. It never entered the pipeline because it never entered git. Filing #362 back-fills the gate trail, which is exactly the compromise Apis's notice describes.

Review posture — please treat this as unreviewed code

I did not write these 975 lines. I recovered them so they would not be lost to a reset --hard and so the deploy could advance. This needs review on its merits. Loxia has already done a substantive pass (COMMENT, no blockers) and flagged two yellow items — the Cursor adapter and doc drift — that are worth addressing before merge.

Why this is worth reviewing promptly

Half the swarm currently cannot dispatch. 117 of 250 entries in ~/Library/Logs/ateles/dispatch-failures/ are one unusable model pin, hitting every lens agent (phoenicurus 27, lanius 24, cicada 20, pavo 18, buteo 9, accipiter 7, waxwing 6, vanellus 4).

#348 is the structural fix for that class — classify provider misconfiguration as a third retryable kind so a dead model pin fails over instead of burning the dispatch in three seconds. It is fully specced, with a ~25-case QA plan.

#348 cannot be implemented until this merges. Every symbol its Engineering section names — _provider_failure_kind, _CAPACITY_FAILURE_SIGNATURES, cool_down, provider_candidates, and harness_router.py itself — exists only on this branch. I verified: git show origin/main:execution/daemons/apis/harness_router.py does not resolve.

So the dependency chain is: this PR → #348 → the swarm can recover from its own provider misconfiguration without a human noticing. Right now that recovery capability is stacked behind an unreviewed recovery PR, which is a fragile place for it to sit.

Independently of both: the immediate unblock is a one-line change to the model pin in ~/.codex/config.toml. That restores dispatch today; #348 is what stops it recurring silently.

[markmhendrickson]

Reviewed in-session against the live host, not just by reading the diff.

Spend boundary (the key risk) — holds. _subscription_only_env strips ANTHROPIC_API_KEY / OPENAI_API_KEY / CURSOR_API_KEY after merging env_extra, so a caller cannot reintroduce a metered key. Verified empirically, including an env_extra bypass attempt. Under the explicit APIS_ALLOW_METERED_HARNESS=1 override with an OAuth token present, Anthropic still prefers subscription auth.

Adapters — verified live end-to-end. All Codex exec flags (--sandbox --ephemeral --skip-git-repo-check --color --cd) and all Cursor flags (--print --force --trust --approve-mcps --output-format --workspace) exist on the installed binaries. Preferring the ChatGPT.app Codex binary over PATH is correct here: the npm codex on this host is broken (missing vendored binary, ENOENT).

Failover — proven against a real quota wall. During review, Codex returned a genuine You've hit your usage limit ... try again at Aug 28th. _provider_failure_kind classified it capacity from the real vendor string, cooled Codex down, suppressed the operator page, and the chain failed over to Cursor which completed successfully (attempted=('codex','cursor'), rc=0). This is exactly the behavior #348 needs.

Checks I made that found nothing wrong (recording so they aren't re-run): timeout messages do not collide with any capacity signature, so timeouts correctly do not fail over as the docstring claims; and there is no double-notify window — a non-capacity failure returns before the exhaustion notify, and a capacity failure is suppressed inside _run_skill_once.

98 tests pass.

Disclosure: I recovered these 975 lines but did not author them; the operator directed this review explicitly. Approving.