fix: stop runaway submit_verdict loop in leaf turn loop#149
Conversation
submit_verdict now sets AgentToolResult.terminate: true on a successful verdict, which agent-loop.ts (pi-fork) already respects as a signal to stop the turn loop immediately once every tool call in a batch sets it. Without this, a leaf that submitted a valid verdict kept getting re-prompted and would call submit_verdict again — one observed leaf did this ~1,269 times before an external pod kill stopped it. Add verdictTerminationExtension as a backstop: caps total turns (default 40, or LeafEnvelope.maxTurns when set) and blocks further tool calls once a verdict has already been captured, covering leaves that never call submit_verdict cleanly (e.g. a model that keeps calling other tools instead). Wired in unconditionally in run-leaf.ts's realProduceVerdict so the cap applies even during gate-approval-pending phases. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Jeremy Cohn <Jeremy.Cohn@ibm.com>
cwiklik
left a comment
cwiklik
left a comment
There was a problem hiding this comment.
Fixes a real runaway loop (an Act 2 leaf called submit_verdict ~1,269× before an external kill). Two-layer fix: primary is submit-verdict-tool returning terminate: true on a valid verdict (agent-loop already honors it); the new verdict-termination-extension backstops with a turn cap (default 40) and blocks tool calls once a verdict is captured. The docstrings are unusually honest about the mechanism's limits, and the unit tests are assertive with good edge coverage (off-by-one at the cap, default-cap-when-omitted, no-terminate-on-invalid).
No secrets, no .claude/.vscode, no dependency changes. Two non-blocking notes inline, both on the backstop.
CI note: trivy-scan is failing, but it's the pre-existing repo-wide dependency CVE — this PR adds only .ts files (no dependency manifests), so it's not attributable to #149. That check still gates merge and needs an org-wide fix (dep bump or scoped .trivyignore), separate from this review.
LGTM.
Assisted-By: Claude Code
| pi.on("turn_start", () => { | ||
| counter.turns += 1; | ||
| }); | ||
| pi.on("tool_call", (event) => { |
There was a problem hiding this comment.
suggestion (non-blocking): the backstop only blocks tool calls — which (as your docstring notes) re-prompts and costs a turn rather than hard-stopping. So a model that keeps emitting tool calls after a verdict, or past maxTurns, still advances turns and only stops when it stops calling tools. The docstring's claim that hitting the cap "fails cleanly instead of hanging" relies on an independent hard iteration ceiling in agent-loop.ts. Worth a one-line test/assert that exceeding maxTurns actually ends the leaf (not just blocks the tool), so the backstop's guarantee doesn't silently depend on the model choosing to stop.
| if (capture.verdict) { | ||
| return { block: true, reason: "Verdict already submitted for this item — task complete." }; | ||
| } | ||
| if (typeof maxTurns === "number" && maxTurns > 0 && counter.turns > maxTurns) { |
There was a problem hiding this comment.
nit: asymmetry in the cap. Omitting maxTurns falls back to DEFAULT_MAX_TURNS (40), but an explicit maxTurns: 0 survives the ?? (nullish coalescing) and then fails the maxTurns > 0 guard — so 0 silently disables the turn cap (unbounded), the opposite of "cap immediately." If a LeafEnvelope could ever carry 0, consider treating <= 0 as "use default" (or documenting that 0 means unlimited).
2 participants
Background
An async (Act 2) leaf could keep calling
submit_verdictafter already recording a valid verdict — one observed leaf did this ~1,269 times before an external pod kill stopped it. The leaf turn loop had no signal to stop once a verdict was captured, andLeafEnvelope.maxTurnswas never enforced.Fix
submit-verdict-tool.ts— a successful verdict now returnsterminate: true, whichagent-loop.tsalready honors to stop the turn loop. Invalid verdicts don't terminate (the agent should retry).verdict-termination-extension.ts(new) — backstop that caps total turns (default 40 /maxTurns) and blocks further tool calls once a verdict is captured, covering leaves that never callsubmit_verdictcleanly. Wired intorun-leaf.ts.Test plan
Assisted-By: Claude (Anthropic AI) noreply@anthropic.com