Skip to content

fix(agent): retry empty final turns instead of silently abandoning the task - #1896

Open
snimu wants to merge 15 commits into
mainfrom
snimu/empty-turn-retry
Open

fix(agent): retry empty final turns instead of silently abandoning the task#1896
snimu wants to merge 15 commits into
mainfrom
snimu/empty-turn-retry

Conversation

@snimu

@snimu snimu commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Providers occasionally end a turn with a normal stop reason and zero usable content: no text, no tool calls, sometimes a thinking-only message (observed and raw-SSE-confirmed for moonshotai/kimi-k3 via Prime Inference: single-chunk stream, no content, usage.output=0; ~1/10 raw probes). The agent loop treated any no-tool-call turn as completion, so the agent silently abandoned its task: print mode printed nothing and exited 0; interactive mode thought for minutes and then did nothing.

The fix

  • streamAssistantResponse now retries an empty final turn silently, up to 3 attempts. The retry happens before message_end is emitted, which is the durability edge: discarded attempts are popped from the request context and never reach session persistence or transcript consumers; their paid usage is carried on the surviving message as discardedUsage, so cost surfaces and goal token budgets still account for it.
  • Empty means: no tool calls and no non-whitespace text (thinking does not count). error, aborted, and length stops are excluded - they are signals of their own, and an identical resend cannot help.
  • Context-overflow shapes pass through untouched (!isContextOverflow(message, contextWindow) guards the retry, reusing the existing detector): a silent-overflow turn is empty by definition, and retrying it would burn full-context requests and then mask the stop reason that auto-compaction recovery keys on.
  • After 3 consecutive empty turns, the final message becomes a standard error turn (Model returned an empty response ... 3 times in a row) and flows the existing error path: print mode exits non-zero with the message on stderr; session-level retry/backoff applies as for any other turn error.
  • RLM children: a child whose turn ends in an error now delivers RLM child <name> failed: <error> to its parent instead of the misleading "completed without sending a reply" notice. (Found while wiring exhaustion propagation: graceful error turns resolve promptAndWait, so the parent never saw the error text.)

Tests

Eight new tests in packages/agent (silent retry with clean transcript and exactly one message_end; whitespace-only retry; exhaustion after 3 attempts; no-retry guards for real content, tool calls, aborts, empty length stops, and silent-overflow turns - the last two pin the compaction-recovery contract) plus one in agent-session-recursion (parent receives the child failure message, no terminal notice), an interactive-mode pin (a superseding assistant stream replaces a discarded attempt's bubble), and discarded-spend accounting asserts in the loop pin and the scan/resident usage-parity test. All retry-path pins were proven red against the pre-fix code.

Linear: ENG-5795


Note

Medium Risk
Changes core turn completion, retry, and token accounting across agent, session, and RLM paths, though behavior is bounded (3 retries) and covered by new tests.

Overview
Fixes a case where a normal-stop assistant message with no visible text and no tool calls (e.g. thinking-only) was treated as success and the agent stopped with no output.

Agent loop: streamAssistantResponse now silently retries up to 3 times before message_end. Discarded attempts are popped from provider context and never become durable transcript turns; after three empties the turn becomes a standard error message. Context-overflow empty shapes are not retried so compaction recovery still sees them. Paid tokens from dropped attempts are stored on the surviving message as optional discardedUsage (context sizing still uses usage only). A later stream failure during retries wraps prior discard spend in EmptyTurnRetryFailure; Agent.handleRunFailure unwraps it and attaches discardedUsage to the synthetic failure message.

Product layers: Coding-agent goal, autonomous, RLM child usage, telemetry, session stats/scan, and interactive streaming now count discardedUsage and fix UX (RLM child failure notice instead of “completed without reply”; remove orphaned empty streaming bubbles when a superseding stream starts).

Reviewed by Cursor Bugbot for commit d852d24. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Retry empty assistant turns up to 3 attempts instead of silently abandoning the task

  • Adds a retry loop in streamAssistantResponse (agent-loop.ts) that detects empty assistant turns (no text, no tool calls, not error/aborted/length-stop) and retries up to three times. Discarded attempts are removed from context and transcripts, their usage is preserved, and a third consecutive empty response becomes an assistant error.
  • Adds discardedUsage to AssistantMessage (types.ts) so paid-but-discarded attempts carry their spend onto the surviving or failure message.
  • Updates usage accounting across the coding agent to include discarded spend: session statistics, goal token budgets, autonomous mode counters, telemetry, context-tree totals, and session-file scanning.
  • Changes RLM child completion so a child ending with an assistant error reports rlm_child_failure to the parent instead of a completed-without-reply notice.
  • Cleans up interactive streaming so a superseding message_start removes orphaned thinking-only components but preserves visible partial output.
  • Risk: isEmptyAssistantTurn classifies turns with only thinking or whitespace as empty; turns that end with length stop reason or context-overflow messages bypass the retry path. Reviewers should verify that legitimate thinking-only responses with meaningful reasoning are not discarded.

Macroscope summarized d852d24.

Linear: RES-1279 (https://linear.app/prime-intellect/issue/RES-1279)

snimu added 2 commits August 29, 2026 14:53
…pleting (ENG-5795)

Providers occasionally end a stream with a normal stop reason but no usable
output (no text, no tool calls; sometimes thinking-only). The agent loop
treated any no-tool-call turn as completion, silently abandoning the task
(-p printed nothing and exited 0).

- agent: an empty final turn (no tool calls, no non-thinking content) is
  silently resent up to 3 attempts; empty attempts are dropped from the
  request context and never emitted as message_end, so they do not pollute
  the transcript. After the third empty response the turn ends as a normal
  turn error (print mode exits non-zero via the existing error path).
- coding-agent: an RLM child whose final turn ended in a graceful error and
  that never replied now surfaces to the parent as rlm_child_failure with
  the error text instead of a bare completed-without-reply notice.
…G-5795)

Review follow-up: an empty length-stop turn (Xiaomi MiMo overflow shape) and
a silent stop-overflow turn (z.ai shape, usage.input past the context window)
must pass through untouched so agent_end compaction recovery can see them.
The retry now skips stopReason length entirely (truncation is a signal; an
identical resend cannot change the outcome) and guards with the existing
isContextOverflow detector instead of re-deriving overflow logic.
Comment thread packages/agent/src/agent-loop.ts
Dmatut7 added a commit to Dmatut7/prime-agent that referenced this pull request Sep 5, 2026
The agent loop retries an empty assistant turn in place up to three times and
then synthesizes a terminal error. That message matched no exclusion in the
session-level retry predicate, so the session retried the whole turn again:
3 inner attempts x 4 session attempts = up to 12 full-context requests, the
inner 9 with no backoff. isContextOverflow stayed false, so the run never
reached compaction - on a provider that does not report usage a silent
overflow just burned tokens.

Mark the terminal message with a synthetic stopReasonRaw, export
isEmptyTurnRetryExhausted() from the agent loop, and exclude it in
_isRetryableError. A provider-level "empty response" error still retries: it
carries no marker, and one session retry is the right response there.

The discarded attempts' usage is deliberately not accumulated into the final
message. That would push input + cacheRead over the context window and make
isContextOverflow misfire on an ordinary empty response.
Dmatut7 added a commit to Dmatut7/prime-agent that referenced this pull request Sep 5, 2026
…meIntellect-ai#1896 uptake

An empty-turn retry emits a fresh message_start per attempt and no message_end
for the dropped one. startAssistantStreamingMessage settled that unmatched
component into a final bubble, but its message was popped from the transcript
and never persisted, so the bubble disappeared on /resume or rebuild: the live
view and the restored view disagreed.

Both message_end and agent_end clear streamingComponent, so a live component
here is always an unmatched start. Remove it instead of settling it. The
streamingComponent = undefined that preceded the reassignment was dead, and
clearing streamingMessage here would be dead too: both fields are reassigned
below.
Dmatut7 added a commit to Dmatut7/prime-agent that referenced this pull request Sep 5, 2026
A discarded empty-turn attempt is popped from the transcript and never emits
message_end, so any accounting that reads usage off the transcript lost its
spend: with three attempts, up to two thirds of the cost and output tokens
silently disappeared.

Carry the discarded attempts' cost and output tokens onto the terminal message.
Input tokens, cacheRead, cacheWrite and totalTokens are deliberately left at the
final attempt's values. isContextOverflow compares input + cacheRead against the
context window, so inflating them would turn an ordinary empty response into a
false overflow, which _isRetryableError then treats as non-retryable and routes
to compaction instead of a retry.
Dmatut7 added a commit to Dmatut7/prime-agent that referenced this pull request Sep 5, 2026
R3 merge-back. F4 cleared first: the parallel lane's uncommitted
preserve_thinking/enable_search WIP was committed at aa80279 (it was
breaking the repo type check), which unblocked this merge.

Brings in 237 files from the upstream 0.9.x line: daemon schema 26->27
(digest 589a2219bc8b), version 0.8.1->0.9.1, the mermaid rendering
feature (grok-mermaid), and cherry-picked upstream PRs PrimeIntellect-ai#2027/PrimeIntellect-ai#1947/PrimeIntellect-ai#1896.
All 264 local-only files and 54 local-only tests preserved (merge, never
whole-tree checkout).

openai-completions.ts auto-merged (R3 changes and the WIP compat feature
touch disjoint regions). FORK_NOTES.md conflict resolved by folding the
R3 sync detail into the running update log. Installed grok-mermaid (new
upstream dep, already in the merged lockfile) into node_modules.

Verified on the merged tree: biome 1036 files clean, tsgo 0 errors,
installer + browser-smoke pass, openai-completions tests 20/20.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/agent/src/agent-loop.ts
…tempt leaves behind

Review follow-up (Bugbot + fresh-eyes): a discarded attempt emits message_start (and thinking updates) but never message_end, and startAssistantStreamingMessage appends a component per start - the dead attempt lingered as a streaming thinking bubble. A still-open streaming component at the next assistant message_start is now replaced, mirroring the existing dangling-component removal at agent_end. Pin covers a thinking-only discarded attempt.
…tempts

Review follow-up: discarded attempts were paid for (thinking output plus full input context) but their usage vanished with the pop, undercounting cost surfaces and goal token budgets. The loop now aggregates dropped attempts onto the surviving message as discardedUsage (new optional AssistantMessage field); spend consumers add it (context-tree own/total usage, session stats, goal accounting, telemetry, and the readSessionInfo scan - kept in lockstep with the resident computation via the existing parity test), while context estimation and overflow classification keep reading the per-request usage untouched.
… asserts

Cleanup pass per standing policy: multi-line narrative comments collapsed to one-line invariants; three incidental errorMessage/stopReason asserts dropped from the bundled no-retry pin (the identity and message_end asserts carry the behavior).
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/coding-agent/src/core/agent-session.ts
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
…mous limits, and child attribution

Review follow-ups on discardedUsage: three more spend consumers ignored it.

- Goal accounting returned early for error turns, so an exhausted
  empty-retry sequence charged nothing; discarded attempts were
  normal-stop spend and now count even when the turn itself errored
  (the failed turn's own usage stays excluded, per existing policy).
- Autonomous limits: discarded spend now adds to tokensUsed without
  consuming a turn (addAutonomousDiscardedUsage).
- RLM child attribution folds discardedUsage into the parent aggregate,
  unindexed spend, and the pending flush bucket; a failed child turn
  still attributes its discarded (paid) portion.
…rows, retry drops, and real partials

Fresh-eyes delta round:

- discardedUsage is now a per-request Usage[] instead of one aggregate,
  so telemetry counts one model call per discarded attempt (calls =
  model requests stays true) and consumers keep per-attempt truth.
- A thrown later attempt no longer erases the accumulated spend: the
  loop attaches the discarded array to the throw and handleRunFailure
  carries it onto the synthetic failure message it persists.
- The session auto-retry drop removes the failed carrier from live
  state while the transcript keeps it; its usage and discardedUsage now
  fold into a live counter that getSessionStats adds, restoring
  live/durable spend parity.
- The interactive orphan replacement distinguishes a discarded empty
  attempt (component dropped) from a real partial interrupted by a run
  failure (finalized in place): visible output is never deleted.
Comment thread packages/coding-agent/src/core/agent-session.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread packages/coding-agent/src/core/agent-session.ts Outdated
…try carriers

Review follow-up: the _droppedRetryUsage accumulator outlived live-state rebuilds - tree navigation could carry phantom spend across branches and a compaction rebuild double-counted carriers that returned to live state. Replaced with a lazy identity-union in getSessionStats: spend sums over live messages plus the current branch entries, deduped by message identity (rebuilds reuse entry objects), so dropped carriers count exactly once and only on their own branch. Also aligns post-compaction live stats with the durable whole-branch spend computation.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 66ceb1f. Configure here.

Comment thread packages/coding-agent/src/core/agent-session.ts
…ats compaction contract

The branch union broke agent-session-stats pins: getSessionStats deliberately reflects the live post-compaction view, so the persisted-but-not-live scan now stops at the latest compaction boundary (pre-compaction spend stays out, exactly like summarized messages). Also removes the write-only _droppedRetryUsage scaffolding the accumulator approach left behind (its removal edit had silently not applied).
Review follow-up: the property-attach on the thrown value lost spend for primitive rejections and threw on frozen/non-extensible errors. The loop now wraps the original throw in its own EmptyTurnRetryFailure (cause = original, message mirrors it); handleRunFailure unwraps and classifies the cause exactly as before. No other consumer branches on the thrown value between loop and Agent (verified: endAgentStreamOnError drops it, runWithLifecycle keys the aborted flag off the signal).
… merge overlapping pins

Cleanup pass, no behavior change: multi-line comments collapsed to one/two-line invariants (~10 lines cut); the plain-Error throw-carry test deleted (redundant - the primitive and frozen table rows pin the same spend-carried behavior strictly more strongly); the two renderer direction pins merged into one classifier pin sharing a scaffold. Fail-unfixed re-proven post-reduction for the renderer classifier, the throw carrier rows, and the goal-on-error charge.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant