buzz-agent: move proactive context handoff to end of turn - #6584
buzz-agent: move proactive context handoff to end of turn#6584tlongwell-block wants to merge 2 commits into
Conversation
The proactive handoff gate ran at the start of every tool-loop round. That is the worst moment to compact: the turn's working set (file reads, tool results) is exactly what the model is using, and summarizing it away mid-task forces a re-read storm that refills the window. The round-start gate also projected history growth at a conservative bytes-per-token rate, so one large tool result could trip it on its own. Run the gate once, after `run()` returns, so the NEXT turn starts on a fresh context and the current turn keeps what it was working with. - handoff.rs: `maybe_handoff(&mut attempts)` → `end_of_turn_handoff()`. Gates on `max_handoffs == 0` (disabled) and `should_handoff()`, which reads the final successful request's measured usage. `handoff()` takes a `Reseat` flag: end of turn does not re-append the already-answered prompt; the reactive context-400 path keeps `Reseat::LivePrompt` because it continues the same turn. - agent.rs: drop the per-turn `handoff_attempts` counter and the round- start gate; `truncate_history` is now unconditional after `drain_steers()`. Mid-turn overflow is handled only by the reactive context-400 ladder. - lib.rs: call `end_of_turn_handoff()` after `ctx.run()` when the result is `Ok` and the stop reason is not `Cancelled` — `run()` reports cancellation as `Ok(StopReason::Cancelled)`, so an `is_ok()` gate would spend a summarize round trip the user just asked us to stop. - config.rs / README.md: `BUZZ_AGENT_MAX_HANDOFFS` is now a switch — 0 disables proactive compaction, any positive value enables it; the gate runs at most once per turn by construction so the magnitude no longer bounds anything. Reactive recovery bypasses it as before. - The 90% threshold is unchanged; timing and threshold are not tuned in the same change. Tests (tests/regressions.rs) rewritten for the new boundary: `token_usage_over_budget_triggers_handoff` asserts the summarize lands inside the over-threshold turn and that the next request is [handoff block, new prompt] with no re-seated answered prompt; `history_growth_mid_turn_does_not_trigger_handoff` proves a 6 KB tool result mid-turn fires nothing and that the gate reads the final request's usage, not the turn's cumulative input; `cancelled_turn_skips_end_of_turn_handoff` cancels during an in-flight provider call; `handoff_fires_on_each_over_threshold_turn`, `max_handoffs_zero_disables_end_of_turn_handoff`, and `failed_end_of_turn_summarize_leaves_history_intact` replace the per-turn-cap tests. Existing handoff fixtures shift one request because the summarize now lands at the end of the seeding turn. Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz> Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Review finding (Wren): `should_handoff()` still routed its measured-usage
arm through `projected_handoff_input_tokens()`, which added a 1-byte/token
estimate of history appended since the usage was recorded. At end of turn
the only such history is the final assistant reply, so an under-threshold
turn could compact merely because its answer was long (8,500 measured +
a 500-byte reply crossed a 9,000 gate). That contradicted the documented
contract ("final request used 90%") and kept the estimator this change
set out to retire.
- handoff.rs: the `Some(measured_tokens)` arm compares the provider's
count directly against `token_threshold`. `projected_handoff_input_tokens`
is deleted; the handoff log line's "before" count uses the measurement
when present, else the byte-derived bound. The `None` byte fallback is
unchanged. Doc comments no longer claim "nothing is appended after" the
final request.
- agent.rs / lib.rs: remove `last_request_history_bytes` (RunCtx field,
Session field, acquire/run_prompt plumbing, the two clears) — it existed
only to feed the estimator.
- tests/regressions.rs: add `large_final_reply_does_not_trigger_end_of_turn_handoff`
(usage 8500 < 9000, 2 KB reply, exactly one request). Fails on the
parent commit; passes here. Reword
`reactive_reset_clears_usage_baseline_so_the_gate_is_not_blind` — the
scenario still holds (a stale sub-threshold reading blinds the gate),
the "paired bytes / grown" mechanics it described do not.
Co-authored-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
Signed-off-by: Meli <5aaa86bce934fc3445fc254aab560a40923f10252f92107e665073dede0e04d3@buzz.block.builderlab.xyz>
wesbillman
left a comment
There was a problem hiding this comment.
Carl, an automated reviewer, commenting via Wes’s GitHub account.
I reviewed exact head 7f3b63ea7fc79ce3881c6a78931325436ea28f29 against base 4baccd5394d6166bb68ff03b24e376e322281a59. Requesting changes for two user-visible correctness defects introduced by moving the handoff await to the turn boundary.
-
The final-request gate can use an earlier round’s usage.
agent.rspreserveslast_request_input_tokenswhen a response omits usage, thenshould_handoff()treats that retained value as the final request’s measurement. In a two-round tool turn where round 1 reports 950/1000 tokens and the terminal response has no usage block, the agent performs an unwanted summary (three provider requests instead of two). This contradicts the implementation and README contract that the turn’s final request controls the gate, and can either compact an under-threshold final context or miss compaction after an over-threshold final request. Track whether the current/final request reported usage; on absence use the documented byte fallback rather than a prior round, and add this multi-round regression. -
Steers accepted during the synchronous end-of-turn summary are silently lost.
run_prompt()awaitsend_of_turn_handoff()before clearingbusy,active_run_id, andsteer_tx. Oncectx.run()has returned there is no next round to calldrain_steers(), but a steer arriving while the summary is in flight is still accepted with{runId,messageId}and queued into that orphaned receiver. I reproduced this with a hanging summarize request: the late steer returned success and could never reach provider history. Stop advertising/accepting steers before awaiting the post-turn summary, or explicitly persist and route them to a subsequent turn; add a regression around this interval.
Non-blocking test gap: failed_end_of_turn_summarize_leaves_history_intact says the next over-threshold turn retries, but its second terminal response reports usage 10, so it only proves history preservation, not retry. Use a second over-threshold response and assert a second summarize attempt.
Focused validation at exact head:
- final-response-missing-usage repro: failed,
3captured requests vs expected2; - steer-during-handoff repro: failed because the completed turn accepted the steer instead of rejecting/routing it;
- existing focused steer test remained green under the minimal state-lifetime correction I used to validate the seam.
GitHub CI is green and the PR is mergeable, but those checks do not cover these boundary races/contracts.
Summary
Move
buzz-agent's proactive context handoff from the start of every tool-loop round to once, at the end of the turn.Problem. The round-start gate compacted while the turn's working set (file reads, tool results) was exactly what the model was still using. Summarizing it away mid-task forced a re-read storm that refilled the window. The gate also projected history growth at 1 byte/token, so a single large tool result could trip it on its own.
Change.
handoff.rs:maybe_handoff(&mut attempts)→end_of_turn_handoff(). Gates onmax_handoffs == 0(disabled) andshould_handoff(), which now compares the provider's measured input usage for the turn's final request directly against the 90% threshold. The growth estimator (projected_handoff_input_tokens) and its byte baseline (last_request_history_bytes) are deleted end-to-end. The pre-usage byte fallback is unchanged.handoff()takesenum Reseat { LivePrompt, None }: end of turn does not re-append the already-answered prompt; the reactive context-400 path keepsLivePromptbecause it continues the same turn.agent.rs: per-turnhandoff_attemptscounter and round-start gate removed;truncate_historyis unconditional afterdrain_steers(). Mid-turn overflow is handled only by the reactive context-400 ladder (unchanged: in-loop retry, round refund, independent 3-rung bound).lib.rs:end_of_turn_handoff()runs afterctx.run()when the result isOkand the stop reason is notCancelled—run()reports cancellation asOk(StopReason::Cancelled), so anis_ok()gate would spend a summarize the user just asked us to stop.BUZZ_AGENT_MAX_HANDOFFSis now a switch:0disables proactive compaction, any positive value enables it (≤1 per turn by construction). Reactive recovery bypasses it as before.config.rs/README.mdupdated.Deliberately not changed: the 90% threshold. Timing and threshold are not tuned in the same PR; measure first.
Related issue
None found. Supersedes the relay-hosted draft (
buzz://PR #1 by Pinky/Brain) that proposed the same move.Testing
tests/regressions.rsrewritten for the new boundary (54 regressions, all green):token_usage_over_budget_triggers_handoff— summarize lands inside the over-threshold turn; next request's user items are exactly[Context Handoff], new prompt (no re-seated answered prompt).history_growth_mid_turn_does_not_trigger_handoff— 6 KB tool result mid-turn fires nothing; gate reads the final request's usage (8500+8900 cumulative > 9000 gate, final 8900 < 9000 → no handoff).large_final_reply_does_not_trigger_end_of_turn_handoff— usage 8500 < 9000 with a 2 KB reply → exactly one request. Fails on the first commit, passes on the second (this was the review blocker: the old estimator charged the reply at 1 byte/token).cancelled_turn_skips_end_of_turn_handoff— cancel during an in-flight provider call →stopReason: cancelled, no summarize.handoff_fires_on_each_over_threshold_turn,max_handoffs_zero_disables_end_of_turn_handoff,failed_end_of_turn_summarize_leaves_history_intactreplace the per-turn-cap tests.Mutants caught:
is_ok()gate;Reseat::LivePromptat end of turn; mid-turn gate restored; reply-bytes growth term re-added.Verification at
7f3b63ea:cargo test -p buzz-agentfull package green (456 unit / 54 regressions); fmt + clippy-D warningsclean; pre-push rust-tests + desktop-tauri-checks green.Independent review: Wren — Minimalness 9/10, Elegance 9/10, Correctness 9/10 at
7f3b63ea, no findings (full suite re-run in a clean worktree).Live e2e (Max): built from a clean worktree at
7f3b63ea, real OpenAIgpt-5.5through a recording proxy. 4,000-token window: request 1 reported 4,701 input tokens, 14 KB tool result, turn completed with no mid-turn handoff; summarize +handoff #1only after the final response; next turn's provider history was exactly[Context Handoff]+ new prompt. Cancel mid-provider-call →cancelled, no handoff. 8,000-token window: 1,154 output tokens appended on a turn ending at measured input 6,413 → no handoff. Receipt:.scratch/eot-live-7f3b63ea/on the dev Mac.