Skip to content

fix: collapse OpenAI Responses-API function-call streams into toolCalls - #380

Merged
contextablemark merged 1 commit into
mainfrom
fix/openai-responses-toolcall-collapse
Aug 18, 2026
Merged

fix: collapse OpenAI Responses-API function-call streams into toolCalls#380
contextablemark merged 1 commit into
mainfrom
fix/openai-responses-toolcall-collapse

Conversation

@jpr5

@jpr5 jpr5 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Problem

collapseOpenAISSE() handles OpenAI Responses API text deltas, reasoning, and web-search output items, then a catch-all (if (parsed.type?.startsWith("response.")) continue;) silently skips every other response.* event — including the entire function-call sequence (response.output_item.added with a function_call item, response.function_call_arguments.delta/.done, and response.output_item.done for a function_call). Recording a tool-call-only turn therefore collapses to empty content with no toolCalls (logged: Stream collapse produced empty content — fixture may be incomplete), and replaying that fixture yields an empty assistant turn / no tool invocation.

The older Chat-Completions path already handles tool calls (choices[0].delta.tool_calls), which is why pre-existing fixtures work and only fresh Responses-API recordings break.

This surfaced downstream: the CopilotKit 1.68.1 showcase deploy shifted integrations onto the Responses-API tool-call shape, and any attempt to re-record D4 chat-smoke fixtures produced empty fixtures → whole-column "empty assistant response" flap across staging integrations.

Fix

Add a Responses-API function-call handler in collapseOpenAISSE (before the response.* catch-all), mirroring the Chat-Completions accumulation:

  • response.output_item.added (item.type === "function_call") → start a tool-call accumulator keyed by output_index, capturing call_id as the tool-call id (the id a tool result references, not the internal fc_… item id) and name.
  • response.function_call_arguments.delta → append delta to arguments (get-or-create, robust to delta-before-added).
  • response.function_call_arguments.done → adopt full args only if deltas produced none.
  • response.output_item.done (function_call) → finalize (fill id/name if .added was missed; adopt args only if empty).

Reuses the existing toolCallMap + orderAtoms and the existing emit path, so output is { id, name, arguments } (arguments normalized to valid JSON) — identical to the Chat-Completions shape and the checked-in fixtures. output_index keyspace is disjoint from Chat-Completions tool_calls[].index (a stream is never both shapes).

Proof — local red→green (unit)

New describe("collapseOpenAISSE Responses API function calls") in src/__tests__/stream-collapse.test.ts feeds a synthetic Responses-API function-call SSE (single tool, multiple tools keyed by output_index, and a text-only no-regression case) using the exact wire shape aimock emits.

  • RED (src reverted to origin/main, new tests present): the two tool-call tests fail — result.toolCalls undefined. 2 failed | 1 passed.
  • GREEN (fix applied): 3 passed.

Proof — live record → replay (independent, real OpenAI)

Validated independently by @mark against a real OpenAI key, aimock image built from this exact commit, on the built-in-agent D4 weather flow:

  • Record (--record, Loaded 0 fixture(s)): D4 L3 greeting green, L4 "What's the weather in San Francisco?" green. Journal held exactly 3 correlated 200s (greeting, tool-call turn, post-tool-result turn). The initial weather fixture now contains real toolCalls (no empty-content fallback):
    { "toolCalls": [ { "name": "get_weather", "arguments": "{\"location\":\"San Francisco\"}", "id": "call_WTiZY5LQCFpwqkbDm6CdMGhN" } ] }
    The post-tool-result entry had turnIndex: 1, hasToolResult: true, and a non-empty final assistant response.
  • Fresh replay (new process, no --record/--proxy-only, Loaded 3 fixture(s)): D4 L3+L4 green; replay journal shows the full tool chain; DOM assertions verified the weather card, tool result values (68, 55%, 10 mph), and the final assistant text in [data-testid="copilot-assistant-message"].

Checks

  • stream-collapse.test.ts: 205 passed.
  • Full suite: 5,271 passed / 46 skipped / 0 failed.
  • pnpm build, pnpm format:check, pnpm lint: all clean.

Scope / follow-up

This PR fixes the stream collapser only. It does not address the fixture-capture race (a post-tool-result turn can still be draining after the probe exits, landing a fixture in the next run's dir). That belongs in a separate PR that makes the capture workflow wait for the request journal to drain before moving fixtures / restarting aimock. Per @mark's live findings, a replay drain barrier should key on response.status === 200 && response.fixture != null (on ordinary replay response.source is left unset; record-mode upstream completion is response.source === "proxy").

D4 fixture re-recording stays paused until that capture-workflow barrier lands.

🤖 Generated with Claude Code

https://claude.ai/code/session_01V5dSyNeU6xH9ofPf9ZLaNQ

collapseOpenAISSE handled Responses-API reasoning, web-search, and text
events but let a catch-all skip every other response.* event, dropping the
function-call sequence (output_item.added(function_call),
function_call_arguments.delta/.done, output_item.done(function_call)). A
tool-call-only turn therefore collapsed to empty content with no toolCalls,
recording an empty assistant turn.

Accumulate those events into the existing toolCallMap keyed by output_index,
mirroring the Chat-Completions tool-call path, and capture the Responses
call_id as the tool-call id. Add unit coverage for single/multiple tool-only
Responses streams plus a text-only no-regression case.
@pkg-pr-new

pkg-pr-new Bot commented Aug 18, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@380

commit: 23a8c3c

@contextablemark contextablemark left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Confirmed this correctly preserves Responses API function-call events as toolCalls, with unit coverage and successful live record → fresh replay validation. The journal-drain race is separate follow-up work and is outside this PR’s scope.

@contextablemark
contextablemark merged commit e152680 into main Aug 18, 2026
24 checks passed
@contextablemark
contextablemark deleted the fix/openai-responses-toolcall-collapse branch August 18, 2026 04:53
jpr5 added a commit that referenced this pull request Aug 19, 2026
…up to #380) (#382)

Follow-up to #380: fixes two CR findings on that PR.

**A1 (correctness):** `collapseOpenAISSE`'s Responses-API
`response.output_text.delta` handler only did `content += parsed.delta`
and never pushed a text order-atom, so `buildOrderedBlocks` returned
undefined for any Responses stream — a tool-first Responses stream
(`function_call` before `output_text`) collapsed to a fixture with no
`blocks`, silently losing tool-before-text ordering on replay (the
Responses replay path consumes `blocks`). Fix: push `orderAtoms.push({
kind: "text", text: parsed.delta })` for non-empty deltas, mirroring the
Chat-Completions path. +1 red-green test.

**A2 (coverage):** +8 regression tests for #380's fallback branches
(output_item.done backfill/create/adopt, delta-less done, double-append
guards, no-call_id case). No source change. Each verified genuine by
neutralizing the branch and confirming the test fails.

Net diff vs e152680: `src/stream-collapse.ts` +3,
`src/__tests__/stream-collapse.test.ts` +372.

Branch `cr/pr-380-fixes` was pushed directly (no separate local patch
series); this PR opens it against `main` for CI + review.

---

**Outstanding:** per the CR ledger this escalated Tier1→Tier2 on the A1
correctness finding; a full confirmation round against this post-fix
diff is still required before merge — convergence is NOT yet declared.
contextablemark added a commit that referenced this pull request Aug 19, 2026
…the release (#381)

Adds the four user-facing `[Unreleased]` CHANGELOG entries that landed
since the `v1.38.0` tag but had no entry, so the release notes are
complete before the version is cut.

Docs-only: `git diff --name-only origin/main..HEAD` is `CHANGELOG.md`
and nothing else. Version-neutral — no bump, so it does not trigger
`publish-release.yml`. Merge it before the release bump, or lift the
text into the release commit; either works.

Each entry was written from the actual commit diff, not the subject
line:

- **AG-UI `usage` on `RUN_FINISHED`/`RUN_ERROR`** — `aa68b26` (#378):
the new optional `usage?: AGUITokenUsage[]`, typed and emitted,
numeric-only, emitted only when supplied.
- **Responses-API function-call collapse → `toolCalls`** — `23a8c3c` /
#380: a tool-call-only Responses turn previously collapsed to empty
content; now accumulates into `toolCalls` keyed by `output_index`,
capturing `call_id`.
- **Gemini Live speak-and-call ordering** — `a7d9b64` (#378): a
`serverContent` (audio + text companion) strictly before the `toolCall`,
then `turnComplete`; audio turns keep their companions.
- **CLI signal handlers before readiness** — `1079b46` (#374):
`SIGINT`/`SIGTERM` registered before the readiness log, closing the
window where a supervisor's `SIGTERM` killed the process ungracefully
(55/60 → 0/60). The residual — a signal strictly before readiness — is
stated in the entry, drawn from code placement rather than a documented
commit note.

Verified: prettier clean, commitlint RC 0, no version-bearing file
touched.
@jpr5 jpr5 mentioned this pull request Aug 19, 2026
contextablemark added a commit that referenced this pull request Aug 19, 2026
Release cut for v1.39.0. No source changes.

- Bumps root `package.json` version 1.38.0 → 1.39.0.
- Converts the merged `[Unreleased]` material into a dated `## [1.39.0]
- 2026-08-18` entry and opens a fresh empty `[Unreleased]`. All entries
preserved in order (Added / Changed / Deprecated / Fixed), covering
#382, #380, #378, #374, and the reset-canonicalization work (#358).

Follow-up (not in this PR):
`packages/aimock-pytest/src/aimock_pytest/_version.py` still pins
`AIMOCK_VERSION = "1.38.0"`; bump it on the aimock-pytest release
cadence per the changelog note.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants