Skip to content

ChatDatabricks Responses API streaming emits the final text twice #464

Description

@MarcusKoolaard

_convert_responses_api_chunk_to_lc_chunk suppresses the terminal full-text block only when the immediately previous chunk is response.output_text.delta. Providers close a message with response.output_text.done and response.content_part.done first, so the check never fires and the completed text is appended on top of the streamed deltas.

Repro (no network, released 0.20.0) — feeds the documented event order through _stream with a stubbed client:

EVENTS = [
    ResponseTextDeltaEvent.model_construct(type="response.output_text.delta", item_id="msg_1", delta="hello"),
    ResponseTextDeltaEvent.model_construct(type="response.output_text.delta", item_id="msg_1", delta=" world"),
    ResponseTextDoneEvent.model_construct(type="response.output_text.done", item_id="msg_1", text="hello world"),
    ResponseContentPartDoneEvent.model_construct(type="response.content_part.done", item_id="msg_1",
        part=ResponseOutputText.model_construct(type="output_text", text="hello world", annotations=[])),
    ResponseOutputItemDoneEvent.model_construct(type="response.output_item.done",
        item=ResponseOutputMessage.model_construct(type="message", id="msg_1",
            content=[ResponseOutputText.model_construct(type="output_text", text="hello world")])),
]
llm = ChatDatabricks(model="m", use_responses_api=True)  # client stubbed to yield EVENTS
merged = functools.reduce(operator.add, llm.stream("hi"))

Actual:

[{'type': 'text', 'text': 'hello'}, {'type': 'text', 'text': ' world'}, {'type': 'text', 'text': 'hello world', 'annotations': []}]
joined -> 'hello worldhello world'

Expected: 'hello world'.

Also observed live against Unity AI Gateway (use_ai_gateway_native_api=True, Azure Foundry GPT-5.6), where every agent reply came back doubled.

Fix: treat response.output_text.done and response.content_part.done with a matching item_id as "already streamed" alongside response.output_text.delta. Existing unit tests only cover a delta immediately preceding output_item.done, which is why this passes CI. Happy to send a PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions