Skip to content

The Qwen3 model cannot obtain the meta usage for streaming. #10258

@Silence-Well

Description

@Silence-Well

Describe the bug
The Qwen3 model cannot obtain the usage for streaming.

Reproduce the code:

from haystack.dataclasses import ChatMessage
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.utils import Secret

# Initialize the Chat Generator with the addition tool
chat_generator = OpenAIChatGenerator(
    api_key=Secret.from_token("sk-xxxxx"),
    api_base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
    model="qwen3-max",
    streaming_callback=lambda msg: print(msg),
)

res=chat_generator.run([ChatMessage.from_user("10 + 238")])
print(res)

I attempted to use the deepseek model and was able to obtain the usage.

Image

Using Qwen3 fails to obtain the usage.

Image

However, when I attempted to test using the openai or langchain library, I found that the qwen3 model could also obtain it. I suspect there might be differences in parsing within the OpenAIChatGenerator.

import os
from openai import OpenAI

client = OpenAI(
    api_key="sk-xxxx", 
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)

completion = client.chat.completions.create(
    model="qwen-plus",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "tell me a story"}
    ],
    stream=True,
    stream_options={"include_usage": True}
)


for chunk in completion:
    print(chunk)

Image

System:

  • Haystack version : 2.20.0rc1

Metadata

Metadata

Assignees

Labels

type:bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions