-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Open
Labels
type:bugSomething isn't workingSomething isn't working
Description
Describe the bug
The Qwen3 model cannot obtain the usage for streaming.
Reproduce the code:
from haystack.dataclasses import ChatMessage
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.utils import Secret
# Initialize the Chat Generator with the addition tool
chat_generator = OpenAIChatGenerator(
api_key=Secret.from_token("sk-xxxxx"),
api_base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
model="qwen3-max",
streaming_callback=lambda msg: print(msg),
)
res=chat_generator.run([ChatMessage.from_user("10 + 238")])
print(res)I attempted to use the deepseek model and was able to obtain the usage.
Using Qwen3 fails to obtain the usage.
However, when I attempted to test using the openai or langchain library, I found that the qwen3 model could also obtain it. I suspect there might be differences in parsing within the OpenAIChatGenerator.
import os
from openai import OpenAI
client = OpenAI(
api_key="sk-xxxx",
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)
completion = client.chat.completions.create(
model="qwen-plus",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "tell me a story"}
],
stream=True,
stream_options={"include_usage": True}
)
for chunk in completion:
print(chunk)
System:
- Haystack version : 2.20.0rc1
Metadata
Metadata
Assignees
Labels
type:bugSomething isn't workingSomething isn't working