Skip to content

fix(voice): honor client config for streamed STT - #4532

Open
sylvesterkaczmarek wants to merge 13 commits into
openai:mainfrom
sylvesterkaczmarek:fix/stt-websocket-client-config
Open

fix(voice): honor client config for streamed STT#4532
sylvesterkaczmarek wants to merge 13 commits into
openai:mainfrom
sylvesterkaczmarek:fix/stt-websocket-client-config

Conversation

@sylvesterkaczmarek

@sylvesterkaczmarek sylvesterkaczmarek commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Make streamed OpenAI STT honor the supplied AsyncOpenAI client's endpoint, query, headers, and dynamic credentials instead of bypassing client configuration.

OpenAISTTTranscriptionSession previously connected directly to:

wss://api.openai.com/v1/realtime?intent=transcription

and constructed its authorization header directly from client.api_key. That bypassed client routing/query/header configuration and could also materialize an empty or stale credential when the client uses a callable API-key provider.

Fix

  • derive WebSocket routing from the configured OpenAI client
  • merge client.default_query with the same Omit / NotGiven semantics as Responses WebSocket
  • propagate auth/default headers while filtering omission sentinels
  • refresh callable/rotating API keys before materializing handshake headers
  • keep intent=transcription and OpenAI-Log-Session: 1 as STT-specific call-site values
  • centralize client-derived WebSocket URL/query/header/credential normalization in a shared internal helper used by both Responses WebSocket and streamed STT

Test plan

Coverage verifies:

  • custom base_url and websocket_base_url
  • merged client query parameters and omission semantics
  • auth, organization/project, and custom headers
  • omitted organization/project headers are not stringified
  • callable API keys are refreshed before the STT handshake
  • existing STT connection tests use client doubles with the routing/query/header attributes read by this path

Risk

Low. With the default OpenAI client configuration, streamed STT still connects to wss://api.openai.com/v1/realtime?intent=transcription. Behavior changes only for configured routing, headers, query parameters, or dynamic credentials.

Issue number

None. Found while auditing parity between streamed and non-streamed STT client configuration.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0d40f222a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/voice/models/openai_stt.py Outdated
Comment thread src/agents/voice/models/openai_stt.py Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

Note

You have reached your Codex usage limits. You can see your limits in the Codex usage dashboard.

1 similar comment
@chatgpt-codex-connector

Copy link
Copy Markdown

Note

You have reached your Codex usage limits. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b6b1a35db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/voice/models/openai_stt.py Outdated
Comment thread src/agents/voice/models/openai_stt.py Outdated

@seratch seratch left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. The underlying issue is valid: streamed STT currently bypasses the supplied AsyncOpenAI client's endpoint, query, and header configuration.
Before merging, please update the existing STT connection test doubles with the URL, query, auth-header, and default-header attributes now read by this path; those tests currently fail before reaching the mocked connection. Please also incorporate the callable-key refresh and regression coverage from #4537 before materializing auth headers.

Finally, keep the client-derived WebSocket URL, query, header, and credential normalization in a shared internal helper used by both Responses WebSocket and streamed STT, while leaving each transport's endpoint suffix and fixed parameters at its call site.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bcc8d57066

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +13 to +14
def _is_openai_omitted_value(value: Any) -> bool:
return isinstance(value, Omit | NotGiven)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reuse one WebSocket normalization path

This new module duplicates the omission detection, credential refresh, header merge, and base/query normalization already implemented in openai_responses.py, while the Responses transport continues using its private copies. Consequently, fixes to these client semantics can land in only one transport—the preceding revisions already needed several corrections in this exact area. Route both WebSocket transports through this helper, or adapt STT to the existing normalization, so there is one source of truth.

AGENTS.md reference: AGENTS.md:L93-L93

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 101108b502

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +43 to +45
for key, value in source.items():
if _is_openai_omitted_value(value):
continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Remove inherited headers when defaults use Omit

When an AsyncOpenAI client uses default_headers={"Authorization": omit}—for example, to prevent an OpenAI key from being forwarded to a custom WebSocket proxy—the auth source has already inserted Authorization, and this branch merely skips the later sentinel instead of removing that inherited header. Although the sentinel is no longer stringified, the handshake still sends the credential despite the explicit omission; handle Omit in this layered source case by deleting any case-insensitive existing header, as the extra_headers path already does.

Useful? React with 👍 / 👎.

@sylvesterkaczmarek

Copy link
Copy Markdown
Contributor Author

Addressed. I moved the callable-key refresh/regression from #4537 into this PR, updated the existing STT connection doubles with the client routing/query/header attributes, and centralized the OpenAI-client WebSocket URL/query/header/credential normalization so Responses WebSocket and streamed STT share the same helper while keeping their endpoint-specific suffixes and fixed parameters at the call sites.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants