Skip to content

Silent PCM from NativeAudioSink / AudioStream for SIP-transcoded Opus after libwebrtc m144 #1265

Description

@trippyogi

Summary

Downstream report: Python rtc.AudioStream returns correctly shaped but all-zero PCM for SIP-originated Opus tracks after the FFI stack moved from WebRTC m137 to m144. Same SIP room works on older FFI (Python livekit==1.0.23 / ffi ~0.12.42 / m137) and fails on livekit==1.1.9 (ffi 0.12.60 / m144). Current python-sdks main (1.1.13, ffi 0.12.68) is still on m144 with no receive-path change that would fix this.

Upstream issue: livekit/python-sdks#690

Symptom

  • Track: SIP participant, mime_type=audio/opus (livekit-sip PCMU/PCMA → Opus).
  • Frames arrive at ~100 fps, 48000 Hz, 480 samples, but abs_max == 0 for every frame.
  • Not muted; subscription OK; server/SIP upgrades did not help.

Why this looks like rust-sdks / libwebrtc (not Python)

Python AudioStream only requests AUDIO_STREAM_NATIVE and copies PCM from FFI OwnedAudioFrameBuffer. The Python deltas in 1.0.23→1.1.x (queue filter, queue_size_frames=0, memoryview hardening) cannot invent continuous silence with correct framing.

Receive path of interest:

AudioTrackSinkInterface::OnDatawebrtc-sys NativeAudioSink::OnData (optional RemixAndResample) → libwebrtc::NativeAudioStreamlivekit-ffi audio_stream events.

Strong bisect hint

Client WebRTC pin
Python 1.0.23 / 1.1.0 / 1.1.3 m137_release
Python 1.1.4+ (incl. 1.1.9, 1.1.13) m144_release (PR #965)

Please confirm whether 1.1.3 works and 1.1.4 fails on the same SIP track.

Asks

  1. Reproduce with rust NativeAudioStream / FFI against a SIP Opus track (or capture RTP and replay).
  2. Compare decoder/PLC stats (concealed_samples, etc.) on m137 vs m144.
  3. Check whether m144 Opus / RED / DTX handling rejects or conceals livekit-sip’s transcoder output.
  4. Once fixed, cut a livekit-ffi release so python-sdks / node can bump the submodule.

Investigation notes

Full write-up on the python-sdks investigation branch: ISSUE-690-STATUS.md on issue-690-investigate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions