Environment: @livekit/agents 1.4.5 · @livekit/agents-plugin-elevenlabs 1.4.5 · @livekit/agents-plugin-deepgram 1.4.5 · @livekit/rtc-node 0.13.29 · Node v24.16.0 · Windows 11 (issue 1's contamination variant also reproduced on a LiveKit Cloud-hosted worker).
Pipeline: Silero VAD + Deepgram nova-3 (streaming) + external LLM + ElevenLabs TTS (eleven_multilingual_v2) + EnglishModel turn detection + interruption: { minDuration: 800, minWords: 2, mode: 'adaptive' }.
Component: agents-js voice/generation.ts (forwardAudio) / transcription/synchronizer.ts / room_io/_output.ts + rtc-node AudioSource.captureFrame.
Summary. During a session with several rapid interruptions (explicit user barge-ins killing replies in quick succession), the audio forwarding chain threw an unhandled promise rejection:
Unhandled promise rejection
Error: an RtcError occurred: InvalidState - sample_rate and num_channels don't match
at AudioSource.captureFrame (rtc-node/src/audio_source.ts:143)
at ParticipantAudioOutput.captureFrame (agents/src/voice/room_io/_output.ts:446)
at SyncedAudioOutput.captureFrame (agents/src/voice/transcription/synchronizer.ts:734)
at forwardAudio (agents/src/voice/generation.ts:897)
After this, the session is permanently degraded: subsequent committed user turns log end of user turn → AgentActivity.userTurnCompleted started but no speech handle is ever created again; the pause machinery churns (resumed false interrupted speech), and the session stays deaf-mute until the worker restarts. Nothing catches or recovers from the rejection.
Context/likely trigger. ~50s earlier the ElevenLabs plugin hit its context cap and retried on a fresh WebSocket (see companion report). The fatal frame plausibly came from a retry/replacement TTS stream whose output format differed from the AudioSource's configured sample_rate/num_channels. Bracketing warnings: rotateSegment called while previous segment is still being rotated, SegmentSynchronizerImpl.markPlaybackFinished called before text/audio input is done, flush called while playback is in progress (ERROR).
Asks. (1) forwardAudio/captureFrame failures should be caught and surfaced as a recoverable pipeline error (close the speech, keep the session usable) rather than an unhandled rejection; (2) frames should be resampled/validated against the AudioSource format instead of hard-failing; (3) whatever leaves userTurnCompleted unable to create speech handles after this fault should fail loudly instead of silently wedging.
Expected. A single malformed audio frame must not end the session.
Environment:
@livekit/agents1.4.5 ·@livekit/agents-plugin-elevenlabs1.4.5 ·@livekit/agents-plugin-deepgram1.4.5 ·@livekit/rtc-node0.13.29 · Node v24.16.0 · Windows 11 (issue 1's contamination variant also reproduced on a LiveKit Cloud-hosted worker).Pipeline: Silero VAD + Deepgram nova-3 (streaming) + external LLM + ElevenLabs TTS (
eleven_multilingual_v2) +EnglishModelturn detection +interruption: { minDuration: 800, minWords: 2, mode: 'adaptive' }.Component:
agents-jsvoice/generation.ts(forwardAudio) /transcription/synchronizer.ts/room_io/_output.ts+rtc-nodeAudioSource.captureFrame.Summary. During a session with several rapid interruptions (explicit user barge-ins killing replies in quick succession), the audio forwarding chain threw an unhandled promise rejection:
After this, the session is permanently degraded: subsequent committed user turns log
end of user turn→AgentActivity.userTurnCompleted startedbut no speech handle is ever created again; the pause machinery churns (resumed false interrupted speech), and the session stays deaf-mute until the worker restarts. Nothing catches or recovers from the rejection.Context/likely trigger. ~50s earlier the ElevenLabs plugin hit its context cap and retried on a fresh WebSocket (see companion report). The fatal frame plausibly came from a retry/replacement TTS stream whose output format differed from the
AudioSource's configured sample_rate/num_channels. Bracketing warnings:rotateSegment called while previous segment is still being rotated,SegmentSynchronizerImpl.markPlaybackFinished called before text/audio input is done,flush called while playback is in progress(ERROR).Asks. (1)
forwardAudio/captureFramefailures should be caught and surfaced as a recoverable pipeline error (close the speech, keep the session usable) rather than an unhandled rejection; (2) frames should be resampled/validated against theAudioSourceformat instead of hard-failing; (3) whatever leavesuserTurnCompletedunable to create speech handles after this fault should fail loudly instead of silently wedging.Expected. A single malformed audio frame must not end the session.