Skip to content

elevenlabs plugin: rapid supersedes exhaust the 5-context WebSocket cap (max_active_conversations) + teardown instability #1985

Description

@Bab95d23

Environment: @livekit/agents 1.4.5 · @livekit/agents-plugin-elevenlabs 1.4.5 · @livekit/agents-plugin-deepgram 1.4.5 · @livekit/rtc-node 0.13.29 · Node v24.16.0 · Windows 11 (issue 1's contamination variant also reproduced on a LiveKit Cloud-hosted worker).
Pipeline: Silero VAD + Deepgram nova-3 (streaming) + external LLM + ElevenLabs TTS (eleven_multilingual_v2) + EnglishModel turn detection + interruption: { minDuration: 800, minWords: 2, mode: 'adaptive' }.

Component: agents-plugin-elevenlabs tts.ts (multi-context WebSocket).

Summary. With rapid successive interruptions (user kills a reply; a new reply starts; kills that too — several cycles within ~60s), the shared multi-context WebSocket exceeds ElevenLabs' 5-context limit:

ERROR elevenlabs tts returned error
  message: "Maximum simultaneous contexts per WebSocket connection exceeded (5). Please close an existing context before opening a new one."
  error: "max_active_conversations", code: 1008
WARN recv loop error: APIStatusError: ElevenLabs websocket connection closed unexpectedly
WARN failed to synthesize speech, retrying in 0.1ms

i.e., contexts belonging to already-interrupted speeches are not closed fast enough (context close is queued/asynchronous via close_context; #activeContexts cleanup lags the kill), so live + zombie contexts exceed the cap, the provider closes the socket (1008), and the retry path introduces further instability (in our incident, the retry window immediately preceded a fatal output-format mismatch — see companion report; causal identity not established). Related teardown warnings cluster around rapid supersedes: SegmentSynchronizerImpl.markPlaybackFinished called before text/audio input is done (textDone:false audioDone:true), rotateSegment called while previous segment is still being rotated, SegmentSynchronizerImpl.onPlaybackStarted called after startFuture is set. We have also observed a possibly related truncated/failed synthesis on a rapid double-commit turn on a Cloud-hosted worker (no playout completed, same markPlaybackFinished signature) — flagged as possibly related rapid-supersession / TTS-segment instability; causal identity not established.

Asks. (1) Close/abort a speech's TTS context synchronously-with-priority when the speech is interrupted (before opening the next context); (2) cap-awareness: block/queue new context creation when #activeContexts is at the provider limit instead of triggering a provider-side socket close; (3) on max_active_conversations, prefer closing zombie contexts over reconnecting the socket while segments are mid-rotation.

Expected. Rapid legitimate barge-ins are a core voice-agent pattern; the TTS transport should degrade by queueing, not by socket resets mid-session.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions