Skip to content

fix(xai): self-heal the realtime chat context when an item's anchor is missing#6392

Open
thecarlmartin wants to merge 1 commit into
livekit:mainfrom
thecarlmartin:fix/xai-realtime-item-desync-selfheal
Open

fix(xai): self-heal the realtime chat context when an item's anchor is missing#6392
thecarlmartin wants to merge 1 commit into
livekit:mainfrom
thecarlmartin:fix/xai-realtime-item-desync-selfheal

Conversation

@thecarlmartin

Copy link
Copy Markdown

Summary

Fixes an unrecoverable conversation-state cascade on livekit-plugins-xai realtime calls (filed as #6391).

On an interruption, truncate() deletes items. xAI's conversation.item.deleted event arrives with an empty item_id, so _handle_conversion_item_deleted guesses which item was removed (the first pending delete future). When several deletes are in flight the guess can pick the wrong item, desyncing the local RemoteChatContext from xAI's server-side list. After that, a server conversation.item.added referencing a still-present-server item finds no local anchor, RemoteChatContext.insert() raises previous_item_id … not found, and the base handler drops the item — which strands every later item that anchors to it. The whole context cascades into nothing and the model stalls (multi-second silences, then hard Item not found API errors).

In one ~2:16 trace the context broke ~69s in and never recovered:

WARN failed to insert item `f6dce9e1…`: previous_item_id `55273a67…` not found
WARN failed to insert item `f8cccbcc…`: previous_item_id `f6dce9e1…` not found   # chains off the dropped f6dce9e1
WARN failed to insert item `c1581c9d…`: previous_item_id `f6dce9e1…` not found
...

Fix

Anchor a newly-added item to the current tail whenever its previous_item_id is missing from the local context, so a single desync self-heals (at worst one item is slightly out of order) instead of dropping the item and cascading:

prev_id = event.previous_item_id
if prev_id is None or self._remote_chat_ctx.get(prev_id) is None:
    prev_id = self._remote_chat_ctx._tail.item.id if self._remote_chat_ctx._tail else None
event.previous_item_id = prev_id

This keeps RemoteChatContext.insert() strict and is scoped to the xAI handler (which already fills a None previous_item_id from the tail). A more thorough fix would also make _handle_conversion_item_deleted identify the deleted item precisely instead of guessing, but that needs the deleted item's id, which xAI's event doesn't currently provide — so this hardens the recovery path regardless.

Tests

Adds two tests to tests/test_realtime/test_xai_realtime_model.py:

  • test_item_added_with_missing_anchor_self_heals_to_tail — an item whose anchor is absent is kept and lands at the tail (previously dropped).
  • test_item_added_missing_anchor_does_not_cascade — a following item that anchors to the self-healed item also inserts (the cascade is broken).

All tests in that file pass locally. I don't have access to run the live-xAI integration suite, so a maintainer verification against the real API would be appreciated.

Fixes #6391.

…s missing

An interruption's truncation/delete traffic can desync the plugin's local
RemoteChatContext from xAI's server-side item list (the conversation.item.deleted
event carries no item id, so the delete is guessed and can drop the wrong item).
After that, a server conversation.item.added referencing a still-present-server item
finds no local anchor, insert() raises previous_item_id-not-found, and the base
handler drops the item -- which strands every later item that chains off it, so the
whole context cascades into nothing and the model stalls.

Anchor a newly-added item to the current tail whenever its previous_item_id is
missing locally, so a single desync self-heals (at worst one item is slightly out of
order) instead of corrupting the rest of the conversation.

Fixes the unrecoverable-cascade case in livekit#6391.
@thecarlmartin thecarlmartin requested a review from a team as a code owner July 12, 2026 06:21
@CLAassistant

CLAassistant commented Jul 12, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xAI realtime: an interruption desyncs the local chat context, then every later item is dropped (unrecoverable)

2 participants