fix(xai): self-heal the realtime chat context when an item's anchor is missing#6392
Open
thecarlmartin wants to merge 1 commit into
Open
fix(xai): self-heal the realtime chat context when an item's anchor is missing#6392thecarlmartin wants to merge 1 commit into
thecarlmartin wants to merge 1 commit into
Conversation
…s missing An interruption's truncation/delete traffic can desync the plugin's local RemoteChatContext from xAI's server-side item list (the conversation.item.deleted event carries no item id, so the delete is guessed and can drop the wrong item). After that, a server conversation.item.added referencing a still-present-server item finds no local anchor, insert() raises previous_item_id-not-found, and the base handler drops the item -- which strands every later item that chains off it, so the whole context cascades into nothing and the model stalls. Anchor a newly-added item to the current tail whenever its previous_item_id is missing locally, so a single desync self-heals (at worst one item is slightly out of order) instead of corrupting the rest of the conversation. Fixes the unrecoverable-cascade case in livekit#6391.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes an unrecoverable conversation-state cascade on
livekit-plugins-xairealtime calls (filed as #6391).On an interruption,
truncate()deletes items. xAI'sconversation.item.deletedevent arrives with an emptyitem_id, so_handle_conversion_item_deletedguesses which item was removed (the first pending delete future). When several deletes are in flight the guess can pick the wrong item, desyncing the localRemoteChatContextfrom xAI's server-side list. After that, a serverconversation.item.addedreferencing a still-present-server item finds no local anchor,RemoteChatContext.insert()raisesprevious_item_id … not found, and the base handler drops the item — which strands every later item that anchors to it. The whole context cascades into nothing and the model stalls (multi-second silences, then hardItem not foundAPI errors).In one ~2:16 trace the context broke ~69s in and never recovered:
Fix
Anchor a newly-added item to the current tail whenever its
previous_item_idis missing from the local context, so a single desync self-heals (at worst one item is slightly out of order) instead of dropping the item and cascading:This keeps
RemoteChatContext.insert()strict and is scoped to the xAI handler (which already fills aNoneprevious_item_idfrom the tail). A more thorough fix would also make_handle_conversion_item_deletedidentify the deleted item precisely instead of guessing, but that needs the deleted item's id, which xAI's event doesn't currently provide — so this hardens the recovery path regardless.Tests
Adds two tests to
tests/test_realtime/test_xai_realtime_model.py:test_item_added_with_missing_anchor_self_heals_to_tail— an item whose anchor is absent is kept and lands at the tail (previously dropped).test_item_added_missing_anchor_does_not_cascade— a following item that anchors to the self-healed item also inserts (the cascade is broken).All tests in that file pass locally. I don't have access to run the live-xAI integration suite, so a maintainer verification against the real API would be appreciated.
Fixes #6391.