Skip to content

Batch streamed message.updates through the rAF flush#2413

Merged
gary149 merged 1 commit into
mainfrom
perf/stream-updates-raf
Jul 2, 2026
Merged

Batch streamed message.updates through the rAF flush#2413
gary149 merged 1 commit into
mainfrom
perf/stream-updates-raf

Conversation

@gary149

@gary149 gary149 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

What

During streaming, .content updates are batched to animation frames, but message.updates was reassigned synchronously on every network chunk. At 100+ chunks/s (fast providers), each assignment re-triggers the full markdown block derivation, so the app does per-chunk work its own rAF batching was designed to avoid.

This PR routes .updates through the same flush as .content: the stream loop maintains a local updatesBuffer (all in-loop readers use it, including the token-merge logic and the final-answer tool check), and the reactive field is assigned only at flush points. Non-stream updates (tool status, final answer, errors) flush immediately, exactly as before, so tool visibility timing is unchanged.

The navigation-abort exit path now flushes the buffers before returning. That path skips the post-stream refresh by design, so anything left in the buffers would otherwise be dropped from the UI for good.

Verification

  • Streamed real generations against a local production build, including tool status updates and an error event, with no rendering differences
  • Full test suite and svelte-check green
  • Reviewed the loop for any remaining direct reads of message.updates: none remain (single flush point)

Part of a performance series measured from a live profile of hf.co/chat. Each PR is file-disjoint and merges independently, in any order: #2409 (compression), #2410 (models payload), #2411 (markdown pipeline), #2412 (conversation switching), #2413 (stream update batching), #2414 (send handler DB), #2415 (sidebar hydration).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant