Skip to content

fix(hermes-memory): append chat turns in place instead of rewriting the transcript - #98

Merged
CahidArda merged 2 commits into
mainfrom
fix/hermes-memory-append-turns
Nov 5, 2026
Merged

CahidArda merged 2 commits into
mainfrom
fix/hermes-memory-append-turns

Conversation

@CahidArda

Copy link
Copy Markdown
Collaborator

Fixes point 1 of Sertug's DX-3182 feedback, ahead of the catalog PR.

Problem

sync_turn → save_chat read the whole chat document (JSON.GET) and wrote it back (JSON.SET) on every turn: all messages plus the extracted userMessages / modelMessages text, about 2x the transcript. One long-running session (Sertug's Bot Chat: ~0.9 MB of text after 3 weeks) sends ~1.7 MB per turn and would stop saving silently once it passes Upstash's 10 MB request limit.

Fix

  • ChatHistory.append_messages(session_id, messages, user_id=...): one atomic Lua script (#!lua flags=allow-key-locking, one key) that JSON.ARRAPPENDs the messages, JSON.STRAPPENDs the new text to both search fields (with the \n separator only when the field already has text), and sets updatedAt. Text is JSON-encoded in Python (dumps, like JSON.stringify), never by cjson. It returns 0 and writes nothing when the chat doesn't exist, so the provider falls back to a full save_chat for the first turn.
  • The provider appends each turn and only reads and rewrites the whole transcript for /undo. The in-memory transcript cache is gone (it only served /undo, and grew with the session).
  • The document ends up identical to a full save, so the layout stays shared with the TS ChatHistory. The TS saveChat still does a full replace; matching it there is a separate change.

Also fixed: /undo after a restart

RedisJSON returns a whole-number stamp like 1000.0 as 1000 (verified live). _cut_by_time only accepted float, so a transcript read back from Redis fell back to the content cut, and of two identical turns the wrong one could be removed. The in-memory cache used to hide this within one process. Any number now counts.

Bumps upstash-hermes-memory to 0.2.2. Merging publishes it to PyPI.

Tests

  • Live Redis: appending message by message ends exactly where a full save ends (unicode, quotes, /, a whitespace-only message, parts messages, a side's first text); createdAt/title kept; search finds the appended text; append to a missing chat writes nothing.
  • Provider: once the chat exists, a turn sends a single EVAL with only the new messages (no JSON.GET/JSON.SET), even with a 5 KB earlier turn.
  • /undo with stamps read back as ints.
  • Mutation-checked: going back to the full rewrite, or to the float-only check, fails 3 tests.
  • uv run pytest: 72 passed against live Redis. ruff, prettier, sync-version --check, uv sync --locked clean.

Not run: the Hermes e2e (e2e/hermes_e2e.py, needs a real Hermes and a model key).

…he transcript

sync_turn read the whole chat document and wrote it back every turn
(messages plus their extracted search text, about twice the transcript),
so a long-running session sent more on every turn and would stop saving
past Upstash's 10 MB request limit. ChatHistory.append_messages adds the
turn with one atomic Lua script (JSON.ARRAPPEND / JSON.STRAPPEND,
allow-key-locking) and leaves the document exactly as a full save would;
the provider now reads and rewrites the transcript only for /undo, and no
longer caches it in memory.

Also fixes the /undo timestamp cut for resumed sessions: RedisJSON returns
a whole-number stamp like 1000.0 as an int, which made the cut fall back
to matching by content.

Bumps upstash-hermes-memory to 0.2.2.
With the in-memory transcript cache gone, a write that failed (and was
swallowed) was lost for good: a failed first turn left the chat starting at
turn 2, and a failed write right after /undo cleared the undo flag, so the
undone turn stayed in Redis and the post-undo turn was lost. Pending changes
(turns to add, undo cuts) are now kept per session in order and cleared only
after a successful write; normal turns still append only what is new.
@CahidArda
CahidArda merged commit 1a18931 into main Nov 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant