Agent mutations 409 PROJECTION_STALE; stale-epoch browser sessions silently drop edits

[vishalsachdev]

Observed (live deployment, 2026-07-26, doc slug rclchxs4)

Three related failures around the access-epoch / live-authority machinery, observed while an AI agent posted marks via the stateless HTTP mutation path (MCP bridge) with a human editor session open in the browser:

1. Parallel agent mutations trip the write gate

Three concurrent suggest mutations raced the epoch bookkeeping. After the burst, every subsequent mutation returned 409 PROJECTION_STALE even though the projection row was verifiably healthy (document_projections.health = 'healthy', revision matching documents). Sequential writes eventually succeed, but only ~one per background repair cycle (60-90s) — a throughput cliff for any agent workflow that posts several marks.

2. Misleading error code

The 409 is produced by projectionStaleMutationResult() (server/document-engine.ts), but the actual denial comes from the live-authority gate in getCanonicalReadableDocumentSync (server/collab.ts): active collab presence + getLocalLiveAuthorityDecision(...).allowed === false → mutation_ready: false. Debugging chased the projection (which was fine) because the code said PROJECTION_STALE. The live-authority denial deserves its own error code (e.g. LIVE_AUTHORITY_UNAVAILABLE) and log line.

3. Stale-epoch browser sessions silently drop human edits (data loss)

After the epoch bump (access_epoch 0 → 3), the open browser session kept editing with no UI indication, while the server logged for every change:

[collab] stale-epoch write dropped { slug: 'rclchxs4', source: 'onChange', sessionAccessEpoch: 0, currentAccessEpoch: 3 }

From the user's perspective the editor was working; in reality nothing persisted until a manual hard refresh re-attached the session at the current epoch. This is the highest-severity piece: silent human data loss.

Suggested fixes (in priority order)

1. On stale-epoch write drop, close that collab connection so the client auto-reconnects at the current epoch (or push a force-resync). One dropped write is a race; a session that keeps dropping writes should never linger silently.
2. Distinct error code + client banner for live-authority denial vs genuine projection staleness.
3. Serialize or lease agent mutations per doc (or make the epoch bump not invalidate live-authority for in-flight sessions) so a burst of agent marks doesn't wedge the write path for minutes.

Environment

Fork vps branch at 759aabf (flag-rendering fix), deployed via proof-vps Docker; single replica; SQLite store.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ro6KiE6zCSSzsqeyVoAu8M

[vishalsachdev]

Status after vps @ 18f77a0 (deployed to proof.illinihunt.org 2026-07-26):

Finding 3 (silent data loss) — fixed in e56c3a6. Root cause: shouldDropStaleContextWrite compares the per-connection context.accessEpoch against the DB, but nothing evicted a connection whose context predated the bump — evictStaleLocalStateForAccessEpoch keys on loadedDocDbMeta.accessEpoch, document-level state that self-heals the first time it fires, so the stale session kept writing into a void indefinitely. Dropping a stale write now also closes the slug's connections (debounced per slug+epoch); the client reconnects, authenticateCollabSession throws session-stale, and that surfaces as authenticationFailed → visible disconnected state. Regression test collab-stale-epoch-session-eviction verified to fail with the eviction removed.

Finding 2 (misleading error code) — fixed in e56c3a6. Live-authority denials now tag the read surface with mutation_block_reason: 'live_authority', and the engine returns 409 LIVE_AUTHORITY_UNAVAILABLE instead of PROJECTION_STALE for that case.

Finding 1 (parallel mutations wedge the write path) — still open. Needs a per-doc serialization/lease design rather than a local patch; leaving this issue open to track it.

Related: cherry-picked upstream EveryInc#68 (11dd41d) in the same deploy — /documents/:slug/ops was replaying stale DB snapshots over concurrent live edits, the server-side sibling of finding 3's client-side loss.