A failed appendRows may poison the seenMessages dedupe so the exporter's retry silently writes nothing

[philcunliffe]

Surfaced by an automated review sweep over the claude telemetry plugin. Not yet verified by the reconcile loop: treat the claim below as a lead with file:line evidence, and reproduce it with a failing-then-passing test before fixing. If it does not reproduce, say so and close.

The claim

hypaware-core/plugins-workspace/claude/src/telemetry/source.js:688 rethrows on a write failure so the exporter retries. The retry is claimed not to recover the batch:

• deps.gateway.recordProjectedExchange reaches exchange_writer.js record(), which calls aiGatewayRowsFromProjectedExchange.
• That adds every message's identity.messageId to the writer's process-lifetime state.seenMessages while building rows (message_projector.js:773), i.e. before storage.appendRows runs.
• If appendRows throws (disk full, transient Iceberg/spool write error), source.js rethrows, the listener answers HTTP 500, and Claude Code re-POSTs the same batch.
• The second record() now hits state.seenMessages.has(...) for every message, continues, and returns rows.length === 0.

Claimed consequence: the batch's user prompts and assistant responses are permanently lost (only transcript backfill can recover them), while rowsWritten reports 0 as though there had been nothing to write - a silent loss, not an error.

Acceptance

A regression test that fails on current master: force appendRows to throw once, re-deliver the identical batch, and assert the rows land on the retry. Today they should be dropped. If the dedupe set is only populated after a successful append, the claim is wrong and this issue should be closed with that evidence.

[philcunliffe]

Corroborated independently, and reproduced. A second review (the reconcile loop's review of PR #851, at head 40e40f9d) reached this defect from a different direction and reports reproducing it:

aiGatewayRowsFromProjectedExchange commits state.seenMessages.add(...) (message_projector.js:772) before exchange_writer.record() calls appendRows, with no rollback, and the writer is memoized per process. I reproduced it: the retry gets rows.length === 0 and stores nothing, so the whole failed batch is lost permanently.

Two things this adds to the original report:

1. It is pre-existing on master, not introduced by any open PR, and it lives in the ai-gateway dataset-owner contract (LLP 0252). The same shared projector state is used by the proxy path, so the blast radius is wider than the OTEL listener.
2. The fix is not a small one. It means making row expansion and append atomic across projector state shared by two capture paths. That is design work, not a one-line reorder, and it was explicitly judged out of scope for a review-rung edit on Deferred #843 findings: listener counters, the refusal log, and the usage a failed batch consumed #851.

So the remaining verification burden is lower than when this was filed: the mechanism is confirmed. What a fix still owes is the failing-then-passing regression test and a chosen atomicity mechanism. Read LLP 0252 before picking one.