Feat/1226 transactional event projection - #1247
Merged
mikewheeleer merged 3 commits intoAug 30, 2026
Merged
mikewheeleer merged 3 commits into
mikewheeleer merged 3 commits into
Conversation
…dule Upstream main (d7a66f9) did not compile: src/dlqStore.ts had a double-colon typo (db::) and src/errors/appError.ts had an invalid ??-optional chain (response?.headers??.get?). The request-context module referenced by eventIngestionService.ts and auth/middleware.ts (getContext, requestContextStorage from '../context') was missing entirely, so builds and tests could not run. Restore a building baseline. - src/dlqStore.ts: change db:: to db - src/errors/appError.ts: change ??.get?. to ?.get? - src/context.ts: add requestContextStorage (AsyncLocalStorage) + getContext, mirroring src/middleware/requestContext.ts
Closes Talenttrust#1226 Persist the event checkpoint and its projection in one database transaction so a partially-applied event can never leave a checkpoint without its read-model (or vice-versa), which previously created skipped/duplicated state across retries. - migration v16: add event_audit (PK = deduplication_key, event identity) and event_projection (PK = entity_id, entity identity; last_event_id for replay idempotency) tables. - SqliteEventAuditRepository: SQLite-backed IEventAuditRepository wired as the production default in events/registry.ts. Adds the explicit transactional persistEventAndProjection() method (save() stays a plain record write). - Retry only serialization failures (SQLITE_BUSY=5, SQLITE_BUSY_SNAPSHOT=517), bounded attempts/delay; constraint and data errors never retried, so a real bug is not masked by a slow retry loop. - External calls (finality/chain evaluation) run before the commit, so a network timeout cannot corrupt the transaction. - Tenant isolation enforced on write and read (audit + projection rows are scoped by the same tenant). - EventAuditService accepts an optional pure projectionBuilder; registry wires a contractId-keyed projection. In-memory repository stays valid for unit tests (persistEventAndProjection is optional on the interface). - Tests cover every edge case: successful atomic commit, projection-constraint rollback, SQLITE_BUSY / SQLITE_BUSY_SNAPSHOT retry, retry exhaustion, no retry on non-serialization errors, external-call timeout (no partial write), duplicate replay (no double projection), and tenant isolation. Also fixes pre-existing upstream syntax errors blocking the build/tests that this work depends on: safeErrors.ts invalid regex /r flag, and appError.ts AppExrror / mapZodErropToDetails typos.
|
@shogun444 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
…al-event-projection
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1226
Why This PR Exists
Persisting event checkpoints separately from their read-model projections creates state drift, resulting in skipped or duplicated state during retries or process crashes. Previously,
EventAuditService.processEventpersisted checkpoints in an in-memory map without a shared transaction boundary across the event audit log and projection state.This PR places event checkpoints and projection writes inside a single atomic database transaction with explicit serialization retry semantics and bounded execution boundaries.
What Changed
src/db/migrations.ts):v16creating:event_audit: Primary keydeduplication_key(event identity) for duplicate replay prevention.event_projection: Primary keyentity_id(entity identity) trackinglast_event_idfor deterministic projection ordering.src/repository/sqliteEventAuditRepository.ts):SqliteEventAuditRepositoryusingbetter-sqlite3transactions to atomically execute event checkpoint and projection updates.withSerializationRetry()to handle concurrency lock contention—strictly retrying onlySQLITE_BUSY(5) andSQLITE_BUSY_SNAPSHOT(517) errors, while letting data/constraint errors fail fast.src/repository/eventAuditRepository.ts,src/events/registry.ts):IEventAuditRepositorywithpersistEventAndProjection()while maintaining backwards compatibility withInMemoryEventAuditRepository.projectionBuildersupport toEventAuditService.SqliteEventAuditRepositoryand a tenant-isolatedcontractProjectionBuilderas production defaults.src/errors/safeErrors.ts(invalid/rregex flag) andsrc/errors/appError.ts(AppExrrorandmapZodErropToDetailstypos).Edge Case Coverage & Architecture
SQLITE_BUSY/SQLITE_BUSY_SNAPSHOT.deduplication_keyonevent_auditandlast_event_idversion checks onevent_projection.Type of Change
v16)Test Evidence
Targeted Test Suite Verification
npm run build— Passednpm run lint— Passed (0 errors)tsc --noEmiton changed modules — Passed (0 errors)