Feature/correlation id tracing v2 - #2177
Open
Benedict315 wants to merge 6 commits into
Open
Benedict315 wants to merge 6 commits into
Benedict315 wants to merge 6 commits into
Conversation
- Add CorrelationIdInterceptor for HTTP edge correlation ID generation/extraction - Store correlation ID in AppLoggerService request context (AsyncLocalStorage) - Include correlation ID in domain events via EventsService - Stamp correlation ID on BullMQ job data via JobsService - Restore correlation ID in job processors (PayoutProcessor, WebhookProcessor) - Add correlation ID to outbound webhook headers - Wire CorrelationIdInterceptor globally in app.module.ts Backward compatible - existing functionality works without correlation IDs.
Contributor
|
Well done but Kindly fix workflow to pass. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Implements end-to-end correlation ID tracing across the backend system to enable correlation of log lines, traces, domain events, background jobs, and outbound webhooks with a shared identifier. This significantly improves incident debugging and latency attribution across async boundaries.
Problem
Previously, requests, emitted domain events, background jobs, and outbound webhooks couldn't be correlated. A single logical operation (e.g., a payout) produced log lines and traces with no shared identifier, making it very difficult to debug incidents and attribute latency across asynchronous boundaries.
Solution
Implementation Overview
HTTP Edge - CorrelationIdInterceptor
Generates or accepts X-Correlation-ID from request headers
Stores correlation ID in AppLoggerService request context (AsyncLocalStorage)
Adds correlation ID to response headers
Domain Events - EventsService
Automatically includes correlation ID from logger context in event emissions
Sets correlation ID on event payloads and metadata
Added optional correlationId field to BaseEvent
BullMQ Jobs - JobsService
Stamps correlation ID onto job data via __correlationId field
Restores correlation ID in job processors before execution
Updated PayoutProcessor and WebhookProcessor to restore context
Outbound Webhooks - WebhookProcessor
Adds correlation ID to outbound webhook headers (X-Correlation-ID)
Enables client-side tracing across service boundaries
Logging - AppLoggerService
Enhanced with correlation ID support in RequestContext
Automatic correlation ID inclusion in all log statements
Files Changed
New Files:
correlation-id.interceptor.ts - HTTP edge interceptor
correlation-id.interceptor.spec.ts - Unit tests
correlation-id.integration.spec.ts - Integration tests
CORRELATION_ID_TRACING.md - Comprehensive documentation
Modified Files:
base.event.ts - Added correlationId field
events.service.ts - Auto-includes correlation ID
jobs.service.ts - Stamps/restores correlation ID
payout.processor.ts - Restores context
webhook.processor.ts - Restores context + adds to headers
app.module.ts - Registered interceptor globally
Documentation:
Updated CHANGELOG files for: jobs, webhooks, tracing, logger, events modules
Added comprehensive implementation guide
Testing
Unit Tests: Tests for CorrelationIdInterceptor covering generation, extraction, and context storage
Integration Tests: End-to-end flow tests covering HTTP → Events → Jobs → Webhooks
Backward Compatibility: Tests verify existing functionality works without correlation IDs
Concurrent Requests: Tests verify unique correlation IDs for parallel requests
Backward Compatibility
✅ Fully backward compatible
Existing HTTP requests without X-Correlation-ID work as before (new ID generated)
Existing events without correlation ID field work as before (field is optional)
Existing jobs without __correlationId field work as before (field is optional)
Existing webhook consumers without correlation ID handling work as before (header is optional)
Performance Impact
Minimal performance impact:
UUID generation: < 1ms per request
AsyncLocalStorage operations: < 0.1ms per context switch
Header addition: negligible
Log metadata enhancement: negligible
Usage Example
Client-side:
bash
Generate new correlation ID
curl https://api.example.com/health
Response: X-Correlation-ID:
Use existing correlation ID
curl https://api.example.com/health -H "X-Correlation-ID: my-id"
Server-side:
typescript
// Access correlation ID
const context = AppLoggerService.getRequestContext();
const correlationId = context?.correlationId;
Checklist
Implementation complete
Unit tests added
Integration tests added
CHANGELOG files updated
Documentation added
Backward compatibility verified
Code follows project conventions
Ready for review
closes #2156