feat(datadog-aws): add AWS SDK tracing for SQS, SNS, and EventBridge by Dogbu-cyber · Pull Request #189 · DataDog/dd-trace-rs

Dogbu-cyber · 2026-03-17T20:39:06Z

What does this PR do?

Adds Datadog trace context propagation for the AWS SDK for Rust, covering SQS, SNS, and EventBridge through separate service crates. Each service crate depends only on its matching AWS SDK crate, with shared span and attribute logic in datadog-aws-core.

Ref: #190
Jira: IDMPL-570

Crate architecture

instrumentation/
├── datadog-aws-core/             # shared request span lifecycle, propagation headers, attribute keys
├── datadog-aws-core-test-utils/  # shared integration/unit test helpers, not published
├── datadog-aws-sqs/              # SQS SDK interceptor and SQS context extraction
├── datadog-aws-sns/              # SNS SDK interceptor
├── datadog-aws-eventbridge/      # EventBridge SDK interceptor
└── datadog-aws-lambda/           # existing Lambda crate, updated for publishing/dependency cleanup

Usage

use datadog_aws_sns::ConfigExt as _;

let config = aws_sdk_sns::config::Builder::from(&sdk_config)
    .datadog_tracing() // install Datadog tracing interceptor
    .build();

let client = aws_sdk_sns::Client::from_conf(config);

Supported behavior

Service	Context injection	Service-specific span tags	Injection point
SQS	`SendMessage`, `SendMessageBatch`	queue tags for send, receive, and delete operations; batch/message counts where available; send message ID; receive span links	`_datadog` message attribute as String JSON
SNS	`Publish`, `PublishBatch`	topic/target tags for publish, batch, topic, subscription, and permission operations; batch counts where available	`_datadog` message attribute as Binary JSON
EventBridge	`PutEvents`	rule tags for rule/target operations; batch counts for `PutEvents`	top-level `_datadog` field in each event `detail` JSON object

Each instrumented AWS call creates a client span named sqs.request, sns.request, or eventbridge.request, with common Datadog/OpenTelemetry attributes including aws.service, aws.operation, aws.region, aws.partition, resource.name, operation.name, span.kind, HTTP method/URL/status, AWS user agent, and AWS request ID.

Notable details

SQS receive requests automatically request the _datadog message attribute, expose datadog_aws_sqs::extract_context, and add span links for received messages with valid propagated context.
SQS and SNS respect the AWS 10-message-attribute limit, but replace an existing stale _datadog attribute when present.
SNS uses Binary message attributes for _datadog to avoid SNS subscription filter policy interference.
EventBridge injects only into existing top-level JSON object details, replaces an existing top-level _datadog, preserves nested _datadog fields, skips missing/invalid/non-object details, and skips details larger than 1 MiB.
Injection failures never fail the AWS SDK call; they are skipped and logged at debug level.
Integration tests cover successful injection, span tags, error status handling, SQS receive links, batch message counts, and EventBridge detail rewrite edge cases.

Trace Examples

Lambda: SQS.SendMessage and EventBridge.PutEvents

SQS.SendMessageBatch

SQS.ReceiveMessage

ygree · 2026-03-18T02:09:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 302fbb5267

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

joeyzhao2018

LGTM

Dogbu-cyber · 2026-03-30T19:21:53Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80424aa3a6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

attributes

Read `_datadog` from both string and binary SQS message attributes so SNS-propagated context can be extracted.

local path deps

Copilot

Pull request overview

Adds Datadog trace context injection for the AWS SDK for Rust by introducing three service-specific interceptor crates (SQS, SNS, EventBridge) built on a shared core, plus shared test infrastructure. This fits into the instrumentation/ workspace to let users depend only on the AWS SDK crate(s) they actually use.

Changes:

Introduce datadog-aws-core request-span helpers + shared attribute keys used by all service interceptors.
Add datadog-aws-sqs, datadog-aws-sns, and datadog-aws-eventbridge crates implementing AWS SDK interceptors and context injection at service-appropriate locations.
Add datadog-aws-core-test-utils and integration tests for each service crate using a mock AWS endpoint + in-memory OTel exporter.

Reviewed changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
instrumentation/Cargo.toml	Expands the instrumentation workspace members and shared dependencies.
instrumentation/Cargo.lock	Updates lockfile for newly added AWS SDK + test dependencies.
instrumentation/datadog-aws-core/Cargo.toml	Defines the shared core crate package + dependencies.
instrumentation/datadog-aws-core/src/lib.rs	Exposes core request span utilities and attribute keys.
instrumentation/datadog-aws-core/src/attribute_keys.rs	Adds shared span attribute keys and propagation payload keys.
instrumentation/datadog-aws-core/src/request_span.rs	Implements common span lifecycle helpers used by all interceptors.
instrumentation/datadog-aws-core-test-utils/Cargo.toml	Adds a non-published test-utils crate for integration/unit test helpers.
instrumentation/datadog-aws-core-test-utils/src/lib.rs	Exposes integration and unit test helper modules.
instrumentation/datadog-aws-core-test-utils/src/integration_test_helpers.rs	Provides mock AWS server + tracer setup used by service integration tests.
instrumentation/datadog-aws-core-test-utils/src/test_helpers.rs	Provides a lightweight test propagator + context helpers for unit tests.
instrumentation/datadog-aws-eventbridge/Cargo.toml	Defines the EventBridge interceptor crate and deps.
instrumentation/datadog-aws-eventbridge/src/lib.rs	Implements EventBridge interceptor + `_datadog` injection into `detail` JSON.
instrumentation/datadog-aws-eventbridge/tests/integration.rs	Adds integration coverage for EventBridge span tags and `detail` injection.
instrumentation/datadog-aws-sns/Cargo.toml	Defines the SNS interceptor crate and deps.
instrumentation/datadog-aws-sns/src/lib.rs	Implements SNS interceptor + binary MessageAttribute injection.
instrumentation/datadog-aws-sns/tests/integration.rs	Adds integration coverage for SNS span tags and binary attribute injection.
instrumentation/datadog-aws-sqs/Cargo.toml	Defines the SQS interceptor crate and deps.
instrumentation/datadog-aws-sqs/src/lib.rs	Implements SQS interceptor + `_datadog` MessageAttribute injection and context extraction/linking on receive.
instrumentation/datadog-aws-sqs/tests/integration.rs	Adds integration coverage for SQS injection, receive linking, queue parsing, and error tagging.
instrumentation/datadog-aws-lambda/Cargo.toml	Adjusts Lambda crate versioning and switches `datadog-opentelemetry` dependency to crates.io.
instrumentation/datadog-aws-lambda/src/invocation.rs	Minor span attribute setting refactor (uses `set_attributes`).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Set the count on SQS SendMessageBatch, SQS ReceiveMessage, EventBridge PutEvents, and SNS PublishBatch request spans.

ygree · 2026-08-03T16:40:59Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: de91cb886b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-03T16:48:34Z

+
+[dependencies]
+datadog-aws-core = { workspace = true }
+aws-sdk-eventbridge = "1.104.0"


Regenerate the third-party license inventory

This commit introduces the AWS SDK, Smithy, and test dependency graphs, but LICENSE-3rdparty.csv is unchanged and contains none of the new AWS or wiremock packages. CONTRIBUTING.md explicitly requires regenerating and committing this inventory whenever dependencies change, so the new publishable crates currently ship without the repository's required third-party attribution; update the inventory, and if the existing generator ignores the nested instrumentation workspace, extend its invocation accordingly.

AGENTS.md reference: AGENTS.md:L3-L4

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-03T16:48:35Z

+    if attrs.len() < MAX_MESSAGE_ATTRIBUTES || attrs.contains_key(DATADOG_ATTRIBUTE_KEY) {
+        attrs.insert(DATADOG_ATTRIBUTE_KEY.to_string(), datadog_attr);


Guard SNS injection against the total message-size limit

When a publish payload is already near SNS's 256 KiB message-size limit, inserting this binary attribute—or replacing a smaller existing _datadog value—can push the message over the service limit and turn a previously valid Publish or PublishBatch request into a rejection. The attribute-count guard does not account for serialized attribute size, so injection should be skipped when its size delta would exceed the SNS limit.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-03T16:48:35Z

+    if attrs.len() < MAX_MESSAGE_ATTRIBUTES || attrs.contains_key(DATADOG_ATTRIBUTE_KEY) {
+        attrs.insert(DATADOG_ATTRIBUTE_KEY.to_string(), datadog_attr);


Guard SQS injection against the total message-size limit

For a message whose body and existing attributes are close to SQS's 1 MiB limit, adding this string attribute—or replacing a smaller _datadog value—can make an otherwise valid SendMessage or batch entry exceed the service quota and fail. Checking only the number of attributes cannot prevent this production-visible behavior; skip injection when the serialized size delta would cross the SQS message limit.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-03T16:48:35Z

+    let attrs = message.message_attributes.as_ref()?;
+    let datadog_attr = attrs.get(DATADOG_ATTRIBUTE_KEY)?;


Extract trace context from the default SNS envelope

For an SNS-to-SQS subscription using the default RawMessageDelivery = false, SNS places the binary _datadog attribute inside the JSON notification envelope in Message::body, not in the SQS message-attribute map. Returning immediately when message.message_attributes is absent therefore makes extract_context return None, and the receive interceptor creates no link for the default SNS-to-SQS configuration even though the SNS interceptor injected context; parse the SNS envelope as a fallback and decode its binary attribute.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-03T16:48:35Z

+        } else if let Some(input) = input.downcast_ref::<SubscribeInput>() {
+            topic_arn = input.topic_arn.as_deref();
+        }


Tag all SNS requests that carry a topic ARN

Operations such as AddPermission and DeleteTopic also carry a topic_arn, but their input types are absent from this dispatch chain, so their request spans omit topicname while otherwise equivalent operations such as RemovePermission and GetTopicAttributes include it. This produces inconsistent resource attribution for common SNS operations; handle every ARN-bearing input type rather than limiting extraction to this subset.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-03T16:48:35Z

+        } else if let Some(input) = input.downcast_ref::<DeleteMessageBatchInput>() {
+            queue_url = input.queue_url.as_deref();
+            batch_message_count = Some(input.entries.as_ref().map_or(0, Vec::len) as i64);


Tag all SQS requests that carry a queue URL

The dispatch only extracts queue_url for send, receive, and delete-message inputs. Queue-scoped operations such as PurgeQueue, DeleteQueue, GetQueueAttributes, and ChangeMessageVisibility therefore create spans without queuename or cloud.resource_id even though their requests contain the same queue URL, making resource attribution depend incorrectly on the operation; extend extraction to the remaining queue-scoped input types.

Useful? React with 👍 / 👎.

Dogbu-cyber added the enhancement New feature or request label Mar 17, 2026

Dogbu-cyber mentioned this pull request Mar 17, 2026

feat(datadog-aws-lambda): add trigger extraction and inferred spans #190

Merged

Dogbu-cyber marked this pull request as ready for review March 17, 2026 22:01

Dogbu-cyber requested a review from a team as a code owner March 17, 2026 22:01

joeyzhao2018 reviewed Mar 17, 2026

View reviewed changes

Comment thread integrations/aws/aws-sdk-rust/README.md Outdated

chatgpt-codex-connector Bot reviewed Mar 18, 2026

View reviewed changes

Comment thread integrations/aws/aws-sdk-rust/Cargo.toml Outdated

Comment thread integrations/aws/aws-sdk-rust/src/services/sns.rs Outdated

Comment thread integrations/aws/aws-sdk-rust/src/services/sqs.rs Outdated

Dogbu-cyber changed the base branch from main to david.ogbureke/ci-integration-msrv March 22, 2026 00:54

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from ee01b82 to 612838b Compare March 22, 2026 00:55

Dogbu-cyber force-pushed the david.ogbureke/ci-integration-msrv branch 2 times, most recently from 141c5dd to 1184437 Compare March 22, 2026 20:52

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 612838b to 4f7442a Compare March 23, 2026 14:54

Dogbu-cyber mentioned this pull request Mar 23, 2026

chore(instrumentation): set up instrumentation workspace #194

Merged

Dogbu-cyber changed the title ~~AWS SDK Integration~~ feat(datadog-aws-sdk): add trace context injection for SQS, SNS, and EventBridge Mar 23, 2026

joeyzhao2018 approved these changes Mar 27, 2026

View reviewed changes

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 3fa3937 to 80424aa Compare March 30, 2026 19:20

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 80424aa to 80fedd0 Compare March 30, 2026 19:26

chatgpt-codex-connector Bot reviewed Mar 30, 2026

View reviewed changes

Comment thread instrumentation/datadog-aws-eventbridge/src/lib.rs Outdated

Dogbu-cyber force-pushed the david.ogbureke/ci-integration-msrv branch 2 times, most recently from 17f2c7a to 78a6876 Compare March 30, 2026 20:30

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 80fedd0 to 18c7914 Compare March 30, 2026 20:48

Dogbu-cyber force-pushed the david.ogbureke/ci-integration-msrv branch from 78a6876 to bf4ea81 Compare April 2, 2026 14:54

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch 2 times, most recently from 35b2e4f to 7d1fbbe Compare April 2, 2026 15:32

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 29211f0 to d999596 Compare April 13, 2026 12:51

ygree reviewed Apr 13, 2026

View reviewed changes

Comment thread instrumentation/aws/datadog-aws/tests/integration.rs Outdated

ygree reviewed Apr 14, 2026

View reviewed changes

Comment thread instrumentation/aws/datadog-aws/src/interceptor.rs Outdated

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch 2 times, most recently from 7e64f6f to f1e120e Compare April 14, 2026 18:06

ygree added 3 commits June 10, 2026 17:36

feat(sqs): link receive spans to message trace context with message id

9967164

attributes

feat(sqs): add receive completion span event

2516f93

fix(sqs): tests rely on the global propagator

10c2daa

ygree force-pushed the david.ogbureke/aws-sdk-rust branch from b9f8ecf to f559889 Compare June 13, 2026 00:39

feat(sqs): extract SNS-to-SQS trace context

ae2e2b7

Read `_datadog` from both string and binary SQS message attributes so SNS-propagated context can be extracted.

ygree force-pushed the david.ogbureke/aws-sdk-rust branch from 86a98be to ae2e2b7 Compare June 13, 2026 01:07

ygree added 6 commits June 15, 2026 09:11

Merge branch 'main' into david.ogbureke/aws-sdk-rust

cfb42d7

Merge branch 'main' into david.ogbureke/aws-sdk-rust

4648860

Update Cargo.lock

14c6d0b

Merge branch 'main' into david.ogbureke/aws-sdk-rust

b36c943

Update Cargo.lock

91ba9e8

chore(instrumentation): reduce instrumentation/Cargo.lock churn from

af0a474

local path deps

ygree force-pushed the david.ogbureke/aws-sdk-rust branch from f548ec1 to af0a474 Compare July 20, 2026 20:45

ygree requested a review from Copilot July 20, 2026 20:46

Copilot started reviewing on behalf of ygree July 20, 2026 20:47 View session

Copilot AI reviewed Jul 20, 2026

View reviewed changes

Comment thread instrumentation/datadog-aws-core/src/request_span.rs

Comment thread instrumentation/datadog-aws-eventbridge/src/lib.rs

Comment thread instrumentation/datadog-aws-eventbridge/src/lib.rs Outdated

ygree added 2 commits July 20, 2026 17:06

Use non-panicking utf-8 conversion

838d41c

refactor(instrumentation): import AWS core helpers directly

46ee81d

ygree self-assigned this Jul 28, 2026

fix(aws): set messaging batch message counts on request spans

ebbe4c3

Set the count on SQS SendMessageBatch, SQS ReceiveMessage, EventBridge PutEvents, and SNS PublishBatch request spans.

ygree force-pushed the david.ogbureke/aws-sdk-rust branch from 922151b to ebbe4c3 Compare July 29, 2026 18:11

ygree added 4 commits July 29, 2026 11:56

fix(eventbridge): skip injection when detail is missing

3085b1b

docs(eventbridge): clarify PutEvents detail size guard

d49e0a1

fix(sqs): tag delete batch message count

b41142c

Merge branch 'main' into david.ogbureke/aws-sdk-rust

0648b56

ygree changed the title ~~feat(datadog-aws): add per-service interceptor crates for SQS, SNS, and EventBridge~~ feat(datadog-aws): add AWS SDK tracing for SQS, SNS, and EventBridge Jul 29, 2026

ygree added 2 commits July 31, 2026 11:38

remove tracing dependency from datadog-aws-core

4de149c

refactor(aws): replace otel_debug with tracing debug

de91cb8

chatgpt-codex-connector Bot reviewed Aug 3, 2026

View reviewed changes

		if attrs.len() < MAX_MESSAGE_ATTRIBUTES \|\| attrs.contains_key(DATADOG_ATTRIBUTE_KEY) {
		attrs.insert(DATADOG_ATTRIBUTE_KEY.to_string(), datadog_attr);

		let attrs = message.message_attributes.as_ref()?;
		let datadog_attr = attrs.get(DATADOG_ATTRIBUTE_KEY)?;

Uh oh!

Conversation

Dogbu-cyber commented Mar 17, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Crate architecture

Usage

Supported behavior

Notable details

Trace Examples

Lambda: SQS.SendMessage and EventBridge.PutEvents

SQS.SendMessageBatch

SQS.ReceiveMessage

Uh oh!

Uh oh!

ygree commented Mar 18, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

joeyzhao2018 left a comment

Choose a reason for hiding this comment

Uh oh!

Dogbu-cyber commented Mar 30, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ygree commented Aug 3, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Dogbu-cyber commented Mar 17, 2026 •

edited by atlassian Bot

Loading