Skip to content

feat(otel-metrics): POC — centralize OTLP metrics export via a shared libdatadog exporter - #282

Draft
mabdinur wants to merge 2 commits into
mainfrom
munir/otel-metrics-libdatadog-poc
Draft

feat(otel-metrics): POC — centralize OTLP metrics export via a shared libdatadog exporter#282
mabdinur wants to merge 2 commits into
mainfrom
munir/otel-metrics-libdatadog-poc

Conversation

@mabdinur

@mabdinur mabdinur commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

What

Proof of concept (draft): move the OTLP metrics export logic into libdatadog's libdd-otel-telemetry crate — which now exposes a Datadog OTLP PushMetricExporter (DatadogMetricExporter, owning encode + transport + retry + attempt/success/failure telemetry counting). dd-trace-rs keeps its own SdkMeterProvider + PeriodicReader + resource + views and just plugs the exporter in.

Companion POC PRs: libdatadog DataDog/libdatadog#2292 (the crate + shared exporter), dd-trace-py DataDog/dd-trace-py#19354 (Python metrics, primitives aggregator).

Why this shape (not the primitives aggregator)

The primitives-only aggregator exists for FFI — Python/Node/Ruby/PHP can't pass SDK objects across the language boundary. dd-trace-rs is Rust-to-Rust in-process and keeps opentelemetry_sdk regardless (its trace pipeline needs it), so a primitives bridge there is pure overhead — an earlier revision hand-reimplemented ~500 lines of the SDK's own metrics API for no benefit. This approach centralizes only the export path (the part actually worth sharing) and is a net reduction in dd-trace-rs.

Two doors over one shared export core: FFI languages use the primitives aggregator; Rust plugs in the shared PushMetricExporter. The cost is that the crate exposes an SDK type on its Rust door and tracks the consumer's opentelemetry version (bumped 0.31→0.32) — a deliberate, Rust-only exception to "no SDK objects cross the boundary," invisible to the FFI consumers.

Changes

  • metrics_reader.rs: builds the exporter via libdd_otel_telemetry::build_datadog_metric_exporter(config, temporality) and feeds it to a PeriodicReader; returns SdkMeterProvider as before. Views keep working.
  • Deletes telemetry_metrics_exporter.rs — the attempt/success/failure counting now lives in the shared exporter; DatadogMetricExporter::counters() exposes a snapshot (wiring it into DD telemetry is a TODO).
  • metrics features no longer pull opentelemetry-otlp (owned by libdatadog); opentelemetry_sdk stays for traces + the provider/reader.

Regression tests (existing suite, run as-is)

  • integration_tests::metrics: 12 passed, 0 failed, 0 ignored (all view tests included).
  • lib metrics_tests: 2 passed, 0 failed, 0 ignored.
  • clippy -D warnings + fmt clean.

Net source delta vs main: +64 / −124.

POC caveats

  • Export-telemetry counting is available via DatadogMetricExporter::counters() but not yet wired into DD's telemetry client (TODO).
  • Temporary dependency pinning: libdd-otel-telemetry is unpublished, so it's a git dep + a minimal [patch.crates-io] for the 4 libdatadog crates it shares with dd-trace-rs. Revert once published.
  • Metrics only — logs/traces untouched.

🤖 Generated with Claude Code

… libdatadog exporter

Proof of concept: move the OTLP metrics export logic (encode + transport +
retry + attempt/success/failure telemetry counting) into libdatadog's
libdd-otel-telemetry crate, which now exposes a Datadog OTLP PushMetricExporter
(DatadogMetricExporter). dd-trace-rs keeps its own SdkMeterProvider +
PeriodicReader + resource + views and just plugs that exporter in.

Rationale: dd-trace-rs keeps opentelemetry_sdk regardless (its trace pipeline
needs it) and shares metrics objects in-process, so the primitives-only
aggregator bridge used for the FFI languages (Python/Node/Ruby/PHP) is pure
overhead here — it reimplemented the SDK's own metrics API. This shared-exporter
approach centralizes only what's worth centralizing (the export path) and is a
net reduction in dd-trace-rs.

- metrics_reader.rs builds the exporter via
  libdd_otel_telemetry::build_datadog_metric_exporter and feeds it to a
  PeriodicReader; returns SdkMeterProvider as before. Views keep working.
- Deletes telemetry_metrics_exporter.rs (counting now lives in the shared
  exporter; DatadogMetricExporter::counters() exposes a snapshot — wiring it
  into DD telemetry is a TODO).
- metrics features no longer pull opentelemetry-otlp (owned by libdatadog);
  opentelemetry_sdk stays for traces + the provider/reader.

Net source delta vs main: +64 / -124. All metrics regression tests pass with
zero ignored (views restored): integration 12 passed, lib metrics_tests 2
passed.

Temporary git/patch dependency pinning on the (unpublished) crate; revert once
libdd-otel-telemetry is published.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mabdinur
mabdinur force-pushed the munir/otel-metrics-libdatadog-poc branch from 3bdd72c to c524744 Compare July 30, 2026 19:11
@mabdinur mabdinur changed the title feat(otel-metrics): POC — centralize OTLP metrics export via libdd-otel-telemetry feat(otel-metrics): POC — centralize OTLP metrics export via a shared libdatadog exporter Jul 30, 2026
… licenses

The instrumentation workspace (datadog-aws-lambda) path-depends on
datadog-opentelemetry, which now pulls libdd-otel-telemetry from git; [patch]
is per-workspace, so mirror the root's crates.io patch there and refresh its
lockfile. Regenerate LICENSE-3rdparty.csv for the new opentelemetry 0.32 +
libdatadog dependencies.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Jul 30, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 34a7903 | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant