feat(heap-profiling): expose libdatadog heap profiling primitive to users [PROF-15190]

[scottgerring]

What does this PR do?

The profiling team is adding heap profiling support to libdatadog and the eBPF full host profiler. This works by exposing a USDT behind a sampled allocation path, so that the profiler can hook profiled applications to obtain statistically significant heap samples without having to hook every allocation. In the Rust case, this is supported by providing a GlobalAllocator wrapper.

Speaking with @r1viollet , it would be nice to get this in behind an experimental flag even before the upstream profiling work is done. Let us know what you think!

[datadog-prod-us1-6]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: d16d7df | Docs | Datadog PR Page | Give us feedback!}

[Copilot]

Pull request overview

Adds an opt-in “memory profiling” capability to datadog-opentelemetry by exposing libdatadog’s sampled heap-allocation allocator wrapper behind a feature flag, enabling out-of-process profilers to collect allocation samples via USDT probes.

Changes:

• Introduces a new memory_profiling module that re-exports libdd_heap_allocator::SampledAllocator and provides a minimal builder-style opt-in API.
• Wires the module behind a new memory-profiling crate feature and documents the feature in crate-level docs.
• Adds a temporary git dependency on an in-progress libdatadog branch and updates the lockfile accordingly.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
datadog-opentelemetry/src/memory_profiling.rs	New public module documenting allocator-based opt-in and providing the builder API.
datadog-opentelemetry/src/lib.rs	Documents and conditionally exposes the memory_profiling module + entrypoint under a feature flag.
datadog-opentelemetry/Cargo.toml	Adds optional libdd-heap-allocator dependency and memory-profiling feature.
Cargo.toml	Adds workspace dependency pointing to a libdatadog git branch for the new allocator crate.
Cargo.lock	Locks the newly introduced git-based libdatadog crates and their transitive deps (e.g., bindgen).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (4)

datadog-opentelemetry/src/lib.rs:258

• Same issue as the module declaration above: this re-export is available when either memory-profiling or memory-profiling-alloc-only is enabled, but rustdoc currently advertises only memory-profiling.

#[cfg(feature = "_memory-profiling")]
#[cfg_attr(docsrs, doc(cfg(feature = "memory-profiling")))]
pub use memory_profiling::memory_profiling;

datadog-opentelemetry/src/lib.rs:255

• memory_profiling items are gated by the internal _memory-profiling feature, which is enabled by both public features (memory-profiling and memory-profiling-alloc-only). The current doc(cfg(feature = "memory-profiling")) is misleading for users who enable memory-profiling-alloc-only, since rustdoc will claim the API requires memory-profiling.

This issue also appears on line 256 of the same file.

#[cfg(feature = "_memory-profiling")]
#[cfg_attr(docsrs, doc(cfg(feature = "memory-profiling")))]
pub mod memory_profiling;

datadog-opentelemetry/src/memory_profiling.rs:44

• This PR introduces a new public API surface (memory_profiling module + SampledAllocator re-export) but does not add any tests exercising/compiling it. CONTRIBUTING.md requires at least one test for functional changes; even a small compile-level unit test would help prevent accidental breakage.

/// Starts building a memory profiling configuration.
pub fn memory_profiling() -> DatadogMemoryProfilerBuilder {
    DatadogMemoryProfilerBuilder { _private: () }
}

datadog-opentelemetry/Cargo.toml:115

• This feature comment references a sgg/heap-prof-poc branch and a "workspace Cargo.toml comment", but the workspace dependency is pinned to a crates.io version and there is no corresponding workspace comment. This looks stale and may confuse maintainers/users.

# Emits sampled allocation USDT probes for out-of-process profilers (e.g.
# dd-otel-host-profiler) to pick up. Depends on libdatadog's in-progress
# sgg/heap-prof-poc branch; see the workspace Cargo.toml comment.
#

[iunanua]

would it be interesting to add an integration test?

[scottgerring]

would it be interesting to add an integration test?

@iunanua I've added some basic 'dd-trace-rs integration' testing in the last commit here. On the libdatadog side there is some reasonably extensive integration testing that validate that the sampling implementation is doing the right thing; I think in aggregate we have quite good coverage here now!

[r1viollet]

LGTM