feat(nativemem): instrument remaining allocation sites - #724
Conversation
…iveness table, thread info, call-trace buffers) Extends the NM_* categorized native-memory accounting (#669) to eight previously-uninstrumented allocation sites: Recording::_method_map and LivenessTracker::_table get new NM_METHOD_MAP/NM_LIVENESS categories; UnwindFailures, ThreadInfo's two maps, Lookup::_vtable_receiver_cache, CallTraceStorage's two working buffers, and wall-clock's thread-reservoir vector are folded into existing categories. Adds a stateless CountingAllocator<T, Category> that wraps ::operator new/delete and reports exact per-node byte counts via STL's allocator-rebinding, used for the STL-container sites; raw-malloc sites (UnwindFailures, LivenessTracker) get direct record()/setLive() calls. Also fixes CodeCache::setDwarfTable() to shrink the DWARF/SFrame FrameDesc table to its exact length via realloc, correcting memoryUsage()'s length-based formula and eliminating the parser's capacity-doubling slack from real RSS. Test fixture fix: NativeMemTest's SetUp()/TearDown() used to zero all live gauges unconditionally, which underflowed UnwindFailures's destructor decrement (a real static-duration object accounted for outside the test) at process exit. Snapshot and restore the pre-test baseline instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Scan-Build Report
Bug Summary
Reports
|
||||||||||||||||||||||||||||||||||||
This comment has been minimized.
This comment has been minimized.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1118c13363
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| @@ -1,4 +1,6 @@ | |||
| #include "countingAllocator.h" | |||
There was a problem hiding this comment.
This commit touches files that still do not satisfy the repository copyright rule: threadInfo.h, threadInfo.cpp, and unwindStats.h still lack a Datadog header, and several modified files with existing 2025 Datadog headers were not updated to the current year. Please add/update the current-year Datadog headers before landing.
AGENTS.md reference: AGENTS.md:L372-L374
Useful? React with 👍 / 👎.
CI Test ResultsRun: #31678457724 | Commit:
Status Overview
Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled Summary: Total: 32 | Passed: 32 | Failed: 0 Updated: 2026-08-13 07:54:05 UTC |
…e shrink-to-fit Add NM_THREAD_INFO/NM_WALLCLOCK categories, track thread-name heap bytes, shrink dwarf/sframe tables to exact size at the source, and add CountingAllocator unit tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Address PR #724 review feedback: add missing Datadog headers to threadInfo.cpp and unwindStats.h; update stale 2025 headers to 2025, 2026 in callTraceHashTable.h, callTraceStorage.{cpp,h}, flightRecorder.h, stress_callTraceStorage.cpp, test_callTraceStorage.cpp.
2 participants
Summary
Extends the categorized native-memory accounting from #669 to eight allocation sites that weren't yet instrumented:
Recording::_method_map-> newNM_METHOD_MAPcategoryLivenessTracker::_table-> newNM_LIVENESScategoryUnwindFailures,ThreadInfo's two maps ->NM_THREAD_LOCALLookup::_vtable_receiver_cache->NM_JFR_BUFFERSCallTraceStorage's two working buffers ->NM_CALLTRACENM_MISC(capacity-based gauge, not allocator-wrapped, sinceReservoirSampleris hard-coded to the default-allocatorstd::vector)Adds
CountingAllocator<T, Category>(ddprof-lib/src/main/cpp/countingAllocator.h): a stateless C++11 allocator wrapping::operator new/deletethat reports exact per-node byte counts via STL's allocator-rebinding, used for the STL-container sites. The two raw-mallocsites (UnwindFailures,LivenessTracker) get directrecord()/setLive()calls instead.Also fixes
CodeCache::setDwarfTable(): the DWARF/SFrame parsers build theirFrameDesctable with capacity-doublingmalloc/realloc, so the incoming buffer is typically larger than the storedlength.memoryUsage()'s formula waslength * sizeof(FrameDesc), silently undercounting real memory whenevercapacity > length.setDwarfTable()now shrinks the buffer to its exact size viareallocbefore storing it, which both corrects the formula and gives back the doubling slack to the allocator.Test fixture fix:
NativeMemTest'sSetUp()/TearDown()used to zero all live gauges unconditionally viaNativeMem::reset(). This is safe most of the time, butUnwindFailuresis a genuine class-static object (UnwindStats::_unwind_failures) constructed beforemain()and destructed at real process exit; if its constructor'srecord()runs before the test and its destructor's matching decrement runs afterTearDown()already zeroed the category, the decrement underflows and tripsassert(updated >= 0). Fixed by snapshotting the pre-test baseline and restoring it inTearDown()instead of assuming zero is a safe baseline to leave in place.Test plan
./gradlew :ddprof-lib:compileRelease— clean, all 66 sourcesflightRecorder_result_ut,methodInfo_hash_ut,methodMapId_ut,lineNumberTableCopy_ut,test_callTraceStorage,stress_callTraceStorage,profiler_null_calltrace_buffer_ut,threadInfo_ut,livenessTracker_ut,wallClockCounters_ut,codeCache_ut,sframe_ut,dwarf_ut,nativeMem_ut) — all pass🤖 Generated with Claude Code