Skip to content

[Debugger Default-On] DEBUG-4341 Forward snapshots to debugger intake #7388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 20 commits into from
Oct 13, 2025
Merged

[Debugger Default-On] DEBUG-4341 Forward snapshots to debugger intake #7388

merged 20 commits into from
Oct 13, 2025

Conversation

@dudikeleti
Copy link
Contributor

@dudikeleti dudikeleti commented Aug 13, 2025 •
edited by GreenMatan
Loading

Summary of changes

Dynamic Instrumentation and Exception Replay are about to be enabled by default with strict redaction rules. To support that, we have to perform redaction in the tracer (already done) and in the backend. Up until now, we uploaded our snapshots through Logs Intake. In Logs Intake there is no built in support for redaction. In order to support redaction at ingestion time, we direct our snapshots through debugger's diagnostics endpoint instead of Logs Intake.

Reason for change

Enable DI and ER features by default.

Implementation details

Direct DI&ER snapshots through Debugger's intake.

@github-actions github-actions bot added the area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) label Aug 13, 2025
@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Aug 13, 2025 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (72ms)  : 71, 73
     .   : milestone, 72,
    master - mean (72ms)  : 70, 73
     .   : milestone, 72,

    section Baseline
    This PR (7388) - mean (68ms)  : 66, 70
     .   : milestone, 68,
    master - mean (68ms)  : 66, 71
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (1,049ms)  : 1002, 1095
     .   : milestone, 1049,
    master - mean (1,049ms)  : 991, 1107
     .   : milestone, 1049,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (106ms)  : 105, 108
     .   : milestone, 106,
    master - mean (106ms)  : 105, 107
     .   : milestone, 106,

    section Baseline
    This PR (7388) - mean (106ms)  : 104, 108
     .   : milestone, 106,
    master - mean (105ms)  : 103, 108
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (744ms)  : 724, 764
     .   : milestone, 744,
    master - mean (743ms)  : 722, 763
     .   : milestone, 743,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (100ms)  : 99, 102
     .   : milestone, 100,
    master - mean (100ms)  : 99, 101
     .   : milestone, 100,

    section Baseline
    This PR (7388) - mean (100ms)  : 97, 102
     .   : milestone, 100,
    master - mean (100ms)  : 97, 102
     .   : milestone, 100,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (779ms)  : 737, 821
     .   : milestone, 779,
    master - mean (780ms)  : 746, 814
     .   : milestone, 780,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (92ms)  : 91, 94
     .   : milestone, 92,
    master - mean (93ms)  : 91, 94
     .   : milestone, 93,

    section Baseline
    This PR (7388) - mean (92ms)  : 90, 93
     .   : milestone, 92,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (657ms)  : 644, 670
     .   : milestone, 657,
    master - mean (663ms)  : 645, 680
     .   : milestone, 663,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (198ms)  : 195, 201
     .   : milestone, 198,
    master - mean (197ms)  : 195, 200
     .   : milestone, 197,

    section Baseline
    This PR (7388) - mean (194ms)  : 190, 197
     .   : milestone, 194,
    master - mean (195ms)  : 190, 199
     .   : milestone, 195,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (1,168ms)  : 1114, 1222
     .   : milestone, 1168,
    master - mean (1,166ms)  : 1113, 1220
     .   : milestone, 1166,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (279ms)  : 275, 284
     .   : milestone, 279,
    master - mean (279ms)  : 274, 284
     .   : milestone, 279,

    section Baseline
    This PR (7388) - mean (278ms)  : 274, 283
     .   : milestone, 278,
    master - mean (278ms)  : 273, 283
     .   : milestone, 278,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (947ms)  : 916, 978
     .   : milestone, 947,
    master - mean (945ms)  : 901, 988
     .   : milestone, 945,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (283ms)  : 278, 289
     .   : milestone, 283,
    master - mean (281ms)  : 277, 286
     .   : milestone, 281,

    section Baseline
    This PR (7388) - mean (283ms)  : 276, 289
     .   : milestone, 283,
    master - mean (282ms)  : 277, 287
     .   : milestone, 282,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (1,003ms)  : 960, 1045
     .   : milestone, 1003,
    master - mean (1,000ms)  : 972, 1028
     .   : milestone, 1000,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7388) - mean (271ms)  : 268, 274
     .   : milestone, 271,
    master - mean (272ms)  : 267, 276
     .   : milestone, 272,

    section Baseline
    This PR (7388) - mean (271ms)  : 267, 275
     .   : milestone, 271,
    master - mean (270ms)  : 265, 274
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (7388) - mean (855ms)  : 828, 881
     .   : milestone, 855,
    master - mean (856ms)  : 832, 880
     .   : milestone, 856,
Loading

@pr-commenter
Copy link

pr-commenter bot commented Aug 13, 2025 •
edited
Loading

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7388 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.713
  • 3 benchmarks are slower, with geometric mean 1.151
  • 4 benchmarks have fewer allocations
  • 4 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7388

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.09 KB 6.03 KB -52 B -0.85%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 11.1μs 61.7ns 380ns 0 0 0 5.51 KB
master StartStopWithChild netcoreapp3.1 13.8μs 53.8ns 208ns 0 0 0 5.72 KB
master StartStopWithChild net472 22.2μs 126ns 893ns 0.963 0.214 0 6.09 KB
#7388 StartStopWithChild net6.0 10.9μs 58.9ns 333ns 0 0 0 5.51 KB
#7388 StartStopWithChild netcoreapp3.1 13.6μs 71.5ns 365ns 0 0 0 5.71 KB
#7388 StartStopWithChild net472 22.3μs 126ns 856ns 1.02 0.34 0.113 6.03 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 960μs 1.21μs 4.67μs 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.03ms 110ns 382ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.21ms 312ns 1.21μs 0 0 0 3.31 KB
#7388 WriteAndFlushEnrichedTraces net6.0 932μs 92ns 344ns 0 0 0 2.71 KB
#7388 WriteAndFlushEnrichedTraces netcoreapp3.1 1.03ms 224ns 837ns 0 0 0 2.7 KB
#7388 WriteAndFlushEnrichedTraces net472 1.21ms 743ns 2.88μs 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 343μs 777ns 3.01μs 0 0 0 178.25 KB
master AllCycleSimpleBody netcoreapp3.1 500μs 2.54μs 11.6μs 0 0 0 184.01 KB
master AllCycleSimpleBody net472 459μs 131ns 506ns 31.2 0 0 203.96 KB
master AllCycleMoreComplexBody net6.0 350μs 1.63μs 6.5μs 0 0 0 181.77 KB
master AllCycleMoreComplexBody netcoreapp3.1 528μs 2.3μs 8.62μs 0 0 0 187.44 KB
master AllCycleMoreComplexBody net472 467μs 90.5ns 326ns 32.4 0 0 207.5 KB
master ObjectExtractorSimpleBody net6.0 320ns 0.167ns 0.648ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 403ns 1.07ns 4.14ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 296ns 0.0157ns 0.0545ns 0.0433 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.25μs 32.7ns 163ns 0 0 0 3.79 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.85μs 35.7ns 143ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.7μs 5.87ns 22.7ns 0.601 0 0 3.8 KB
#7388 AllCycleSimpleBody net6.0 346μs 1.62μs 6.28μs 0 0 0 178.25 KB
#7388 AllCycleSimpleBody netcoreapp3.1 504μs 2.35μs 12.2μs 0 0 0 184.01 KB
#7388 AllCycleSimpleBody net472 460μs 72.3ns 261ns 32.4 0 0 203.97 KB
#7388 AllCycleMoreComplexBody net6.0 351μs 1.7μs 6.78μs 0 0 0 181.77 KB
#7388 AllCycleMoreComplexBody netcoreapp3.1 524μs 1.67μs 6.46μs 0 0 0 187.44 KB
#7388 AllCycleMoreComplexBody net472 469μs 90.3ns 338ns 32.4 0 0 207.5 KB
#7388 ObjectExtractorSimpleBody net6.0 312ns 1.59ns 7.28ns 0 0 0 280 B
#7388 ObjectExtractorSimpleBody netcoreapp3.1 396ns 1.77ns 6.87ns 0 0 0 272 B
#7388 ObjectExtractorSimpleBody net472 297ns 0.0178ns 0.0688ns 0.0434 0 0 281 B
#7388 ObjectExtractorMoreComplexBody net6.0 6.28μs 28.6ns 111ns 0 0 0 3.78 KB
#7388 ObjectExtractorMoreComplexBody netcoreapp3.1 7.83μs 28.1ns 109ns 0 0 0 3.69 KB
#7388 ObjectExtractorMoreComplexBody net472 6.71μs 6.51ns 25.2ns 0.57 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 77.3μs 218ns 845ns 0 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 96.9μs 220ns 822ns 0 0 0 32.4 KB
master EncodeArgs net472 109μs 20.4ns 79.1ns 4.94 0 0 32.5 KB
master EncodeLegacyArgs net6.0 149μs 38.9ns 145ns 0 0 0 2.15 KB
master EncodeLegacyArgs netcoreapp3.1 201μs 471ns 1.82μs 0 0 0 2.14 KB
master EncodeLegacyArgs net472 263μs 107ns 413ns 0 0 0 2.16 KB
#7388 EncodeArgs net6.0 79.9μs 223ns 863ns 0 0 0 32.4 KB
#7388 EncodeArgs netcoreapp3.1 97μs 362ns 1.4μs 0 0 0 32.4 KB
#7388 EncodeArgs net472 110μs 32ns 120ns 4.93 0 0 32.5 KB
#7388 EncodeLegacyArgs net6.0 142μs 52.5ns 196ns 0 0 0 2.15 KB
#7388 EncodeLegacyArgs netcoreapp3.1 197μs 222ns 860ns 0 0 0 2.15 KB
#7388 EncodeLegacyArgs net472 264μs 210ns 814ns 0 0 0 2.16 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #7388

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑netcoreapp3.1 1.713 507,362.95 296,201.41

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 392μs 258ns 964ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 409μs 80.8ns 302ns 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 433μs 48.2ns 187ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 287μs 41.1ns 143ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 507μs 247ns 891ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 313μs 49.8ns 193ns 0 0 0 2.29 KB
#7388 RunWafRealisticBenchmark net6.0 393μs 92.8ns 335ns 0 0 0 4.56 KB
#7388 RunWafRealisticBenchmark netcoreapp3.1 411μs 276ns 1.03μs 0 0 0 4.48 KB
#7388 RunWafRealisticBenchmark net472 446μs 36.2ns 140ns 0 0 0 4.66 KB
#7388 RunWafRealisticBenchmarkWithAttack net6.0 285μs 26.1ns 97.5ns 0 0 0 2.24 KB
#7388 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 297μs 208ns 805ns 0 0 0 2.22 KB
#7388 RunWafRealisticBenchmarkWithAttack net472 310μs 25.9ns 100ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 60.9μs 68ns 245ns 0 0 0 14.52 KB
master SendRequest netcoreapp3.1 71.4μs 98.1ns 380ns 0 0 0 17.42 KB
master SendRequest net472 0.0119ns 0.00284ns 0.011ns 0 0 0 0 b
#7388 SendRequest net6.0 61.7μs 62.4ns 225ns 0 0 0 14.52 KB
#7388 SendRequest netcoreapp3.1 72μs 66ns 238ns 0 0 0 17.42 KB
#7388 SendRequest net472 0.00621ns 0.00276ns 0.0107ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7388

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice‑net6.0 1.153 1,909,657.14 2,201,271.88
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0 1.150 1,360,845.14 1,565,386.72
Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice‑netcoreapp3.1 1.150 2,097,669.53 2,412,136.72

More allocations ⚠️ in #7388

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0 4 B 5 B 1 B 25.00%

Fewer allocations 🎉 in #7388

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑netcoreapp3.1 1 B 0 b -1 B -100.00%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master OriginalCharSlice net6.0 1.91ms 1.39μs 5.21μs 0 0 0 640.01 KB
master OriginalCharSlice netcoreapp3.1 2.1ms 3.34μs 12.5μs 0 0 0 640 KB
master OriginalCharSlice net472 2.6ms 117ns 453ns 100 0 0 641.95 KB
master OptimizedCharSlice net6.0 1.36ms 447ns 1.67μs 0 0 0 4 B
master OptimizedCharSlice netcoreapp3.1 1.67ms 1.22μs 4.72μs 0 0 0 1 B
master OptimizedCharSlice net472 2.02ms 313ns 1.21μs 0 0 0 0 b
master OptimizedCharSliceWithPool net6.0 795μs 30.2ns 117ns 0 0 0 3 B
master OptimizedCharSliceWithPool netcoreapp3.1 837μs 173ns 671ns 0 0 0 1 B
master OptimizedCharSliceWithPool net472 1.18ms 259ns 1μs 0 0 0 0 b
#7388 OriginalCharSlice net6.0 2.2ms 415ns 1.55μs 0 0 0 640.01 KB
#7388 OriginalCharSlice netcoreapp3.1 2.43ms 11.6μs 46.4μs 0 0 0 640 KB
#7388 OriginalCharSlice net472 2.76ms 250ns 967ns 100 0 0 641.95 KB
#7388 OptimizedCharSlice net6.0 1.56ms 740ns 2.87μs 0 0 0 5 B
#7388 OptimizedCharSlice netcoreapp3.1 1.84ms 1.28μs 4.96μs 0 0 0 1 B
#7388 OptimizedCharSlice net472 2.03ms 523ns 2.03μs 0 0 0 0 b
#7388 OptimizedCharSliceWithPool net6.0 803μs 78.2ns 282ns 0 0 0 3 B
#7388 OptimizedCharSliceWithPool netcoreapp3.1 853μs 71.7ns 278ns 0 0 0 0 b
#7388 OptimizedCharSliceWithPool net472 1.17ms 91.3ns 354ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7388

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.71 KB 42.54 KB 826 B 1.98%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 689μs 524ns 2.03μs 0 0 0 41.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 812μs 3.31μs 11.9μs 0 0 0 41.91 KB
master WriteAndFlushEnrichedTraces net472 946μs 2.8μs 10.5μs 8.33 0 0 56.18 KB
#7388 WriteAndFlushEnrichedTraces net6.0 703μs 2.02μs 7.83μs 0 0 0 42.54 KB
#7388 WriteAndFlushEnrichedTraces netcoreapp3.1 747μs 2.57μs 9.94μs 0 0 0 42 KB
#7388 WriteAndFlushEnrichedTraces net472 938μs 2.32μs 8.69μs 4.46 0 0 56.08 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.95μs 9.95ns 47.7ns 0 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 2.51μs 12.1ns 47ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.77μs 4.02ns 15.6ns 0.155 0.0141 0 987 B
#7388 ExecuteNonQuery net6.0 1.94μs 8.48ns 30.6ns 0 0 0 1.02 KB
#7388 ExecuteNonQuery netcoreapp3.1 2.58μs 3.6ns 13.9ns 0 0 0 1.02 KB
#7388 ExecuteNonQuery net472 2.75μs 4.18ns 16.2ns 0.148 0.0135 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.76μs 9.2ns 40.1ns 0 0 0 1.03 KB
master CallElasticsearch netcoreapp3.1 2.23μs 9.07ns 35.1ns 0 0 0 1.03 KB
master CallElasticsearch net472 3.61μs 2.77ns 10.7ns 0.161 0 0 1.04 KB
master CallElasticsearchAsync net6.0 1.92μs 7.1ns 27.5ns 0 0 0 1.01 KB
master CallElasticsearchAsync netcoreapp3.1 2.42μs 10.5ns 40.6ns 0 0 0 1.08 KB
master CallElasticsearchAsync net472 3.91μs 2.86ns 10.7ns 0.156 0 0 1.1 KB
#7388 CallElasticsearch net6.0 1.74μs 9.26ns 47.2ns 0 0 0 1.03 KB
#7388 CallElasticsearch netcoreapp3.1 2.35μs 9.4ns 35.2ns 0 0 0 1.03 KB
#7388 CallElasticsearch net472 3.57μs 1.97ns 7.62ns 0.16 0 0 1.04 KB
#7388 CallElasticsearchAsync net6.0 1.93μs 10.4ns 58.7ns 0 0 0 1.01 KB
#7388 CallElasticsearchAsync netcoreapp3.1 2.43μs 10.2ns 36.7ns 0 0 0 1.08 KB
#7388 CallElasticsearchAsync net472 3.92μs 3.54ns 13.7ns 0.156 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.87μs 4.46ns 17.3ns 0 0 0 952 B
master ExecuteAsync netcoreapp3.1 2.22μs 1.18ns 4.27ns 0 0 0 952 B
master ExecuteAsync net472 2.63μs 1.14ns 4.26ns 0.144 0 0 915 B
#7388 ExecuteAsync net6.0 1.85μs 8.97ns 33.6ns 0 0 0 952 B
#7388 ExecuteAsync netcoreapp3.1 2.25μs 9.05ns 35ns 0 0 0 952 B
#7388 ExecuteAsync net472 2.6μs 5.17ns 20ns 0.143 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.98μs 13ns 50.4ns 0 0 0 2.36 KB
master SendAsync netcoreapp3.1 8.52μs 21.4ns 77.2ns 0 0 0 2.9 KB
master SendAsync net472 12.3μs 7.46ns 27.9ns 0.492 0 0 3.18 KB
#7388 SendAsync net6.0 6.88μs 16.3ns 58.9ns 0 0 0 2.36 KB
#7388 SendAsync netcoreapp3.1 8.4μs 24.3ns 94.2ns 0 0 0 2.9 KB
#7388 SendAsync net472 12.2μs 14.2ns 55.1ns 0.482 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7388

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 274.57 KB 286.72 KB 12.15 KB 4.43%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.48 KB 43.78 KB 296 B 0.68%

Fewer allocations 🎉 in #7388

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 43.58 KB 42.78 KB -792 B -1.82%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 287.19 KB 276.26 KB -10.94 KB -3.81%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 42.4μs 240ns 1.59μs 0 0 0 43.48 KB
master StringConcatBenchmark netcoreapp3.1 49.2μs 240ns 1.12μs 0 0 0 43.58 KB
master StringConcatBenchmark net472 57.6μs 159ns 596ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 515μs 980ns 3.39μs 0 0 0 287.19 KB
master StringConcatAspectBenchmark netcoreapp3.1 499μs 2.41μs 9.94μs 0 0 0 256.11 KB
master StringConcatAspectBenchmark net472 404μs 2.16μs 11.9μs 0 0 0 274.57 KB
#7388 StringConcatBenchmark net6.0 41.9μs 207ns 828ns 0 0 0 43.78 KB
#7388 StringConcatBenchmark netcoreapp3.1 50.5μs 528ns 5.12μs 0 0 0 42.78 KB
#7388 StringConcatBenchmark net472 57.5μs 286ns 1.25μs 0 0 0 57.34 KB
#7388 StringConcatAspectBenchmark net6.0 470μs 1.24μs 4.46μs 0 0 0 276.26 KB
#7388 StringConcatAspectBenchmark netcoreapp3.1 456μs 6.19μs 60.7μs 0 0 0 257.36 KB
#7388 StringConcatAspectBenchmark net472 405μs 2.08μs 9.32μs 0 0 0 286.72 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.5μs 12.2ns 48.9ns 0 0 0 1.7 KB
master EnrichedLog netcoreapp3.1 3.52μs 17.7ns 75.2ns 0 0 0 1.7 KB
master EnrichedLog net472 3.82μs 6ns 23.3ns 0.251 0 0 1.64 KB
#7388 EnrichedLog net6.0 2.56μs 13ns 59.4ns 0 0 0 1.7 KB
#7388 EnrichedLog netcoreapp3.1 3.59μs 13.7ns 53ns 0 0 0 1.7 KB
#7388 EnrichedLog net472 3.91μs 3.63ns 14.1ns 0.254 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 125μs 404ns 1.56μs 0 0 0 4.31 KB
master EnrichedLog netcoreapp3.1 129μs 593ns 2.22μs 0 0 0 4.31 KB
master EnrichedLog net472 168μs 48.2ns 180ns 0 0 0 4.52 KB
#7388 EnrichedLog net6.0 123μs 119ns 446ns 0 0 0 4.31 KB
#7388 EnrichedLog netcoreapp3.1 128μs 489ns 1.89μs 0 0 0 4.31 KB
#7388 EnrichedLog net472 166μs 57.2ns 198ns 0 0 0 4.52 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.8μs 18.9ns 73.1ns 0 0 0 2.26 KB
master EnrichedLog netcoreapp3.1 6.64μs 17.9ns 69.4ns 0 0 0 2.26 KB
master EnrichedLog net472 7.65μs 13.3ns 51.6ns 0.304 0 0 2.08 KB
#7388 EnrichedLog net6.0 4.73μs 23.9ns 95.7ns 0 0 0 2.26 KB
#7388 EnrichedLog netcoreapp3.1 6.79μs 26.4ns 102ns 0 0 0 2.26 KB
#7388 EnrichedLog net472 7.22μs 7.35ns 27.5ns 0.323 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.03μs 1.31ns 4.74ns 0 0 0 1.2 KB
master SendReceive netcoreapp3.1 2.65μs 13.3ns 57.8ns 0 0 0 1.2 KB
master SendReceive net472 3.11μs 1.51ns 5.43ns 0.186 0 0 1.2 KB
#7388 SendReceive net6.0 2.05μs 1.19ns 4.62ns 0 0 0 1.2 KB
#7388 SendReceive netcoreapp3.1 2.63μs 12.6ns 52.1ns 0 0 0 1.2 KB
#7388 SendReceive net472 3.17μs 9.66ns 37.4ns 0.188 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.2μs 2.51ns 9.71ns 0 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 5.41μs 15.6ns 60.4ns 0 0 0 1.63 KB
master EnrichedLog net472 6.69μs 5.7ns 22.1ns 0.302 0 0 2.03 KB
#7388 EnrichedLog net6.0 4.22μs 3.53ns 12.7ns 0 0 0 1.58 KB
#7388 EnrichedLog netcoreapp3.1 5.38μs 11.2ns 43.2ns 0 0 0 1.63 KB
#7388 EnrichedLog net472 6.37μs 5.21ns 19.5ns 0.317 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 737ns 3.6ns 15.7ns 0 0 0 576 B
master StartFinishSpan netcoreapp3.1 982ns 5.1ns 25.5ns 0 0 0 576 B
master StartFinishSpan net472 897ns 0.167ns 0.626ns 0.0904 0 0 578 B
master StartFinishScope net6.0 880ns 0.262ns 0.944ns 0 0 0 696 B
master StartFinishScope netcoreapp3.1 1.15μs 5.76ns 25.1ns 0 0 0 696 B
master StartFinishScope net472 1.07μs 0.219ns 0.79ns 0.102 0 0 658 B
#7388 StartFinishSpan net6.0 749ns 3.71ns 17.8ns 0 0 0 576 B
#7388 StartFinishSpan netcoreapp3.1 913ns 4.67ns 17.5ns 0 0 0 576 B
#7388 StartFinishSpan net472 871ns 0.164ns 0.613ns 0.0917 0 0 578 B
#7388 StartFinishScope net6.0 893ns 4.62ns 23.6ns 0 0 0 696 B
#7388 StartFinishScope netcoreapp3.1 1.14μs 5.41ns 22.3ns 0 0 0 696 B
#7388 StartFinishScope net472 1.12μs 1.03ns 3.97ns 0.101 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1μs 5.1ns 23.9ns 0 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 1.39μs 2.63ns 10.2ns 0 0 0 696 B
master RunOnMethodBegin net472 1.44μs 1.78ns 6.91ns 0.1 0 0 658 B
#7388 RunOnMethodBegin net6.0 1.04μs 5.04ns 19.5ns 0 0 0 696 B
#7388 RunOnMethodBegin netcoreapp3.1 1.43μs 5.92ns 22.9ns 0 0 0 696 B
#7388 RunOnMethodBegin net472 1.41μs 0.74ns 2.56ns 0.0993 0 0 658 B

@dudikeleti dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from 7c13a45 to 864a0f2 Compare August 27, 2025 08:41
@datadog-official

This comment has been minimized.

Sign in to view
@dudikeleti dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from 864a0f2 to cea6960 Compare September 2, 2025 15:59
@dudikeleti dudikeleti changed the title send snapshots to diagnostics endpoint [Dynamic Instrumentation] DEBUG-4341 Forward snapshots to debugger intake Sep 3, 2025
@dudikeleti dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from cea6960 to f1940c3 Compare September 3, 2025 10:00
@dudikeleti dudikeleti marked this pull request as ready for review September 3, 2025 10:13
@dudikeleti dudikeleti requested review from a team as code owners September 3, 2025 10:13
@dudikeleti dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch 2 times, most recently from 518ed8e to 925d913 Compare September 15, 2025 09:14
andrewlock
andrewlock approved these changes Sep 15, 2025
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from the tracer side

GreenMatan
GreenMatan approved these changes Sep 18, 2025
View reviewed changes
Copy link
Contributor

@GreenMatan GreenMatan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tracer/src/Datadog.Trace/Debugger/DynamicInstrumentation.cs
SetProbeStatusToEmitting(probe);
}

internal void AddLog(ProbeInfo probe, string log)
Copy link
Contributor

@GreenMatan GreenMatan Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we can mark it as private

Copy link
Contributor Author

@dudikeleti dudikeleti Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theoretically we can use it outside the class, like AddSnapshot so I'll keep it internal for now. We have to have a final decision about LogProbe, then we can update it accordingly.

tracer/src/Datadog.Trace/Debugger/DynamicInstrumentation.cs
return;
}

if (!probe.IsFullSnapshot)
Copy link
Contributor

@GreenMatan GreenMatan Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you direct only non full snapshot to the LogUploader?

Copy link
Contributor Author

@dudikeleti dudikeleti Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now we forwarding only snapshot probe

tracer/src/Datadog.Trace/Debugger/Upload/LogUploadApi.cs Outdated
IGitMetadataTagsProvider gitMetadataTagsProvider)
: base(apiRequestFactory, gitMetadataTagsProvider)
{
discoveryService.SubscribeToChanges(c => Endpoint = c.DebuggerEndpoint);
Copy link
Contributor

@GreenMatan GreenMatan Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure that we plan to use c.DebuggerEndpoint in here and not DiagnosticsEndpoint or what you're doing in SnapshotUploadApi;
Endpoint = c.DebuggerV2Endpoint ?? c.DiagnosticsEndpoint ?? c.DebuggerEndpoint;?

Copy link
Contributor Author

@dudikeleti dudikeleti Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now we forwarding only snapshot probe

@dudikeleti dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from 38665c7 to 9c363ee Compare September 22, 2025 12:33
@tylfin
Copy link
Member

tylfin commented Sep 22, 2025 •
edited
Loading

FYI, for Python I manually ran the following test which helped QA the code:

  1. Start application / tracer with the latest agent
  2. Add instrumentation and trigger (snapshots should be uploaded)
  3. Kill the agent, wait then trigger (check warning/error logs, not too verbose)
  4. Start the agent v7.47.0 and trigger (no snapshots should be uploaded)
  5. Kill the agent, and restart latest, and trigger (snapshots should against start being uploaded)

Could we test the same scenario here?

@dudikeleti dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch 2 times, most recently from c4f4dcc to 40ce506 Compare September 24, 2025 07:40
@dudikeleti dudikeleti changed the title [Dynamic Instrumentation] DEBUG-4341 Forward snapshots to debugger intake [Debugger Default-On] DEBUG-4341 Forward snapshots to debugger intake Sep 24, 2025
@dudikeleti dudikeleti mentioned this pull request Sep 26, 2025
Merged
6 tasks
@dudikeleti dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch 4 times, most recently from 125ec00 to e07d949 Compare October 2, 2025 11:46
@github-actions
Copy link
Contributor

github-actions bot commented Oct 2, 2025 •
edited
Loading

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

200 occurrences of :

+    "debugger.product": "di",
+    "debugger.type": "snapshot",

@dudikeleti dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch 3 times, most recently from 18b8dd3 to f3db1ed Compare October 8, 2025 10:56
dudikeleti and others added 20 commits October 11, 2025 10:24
@dudikeleti
send snapshots to diagnostics endpoint
b08c093
@dudikeleti
update mock agent
9f5448f
@dudikeleti
add debugger.type and debugger.product
6c694af
@dudikeleti
Support debugger V2 and send logs to V1
cd6f858
@dudikeleti
update debugger type and forwarding based on new logic
0367818
@dudikeleti
Revert "update debugger type and forwarding based on new logic"
636e7d9 
This reverts commit 3e1344c.
@dudikeleti @andrewlock
Update tracer/src/Datadog.Trace/Debugger/Upload/SymbolUploadApi.cs
f3eec7f 
Co-authored-by: Andrew Lock <[email protected]>
@dudikeleti
do not fallback do debugger v1
91de11a
@dudikeleti
update tests
80da5ae
@dudikeleti
Add debugger v2 to mock agent
3ca8f40
@dudikeleti
fix indentation
6bc04b2
@dudikeleti
Fix TracerMockAgent to support new endpoint
003ad16
@dudikeleti
Update snapshots approvals
ba0a934
@dudikeleti
Update snapshots
b4651af
@dudikeleti
update diagnostic approvals
23606a5
@dudikeleti
Update approvals for specific .net versions
7a501c3
@dudikeleti
Do not fallback to v1 endpoint
c5222e6
@dudikeleti
Do not fallback to v1 endpoint for diagnostic
a036553
@dudikeleti
Add debug logs for updated endpoint
681c8ba
@dudikeleti
return the falback for diagnostic and send only diagnostic as multipart
262bbbd
@dudikeleti dudikeleti force-pushed the dudik/send-snapshots-to-diagnostic branch from 23757cc to 262bbbd Compare October 11, 2025 08:24
@dudikeleti dudikeleti merged commit 79b266b into master Oct 13, 2025
157 checks passed
@dudikeleti dudikeleti deleted the dudik/send-snapshots-to-diagnostic branch October 13, 2025 09:43
@github-actions github-actions bot added this to the vNext-v3 milestone Oct 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@GreenMatan GreenMatan GreenMatan approved these changes

@andrewlock andrewlock andrewlock approved these changes

Assignees

No one assigned

Labels

area:debugger area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations)

Projects

None yet

Milestone

3.29.0

Development

Successfully merging this pull request may close these issues.

5 participants

@dudikeleti @tylfin @GreenMatan @andrewlock