Skip to content

Run dsm-throughput macrobenchmarks on master push & schedule #7686

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Oct 21, 2025
Merged

Run dsm-throughput macrobenchmarks on master push & schedule #7686

merged 3 commits into from
Oct 21, 2025

Conversation

@robcarlan-datadog
Copy link
Contributor

@robcarlan-datadog robcarlan-datadog commented Oct 21, 2025 •
edited
Loading

Summary of changes

One of our pipeline scripts started failing because we are failing to check out a tag. We should run dsm-throughput in the same manner as existing macrobenchmarks.

Reason for change

Set up scripts was failing to check out a tag, because we ran this on a tag pipeline in the release.

Implementation details

Test coverage

Other details

@github-actions github-actions bot added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Oct 21, 2025
@pr-commenter
Copy link

pr-commenter bot commented Oct 21, 2025 •
edited
Loading

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7686 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.650
  • 3 benchmarks have fewer allocations
  • 9 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.6μs 58.5ns 336ns 0 0 0 5.52 KB
master StartStopWithChild netcoreapp3.1 13.8μs 55.7ns 216ns 0 0 0 5.71 KB
master StartStopWithChild net472 22.4μs 87.6ns 316ns 0.999 0.444 0.111 6 KB
#7686 StartStopWithChild net6.0 10.7μs 52.2ns 233ns 0 0 0 5.52 KB
#7686 StartStopWithChild netcoreapp3.1 14.6μs 68.8ns 292ns 0 0 0 5.68 KB
#7686 StartStopWithChild net472 22.3μs 97.5ns 377ns 0.997 0.399 0.0997 5.99 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 947μs 220ns 850ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.01ms 316ns 1.22μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.19ms 145ns 542ns 0 0 0 3.31 KB
#7686 WriteAndFlushEnrichedTraces net6.0 934μs 178ns 689ns 0 0 0 2.71 KB
#7686 WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 185ns 693ns 0 0 0 2.7 KB
#7686 WriteAndFlushEnrichedTraces net472 1.19ms 162ns 628ns 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 348μs 1.38μs 5.36μs 0 0 0 179.56 KB
master AllCycleSimpleBody netcoreapp3.1 510μs 1.9μs 7.36μs 0 0 0 185.32 KB
master AllCycleSimpleBody net472 471μs 263ns 983ns 32.4 0 0 205.41 KB
master AllCycleMoreComplexBody net6.0 362μs 1.31μs 5.06μs 0 0 0 183.08 KB
master AllCycleMoreComplexBody netcoreapp3.1 530μs 1.41μs 5.08μs 0 0 0 188.75 KB
master AllCycleMoreComplexBody net472 485μs 135ns 504ns 31.2 0 0 208.94 KB
master ObjectExtractorSimpleBody net6.0 320ns 1.58ns 7.08ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 393ns 2.21ns 13.8ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 298ns 0.0452ns 0.169ns 0.0436 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.25μs 33.4ns 170ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.82μs 37.2ns 144ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.69μs 4.82ns 18.7ns 0.601 0 0 3.8 KB
#7686 AllCycleSimpleBody net6.0 356μs 1.12μs 4.35μs 0 0 0 179.56 KB
#7686 AllCycleSimpleBody netcoreapp3.1 500μs 2.53μs 11μs 0 0 0 185.32 KB
#7686 AllCycleSimpleBody net472 469μs 180ns 696ns 32.4 0 0 205.41 KB
#7686 AllCycleMoreComplexBody net6.0 369μs 652ns 2.53μs 0 0 0 183.08 KB
#7686 AllCycleMoreComplexBody netcoreapp3.1 508μs 2.61μs 12.5μs 0 0 0 188.75 KB
#7686 AllCycleMoreComplexBody net472 481μs 107ns 415ns 32.4 0 0 208.94 KB
#7686 ObjectExtractorSimpleBody net6.0 322ns 0.197ns 0.763ns 0 0 0 280 B
#7686 ObjectExtractorSimpleBody netcoreapp3.1 397ns 1.88ns 7.75ns 0 0 0 272 B
#7686 ObjectExtractorSimpleBody net472 299ns 0.0584ns 0.218ns 0.0437 0 0 281 B
#7686 ObjectExtractorMoreComplexBody net6.0 6.34μs 31.4ns 133ns 0 0 0 3.78 KB
#7686 ObjectExtractorMoreComplexBody netcoreapp3.1 7.85μs 38.8ns 164ns 0 0 0 3.69 KB
#7686 ObjectExtractorMoreComplexBody net472 6.6μs 0.791ns 2.96ns 0.596 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 75.9μs 255ns 954ns 0 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 95.6μs 214ns 828ns 0 0 0 32.4 KB
master EncodeArgs net472 109μs 19.1ns 71.4ns 4.91 0 0 32.51 KB
master EncodeLegacyArgs net6.0 145μs 49.3ns 191ns 0 0 0 2.15 KB
master EncodeLegacyArgs netcoreapp3.1 196μs 293ns 1.14μs 0 0 0 2.14 KB
master EncodeLegacyArgs net472 262μs 207ns 803ns 0 0 0 2.16 KB
#7686 EncodeArgs net6.0 77μs 228ns 885ns 0 0 0 32.4 KB
#7686 EncodeArgs netcoreapp3.1 95.1μs 308ns 1.19μs 0 0 0 32.4 KB
#7686 EncodeArgs net472 109μs 10.5ns 39.3ns 4.89 0 0 32.51 KB
#7686 EncodeLegacyArgs net6.0 143μs 12.9ns 48.3ns 0 0 0 2.15 KB
#7686 EncodeLegacyArgs netcoreapp3.1 200μs 139ns 538ns 0 0 0 2.14 KB
#7686 EncodeLegacyArgs net472 262μs 22.4ns 86.8ns 0 0 0 2.16 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #7686

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑netcoreapp3.1 2.434 732,635.94 300,974.78

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 394μs 118ns 442ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 412μs 116ns 433ns 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 436μs 41.3ns 154ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 285μs 28.7ns 108ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 667μs 14.6μs 146μs 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 313μs 26.1ns 101ns 0 0 0 2.29 KB
#7686 RunWafRealisticBenchmark net6.0 392μs 71.6ns 268ns 0 0 0 4.55 KB
#7686 RunWafRealisticBenchmark netcoreapp3.1 412μs 76.9ns 277ns 0 0 0 4.48 KB
#7686 RunWafRealisticBenchmark net472 441μs 20.7ns 77.5ns 0 0 0 4.66 KB
#7686 RunWafRealisticBenchmarkWithAttack net6.0 285μs 28.9ns 112ns 0 0 0 2.24 KB
#7686 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 301μs 202ns 756ns 0 0 0 2.22 KB
#7686 RunWafRealisticBenchmarkWithAttack net472 310μs 30.5ns 118ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 62.2μs 48ns 186ns 0 0 0 14.52 KB
master SendRequest netcoreapp3.1 71.9μs 48.8ns 176ns 0 0 0 17.42 KB
master SendRequest net472 0.00346ns 0.00135ns 0.00524ns 0 0 0 0 b
#7686 SendRequest net6.0 59.7μs 54.6ns 204ns 0 0 0 14.52 KB
#7686 SendRequest netcoreapp3.1 73μs 387ns 2.05μs 0 0 0 17.42 KB
#7686 SendRequest net472 0.000295ns 0.000216ns 0.000835ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #7686

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0 1.119 1,535,827.78 1,372,783.33

More allocations ⚠️ in #7686

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net472 0 b 73 B 73 B ∞
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑netcoreapp3.1 0 b 1 B 1 B ∞
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net472 0 b 48 B 48 B ∞
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0 4 B 7 B 3 B 75.00%

Fewer allocations 🎉 in #7686

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0 3 B 1 B -2 B -66.67%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master OriginalCharSlice net6.0 1.95ms 4.73μs 17μs 0 0 0 640.01 KB
master OriginalCharSlice netcoreapp3.1 2.15ms 5.05μs 19.5μs 0 0 0 640 KB
master OriginalCharSlice net472 2.59ms 106ns 398ns 100 0 0 641.95 KB
master OptimizedCharSlice net6.0 1.54ms 281ns 1.05μs 0 0 0 4 B
master OptimizedCharSlice netcoreapp3.1 1.69ms 234ns 907ns 0 0 0 1 B
master OptimizedCharSlice net472 1.91ms 169ns 655ns 0 0 0 0 b
master OptimizedCharSliceWithPool net6.0 849μs 20ns 77.3ns 0 0 0 3 B
master OptimizedCharSliceWithPool netcoreapp3.1 797μs 132ns 493ns 0 0 0 0 b
master OptimizedCharSliceWithPool net472 1.13ms 82.7ns 320ns 0 0 0 0 b
#7686 OriginalCharSlice net6.0 1.9ms 8.98μs 34.8μs 0 0 0 640.01 KB
#7686 OriginalCharSlice netcoreapp3.1 2.07ms 3.79μs 14.2μs 0 0 0 640 KB
#7686 OriginalCharSlice net472 2.67ms 284ns 1.1μs 100 0 0 641.95 KB
#7686 OptimizedCharSlice net6.0 1.37ms 202ns 782ns 0 0 0 7 B
#7686 OptimizedCharSlice netcoreapp3.1 1.82ms 371ns 1.44μs 0 0 0 1 B
#7686 OptimizedCharSlice net472 1.91ms 202ns 784ns 0 0 0 73 B
#7686 OptimizedCharSliceWithPool net6.0 888μs 25.9ns 96.9ns 0 0 0 1 B
#7686 OptimizedCharSliceWithPool netcoreapp3.1 815μs 111ns 430ns 0 0 0 1 B
#7686 OptimizedCharSliceWithPool net472 1.2ms 125ns 484ns 0 0 0 48 B
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7686

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 55.94 KB 56.36 KB 417 B 0.75%

Fewer allocations 🎉 in #7686

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 42.35 KB 41.85 KB -499 B -1.18%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 731μs 909ns 3.52μs 0 0 0 42.35 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 749μs 4.12μs 24μs 0 0 0 41.78 KB
master WriteAndFlushEnrichedTraces net472 940μs 4.46μs 17.3μs 4.81 0 0 55.94 KB
#7686 WriteAndFlushEnrichedTraces net6.0 711μs 3.85μs 21.4μs 0 0 0 41.85 KB
#7686 WriteAndFlushEnrichedTraces netcoreapp3.1 732μs 4.22μs 42μs 0 0 0 41.86 KB
#7686 WriteAndFlushEnrichedTraces net472 862μs 2.17μs 8.39μs 8.33 0 0 56.36 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.96μs 9.75ns 41.4ns 0 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 2.62μs 10.3ns 39.7ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.86μs 4.37ns 16.9ns 0.143 0.0143 0 987 B
#7686 ExecuteNonQuery net6.0 1.94μs 4.54ns 17.6ns 0 0 0 1.02 KB
#7686 ExecuteNonQuery netcoreapp3.1 2.59μs 9.22ns 35.7ns 0 0 0 1.02 KB
#7686 ExecuteNonQuery net472 2.75μs 4.46ns 17.3ns 0.151 0.0137 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.77μs 9.27ns 43.5ns 0 0 0 1.03 KB
master CallElasticsearch netcoreapp3.1 2.33μs 10.2ns 36.6ns 0 0 0 1.03 KB
master CallElasticsearch net472 3.54μs 1.11ns 4.31ns 0.159 0 0 1.04 KB
master CallElasticsearchAsync net6.0 1.85μs 1.09ns 3.92ns 0 0 0 1.01 KB
master CallElasticsearchAsync netcoreapp3.1 2.37μs 6.58ns 24.6ns 0 0 0 1.08 KB
master CallElasticsearchAsync net472 3.71μs 1.96ns 7.59ns 0.166 0 0 1.1 KB
#7686 CallElasticsearch net6.0 1.79μs 8.63ns 33.4ns 0 0 0 1.03 KB
#7686 CallElasticsearch netcoreapp3.1 2.29μs 11.3ns 42.3ns 0 0 0 1.03 KB
#7686 CallElasticsearch net472 3.55μs 2ns 7.75ns 0.16 0 0 1.04 KB
#7686 CallElasticsearchAsync net6.0 1.9μs 2.81ns 10.9ns 0 0 0 1.01 KB
#7686 CallElasticsearchAsync netcoreapp3.1 2.39μs 5.93ns 22.2ns 0 0 0 1.08 KB
#7686 CallElasticsearchAsync net472 3.72μs 1.54ns 5.78ns 0.167 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.85μs 7.61ns 27.5ns 0 0 0 952 B
master ExecuteAsync netcoreapp3.1 2.29μs 10.8ns 45.9ns 0 0 0 952 B
master ExecuteAsync net472 2.71μs 7.73ns 28.9ns 0.133 0 0 915 B
#7686 ExecuteAsync net6.0 1.75μs 5.34ns 20.7ns 0 0 0 952 B
#7686 ExecuteAsync netcoreapp3.1 2.27μs 8.57ns 33.2ns 0 0 0 952 B
#7686 ExecuteAsync net472 2.62μs 2.78ns 10.4ns 0.143 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.84μs 7.28ns 26.2ns 0 0 0 2.36 KB
master SendAsync netcoreapp3.1 8.42μs 26.3ns 102ns 0 0 0 2.9 KB
master SendAsync net472 12.8μs 6.69ns 25ns 0.448 0 0 3.18 KB
#7686 SendAsync net6.0 6.82μs 7.7ns 28.8ns 0 0 0 2.36 KB
#7686 SendAsync netcoreapp3.1 8.93μs 17.2ns 66.6ns 0 0 0 2.9 KB
#7686 SendAsync net472 12.1μs 8.4ns 32.5ns 0.482 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7686

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 57.34 KB 65.54 KB 8.19 KB 14.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 42.68 KB 45.13 KB 2.45 KB 5.74%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 278.53 KB 286.72 KB 8.19 KB 2.94%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 274.26 KB 278.71 KB 4.45 KB 1.62%

Fewer allocations 🎉 in #7686

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 273.64 KB 258.18 KB -15.46 KB -5.65%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 44.1μs 235ns 1.17μs 0 0 0 43.82 KB
master StringConcatBenchmark netcoreapp3.1 46.8μs 226ns 1.26μs 0 0 0 42.68 KB
master StringConcatBenchmark net472 55.7μs 232ns 898ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 476μs 1.02μs 3.52μs 0 0 0 274.26 KB
master StringConcatAspectBenchmark netcoreapp3.1 524μs 2.52μs 9.78μs 0 0 0 273.64 KB
master StringConcatAspectBenchmark net472 406μs 2.34μs 18.4μs 0 0 0 278.53 KB
#7686 StringConcatBenchmark net6.0 41.3μs 190ns 762ns 0 0 0 43.82 KB
#7686 StringConcatBenchmark netcoreapp3.1 46.5μs 257ns 2.01μs 0 0 0 45.13 KB
#7686 StringConcatBenchmark net472 57.1μs 179ns 694ns 0 0 0 65.54 KB
#7686 StringConcatAspectBenchmark net6.0 479μs 2.35μs 9.7μs 0 0 0 278.71 KB
#7686 StringConcatAspectBenchmark netcoreapp3.1 533μs 1.74μs 6.73μs 0 0 0 258.18 KB
#7686 StringConcatAspectBenchmark net472 401μs 2.01μs 9.84μs 0 0 0 286.72 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.58μs 12ns 49.3ns 0 0 0 1.7 KB
master EnrichedLog netcoreapp3.1 3.61μs 17.9ns 75.8ns 0 0 0 1.7 KB
master EnrichedLog net472 3.81μs 2.51ns 9.41ns 0.247 0 0 1.64 KB
#7686 EnrichedLog net6.0 2.59μs 13.1ns 59.9ns 0 0 0 1.7 KB
#7686 EnrichedLog netcoreapp3.1 3.68μs 13.8ns 53.3ns 0 0 0 1.7 KB
#7686 EnrichedLog net472 3.8μs 3.91ns 15.1ns 0.245 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 122μs 64.1ns 240ns 0 0 0 4.31 KB
master EnrichedLog netcoreapp3.1 128μs 394ns 1.53μs 0 0 0 4.31 KB
master EnrichedLog net472 166μs 16.4ns 56.6ns 0 0 0 4.51 KB
#7686 EnrichedLog net6.0 122μs 69.8ns 261ns 0 0 0 4.31 KB
#7686 EnrichedLog netcoreapp3.1 127μs 367ns 1.37μs 0 0 0 4.31 KB
#7686 EnrichedLog net472 166μs 53.2ns 184ns 0 0 0 4.51 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.88μs 16.6ns 64.1ns 0 0 0 2.26 KB
master EnrichedLog netcoreapp3.1 6.63μs 18.6ns 69.5ns 0 0 0 2.26 KB
master EnrichedLog net472 7.75μs 7.52ns 29.1ns 0.313 0 0 2.08 KB
#7686 EnrichedLog net6.0 4.98μs 5.2ns 20.1ns 0 0 0 2.26 KB
#7686 EnrichedLog netcoreapp3.1 6.83μs 18.7ns 72.5ns 0 0 0 2.26 KB
#7686 EnrichedLog net472 7.76μs 6.02ns 23.3ns 0.311 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.08μs 10.2ns 43.4ns 0 0 0 1.2 KB
master SendReceive netcoreapp3.1 2.61μs 0.898ns 3.48ns 0 0 0 1.2 KB
master SendReceive net472 3.45μs 4.91ns 19ns 0.188 0 0 1.2 KB
#7686 SendReceive net6.0 2.01μs 9.44ns 36.6ns 0 0 0 1.2 KB
#7686 SendReceive netcoreapp3.1 2.65μs 13.4ns 56.7ns 0 0 0 1.2 KB
#7686 SendReceive net472 3.27μs 2.64ns 9.88ns 0.178 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.28μs 15.1ns 58.4ns 0 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 5.67μs 12.2ns 45.5ns 0 0 0 1.63 KB
master EnrichedLog net472 6.67μs 9.05ns 35ns 0.3 0 0 2.03 KB
#7686 EnrichedLog net6.0 4.34μs 14.7ns 55.1ns 0 0 0 1.58 KB
#7686 EnrichedLog netcoreapp3.1 5.45μs 15.9ns 61.5ns 0 0 0 1.63 KB
#7686 EnrichedLog net472 6.68μs 8.75ns 33.9ns 0.299 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 769ns 3.61ns 14.4ns 0 0 0 576 B
master StartFinishSpan netcoreapp3.1 959ns 4.89ns 23.5ns 0 0 0 576 B
master StartFinishSpan net472 931ns 1.28ns 4.97ns 0.0882 0 0 578 B
master StartFinishScope net6.0 894ns 0.398ns 1.44ns 0 0 0 696 B
master StartFinishScope netcoreapp3.1 1.16μs 3.88ns 15ns 0 0 0 696 B
master StartFinishScope net472 1.13μs 0.666ns 2.58ns 0.101 0 0 658 B
#7686 StartFinishSpan net6.0 758ns 0.452ns 1.69ns 0 0 0 576 B
#7686 StartFinishSpan netcoreapp3.1 967ns 0.553ns 2.07ns 0 0 0 576 B
#7686 StartFinishSpan net472 915ns 1.37ns 5.29ns 0.0876 0 0 578 B
#7686 StartFinishScope net6.0 922ns 4.7ns 23ns 0 0 0 696 B
#7686 StartFinishScope netcoreapp3.1 1.16μs 5.65ns 24ns 0 0 0 696 B
#7686 StartFinishScope net472 1.11μs 0.218ns 0.815ns 0.1 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.05μs 5.61ns 28ns 0 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 1.41μs 7.35ns 35.2ns 0 0 0 696 B
master RunOnMethodBegin net472 1.42μs 1.09ns 4.24ns 0.0993 0 0 658 B
#7686 RunOnMethodBegin net6.0 1.07μs 0.563ns 2.11ns 0 0 0 696 B
#7686 RunOnMethodBegin netcoreapp3.1 1.4μs 6.97ns 31.2ns 0 0 0 696 B
#7686 RunOnMethodBegin net472 1.48μs 3.03ns 11.8ns 0.103 0 0 658 B

@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Oct 21, 2025

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7686) - mean (74ms)  : 72, 77
     .   : milestone, 74,
    master - mean (71ms)  : 70, 72
     .   : milestone, 71,

    section Baseline
    This PR (7686) - mean (70ms)  : 68, 72
     .   : milestone, 70,
    master - mean (68ms)  : 66, 69
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7686) - mean (1,063ms)  : 1005, 1122
     .   : milestone, 1063,
    master - mean (1,040ms)  : 997, 1083
     .   : milestone, 1040,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7686) - mean (110ms)  : 107, 113
     .   : milestone, 110,
    master - mean (105ms)  : 104, 107
     .   : milestone, 105,

    section Baseline
    This PR (7686) - mean (109ms)  : 106, 113
     .   : milestone, 109,
    master - mean (105ms)  : 102, 107
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (7686) - mean (751ms)  : 726, 775
     .   : milestone, 751,
    master - mean (739ms)  : 717, 762
     .   : milestone, 739,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7686) - mean (97ms)  : 95, 98
     .   : milestone, 97,
    master - mean (94ms)  : 93, 95
     .   : milestone, 94,

    section Baseline
    This PR (7686) - mean (97ms)  : 94, 99
     .   : milestone, 97,
    master - mean (93ms)  : 91, 95
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (7686) - mean (714ms)  : 684, 745
     .   : milestone, 714,
    master - mean (701ms)  : 676, 727
     .   : milestone, 701,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7686) - mean (96ms)  : 95, 98
     .   : milestone, 96,
    master - mean (92ms)  : 91, 93
     .   : milestone, 92,

    section Baseline
    This PR (7686) - mean (95ms)  : 93, 98
     .   : milestone, 95,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (7686) - mean (670ms)  : 654, 686
     .   : milestone, 670,
    master - mean (658ms)  : 644, 673
     .   : milestone, 658,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7686) - mean (197ms)  : 193, 200
     .   : milestone, 197,
    master - mean (196ms)  : 193, 199
     .   : milestone, 196,

    section Baseline
    This PR (7686) - mean (193ms)  : 190, 197
     .   : milestone, 193,
    master - mean (192ms)  : 188, 196
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (7686) - mean (1,172ms)  : 1099, 1245
     .   : milestone, 1172,
    master - mean (1,164ms)  : 1108, 1220
     .   : milestone, 1164,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7686) - mean (284ms)  : 272, 297
     .   : milestone, 284,
    master - mean (277ms)  : 273, 281
     .   : milestone, 277,

    section Baseline
    This PR (7686) - mean (281ms)  : 270, 292
     .   : milestone, 281,
    master - mean (277ms)  : 271, 282
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (7686) - mean (954ms)  : 899, 1008
     .   : milestone, 954,
    master - mean (945ms)  : 897, 994
     .   : milestone, 945,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7686) - mean (278ms)  : 271, 285
     .   : milestone, 278,
    master - mean (270ms)  : 261, 279
     .   : milestone, 270,

    section Baseline
    This PR (7686) - mean (273ms)  : 266, 280
     .   : milestone, 273,
    master - mean (269ms)  : 264, 274
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7686) - mean (949ms)  : 898, 1000
     .   : milestone, 949,
    master - mean (929ms)  : 872, 986
     .   : milestone, 929,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7686) - mean (273ms)  : 267, 280
     .   : milestone, 273,
    master - mean (268ms)  : 265, 271
     .   : milestone, 268,

    section Baseline
    This PR (7686) - mean (274ms)  : 264, 284
     .   : milestone, 274,
    master - mean (269ms)  : 263, 274
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7686) - mean (863ms)  : 833, 894
     .   : milestone, 863,
    master - mean (853ms)  : 838, 869
     .   : milestone, 853,
Loading

@robcarlan-datadog robcarlan-datadog marked this pull request as ready for review October 21, 2025 16:20
@robcarlan-datadog robcarlan-datadog requested a review from a team as a code owner October 21, 2025 16:20
@robcarlan-datadog robcarlan-datadog changed the title Only run dsm-throughput on branch pipelines Run dsm-throughput macrobenchmarks on master push & schedule Oct 21, 2025
bouwkast
bouwkast approved these changes Oct 21, 2025
View reviewed changes
Copy link
Collaborator

@bouwkast bouwkast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@robcarlan-datadog robcarlan-datadog merged commit bdf4b20 into master Oct 21, 2025
150 checks passed
@robcarlan-datadog robcarlan-datadog deleted the rob.carlan/hotfix-dsm-throughput-ci branch October 21, 2025 17:32
@github-actions github-actions bot added this to the vNext-v3 milestone Oct 21, 2025
@andrewlock andrewlock added the area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc label Nov 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bouwkast bouwkast bouwkast approved these changes

@ddyurchenko ddyurchenko ddyurchenko approved these changes

Assignees

No one assigned

Labels

area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc area:builds project files, build scripts, pipelines, versioning, releases, packages

Projects

None yet

Milestone

3.30.0

Development

Successfully merging this pull request may close these issues.

5 participants

@robcarlan-datadog @bouwkast @ddyurchenko @andrewlock