Skip to content

Setup update system tests #7067

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

Setup update system tests #7067

wants to merge 1 commit into from

Conversation

@TonyCTHsu
Copy link
Contributor

@TonyCTHsu TonyCTHsu commented Jun 5, 2025 •
edited
Loading

Summary of changes

  1. Run system tests with a pinned commit SHA
  2. Automation for upgrade system tests version and open a pull request.

Reason for change

Inconsistent version using HEAD of main from system tests.

Implementation details

In order for the automated upgrade to work:

  1. Install Github App
  2. Add Secrets
  3. Allow certain Github Actions.

@github-actions github-actions bot added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Jun 5, 2025
@TonyCTHsu TonyCTHsu force-pushed the tonycthsu/update-system-tests branch from c803b18 to d7b93c2 Compare June 5, 2025 13:28
@andrewlock
Copy link
Member

andrewlock commented Jun 5, 2025

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7067) - mean (77ms)  : 71, 84
     .   : milestone, 77,
    master - mean (76ms)  : 70, 83
     .   : milestone, 76,

    section CallTarget+Inlining+NGEN
    This PR (7067) - mean (1,082ms)  : 1038, 1126
     .   : milestone, 1082,
    master - mean (1,085ms)  : 1039, 1131
     .   : milestone, 1085,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7067) - mean (114ms)  : 109, 119
     .   : milestone, 114,
    master - mean (114ms)  : 106, 121
     .   : milestone, 114,

    section CallTarget+Inlining+NGEN
    This PR (7067) - mean (763ms)  : 743, 782
     .   : milestone, 763,
    master - mean (755ms)  : 724, 786
     .   : milestone, 755,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7067) - mean (101ms)  : 92, 109
     .   : milestone, 101,
    master - mean (100ms)  : 94, 106
     .   : milestone, 100,

    section CallTarget+Inlining+NGEN
    This PR (7067) - mean (718ms)  : 690, 746
     .   : milestone, 718,
    master - mean (728ms)  : 701, 755
     .   : milestone, 728,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7067) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (191ms)  : 186, 195
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (7067) - mean (1,168ms)  : 1151, 1185
     .   : milestone, 1168,
    master - mean (1,166ms)  : 1145, 1187
     .   : milestone, 1166,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7067) - mean (269ms)  : 265, 273
     .   : milestone, 269,
    master - mean (269ms)  : 265, 272
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7067) - mean (932ms)  : 915, 948
     .   : milestone, 932,
    master - mean (932ms)  : 919, 944
     .   : milestone, 932,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7067) - mean (262ms)  : 259, 265
     .   : milestone, 262,
    master - mean (262ms)  : 259, 265
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (7067) - mean (927ms)  : 910, 944
     .   : milestone, 927,
    master - mean (927ms)  : 913, 942
     .   : milestone, 927,
Loading

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Jun 5, 2025 •
edited
Loading

Datadog Report

Branch report: tonycthsu/update-system-tests
Commit report: d7b93c2
Test service: dd-trace-dotnet

✅ 0 Failed, 244996 Passed, 2460 Skipped, 15h 25m 53.09s Total Time
⌛ 1 Performance Regression

⌛ Performance Regressions vs Default Branch (1)

  • Profiler_allocation_cpu_walltime - scenarios 2.22s (+64.68ms, +3%) - Details

@pr-commenter
Copy link

pr-commenter bot commented Jun 5, 2025 •
edited
Loading

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7067 compared to master:

  • 2 benchmarks are slower, with geometric mean 1.198
  • 3 benchmarks have fewer allocations
  • 7 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7067

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.09 KB 6.14 KB 52 B 0.85%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 11.2μs 59.7ns 322ns 0 0 0 5.58 KB
master StartStopWithChild netcoreapp3.1 14.1μs 67.5ns 286ns 0 0 0 5.75 KB
master StartStopWithChild net472 22.1μs 119ns 651ns 1.03 0.411 0.103 6.09 KB
#7067 StartStopWithChild net6.0 10.5μs 32.8ns 127ns 0 0 0 5.56 KB
#7067 StartStopWithChild netcoreapp3.1 13.8μs 66.9ns 276ns 0 0 0 5.75 KB
#7067 StartStopWithChild net472 21.3μs 115ns 609ns 0.987 0.329 0.11 6.14 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 927μs 26.6ns 103ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 404ns 1.56μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.25ms 357ns 1.38μs 0 0 0 3.33 KB
#7067 WriteAndFlushEnrichedTraces net6.0 923μs 76.9ns 288ns 0 0 0 2.71 KB
#7067 WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 38.4ns 138ns 0 0 0 2.7 KB
#7067 WriteAndFlushEnrichedTraces net472 1.26ms 444ns 1.72μs 0 0 0 3.33 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 331μs 1.76μs 8.42μs 0 0 0 197.06 KB
master AllCycleSimpleBody netcoreapp3.1 510μs 1.45μs 5.61μs 0 0 0 204.77 KB
master AllCycleSimpleBody net472 436μs 119ns 460ns 36.6 2.16 0 236.35 KB
master AllCycleMoreComplexBody net6.0 338μs 1.76μs 8.8μs 0 0 0 200.56 KB
master AllCycleMoreComplexBody netcoreapp3.1 495μs 987ns 3.56μs 0 0 0 208.18 KB
master AllCycleMoreComplexBody net472 446μs 106ns 412ns 36.6 2.16 0 239.87 KB
master ObjectExtractorSimpleBody net6.0 311ns 1.77ns 12.3ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 409ns 1.93ns 8.2ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 303ns 0.175ns 0.676ns 0.0442 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.52μs 29.5ns 110ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.76μs 36.2ns 140ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.66μs 0.89ns 3.33ns 0.599 0 0 3.8 KB
#7067 AllCycleSimpleBody net6.0 331μs 265ns 1.03μs 0 0 0 197.41 KB
#7067 AllCycleSimpleBody netcoreapp3.1 483μs 878ns 3.4μs 0 0 0 205.17 KB
#7067 AllCycleSimpleBody net472 437μs 126ns 490ns 36.6 2.16 0 236.95 KB
#7067 AllCycleMoreComplexBody net6.0 342μs 770ns 2.88μs 0 0 0 200.91 KB
#7067 AllCycleMoreComplexBody netcoreapp3.1 512μs 262ns 909ns 0 0 0 208.58 KB
#7067 AllCycleMoreComplexBody net472 448μs 132ns 510ns 37.9 2.23 0 240.46 KB
#7067 ObjectExtractorSimpleBody net6.0 321ns 1.7ns 9.17ns 0 0 0 280 B
#7067 ObjectExtractorSimpleBody netcoreapp3.1 393ns 2.18ns 13.4ns 0 0 0 272 B
#7067 ObjectExtractorSimpleBody net472 304ns 0.0644ns 0.249ns 0.0444 0 0 281 B
#7067 ObjectExtractorMoreComplexBody net6.0 6.33μs 14.6ns 56.5ns 0 0 0 3.78 KB
#7067 ObjectExtractorMoreComplexBody netcoreapp3.1 7.85μs 37.7ns 151ns 0 0 0 3.69 KB
#7067 ObjectExtractorMoreComplexBody net472 6.84μs 2.47ns 9.55ns 0.581 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7067

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0 2.16 KB 2.14 KB -12 B -0.56%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 73.3μs 263ns 984ns 0 0 0 32.41 KB
master EncodeArgs netcoreapp3.1 95.7μs 36.2ns 130ns 0 0 0 32.4 KB
master EncodeArgs net472 107μs 17.6ns 65.8ns 4.82 0 0 32.51 KB
master EncodeLegacyArgs net6.0 143μs 122ns 472ns 0 0 0 2.16 KB
master EncodeLegacyArgs netcoreapp3.1 197μs 42.5ns 147ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 261μs 66.8ns 250ns 0 0 0 2.16 KB
#7067 EncodeArgs net6.0 74.5μs 160ns 599ns 0 0 0 32.41 KB
#7067 EncodeArgs netcoreapp3.1 95.1μs 93.7ns 363ns 0 0 0 32.4 KB
#7067 EncodeArgs net472 104μs 15.2ns 57ns 4.67 0 0 32.51 KB
#7067 EncodeLegacyArgs net6.0 147μs 12ns 46.4ns 0 0 0 2.14 KB
#7067 EncodeLegacyArgs netcoreapp3.1 197μs 189ns 732ns 0 0 0 2.14 KB
#7067 EncodeLegacyArgs net472 264μs 38.8ns 140ns 0 0 0 2.16 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 271μs 133ns 499ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 294μs 264ns 989ns 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 307μs 38.1ns 147ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 181μs 80.1ns 300ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 198μs 104ns 404ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 207μs 45.3ns 169ns 0 0 0 2.28 KB
#7067 RunWafRealisticBenchmark net6.0 270μs 69ns 267ns 0 0 0 4.55 KB
#7067 RunWafRealisticBenchmark netcoreapp3.1 291μs 414ns 1.6μs 0 0 0 4.48 KB
#7067 RunWafRealisticBenchmark net472 306μs 25.5ns 98.8ns 0 0 0 4.66 KB
#7067 RunWafRealisticBenchmarkWithAttack net6.0 182μs 93.8ns 351ns 0 0 0 2.24 KB
#7067 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 194μs 66.5ns 249ns 0 0 0 2.22 KB
#7067 RunWafRealisticBenchmarkWithAttack net472 206μs 56ns 217ns 0 0 0 2.28 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.1μs 29ns 104ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 69.9μs 118ns 440ns 0 0 0 17.42 KB
master SendRequest net472 0.0171ns 0.00147ns 0.00551ns 0 0 0 0 b
#7067 SendRequest net6.0 61.6μs 68.2ns 264ns 0 0 0 14.53 KB
#7067 SendRequest netcoreapp3.1 72μs 367ns 1.76μs 0 0 0 17.42 KB
#7067 SendRequest net472 0.0238ns 0.00178ns 0.00691ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7067

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 1.166 631,437.50 736,559.72 several?

More allocations ⚠️ in #7067

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 55.75 KB 56.39 KB 636 B 1.14%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.89 KB 42.27 KB 387 B 0.92%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 645μs 512ns 1.98μs 0 0 0 41.73 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 631μs 1.67μs 6.67μs 0 0 0 41.89 KB
master WriteAndFlushEnrichedTraces net472 923μs 1.93μs 7.2μs 4.46 0 0 55.75 KB
#7067 WriteAndFlushEnrichedTraces net6.0 665μs 1.05μs 4.07μs 0 0 0 41.69 KB
#7067 WriteAndFlushEnrichedTraces netcoreapp3.1 724μs 3.83μs 19.9μs 0 0 0 42.27 KB
#7067 WriteAndFlushEnrichedTraces net472 907μs 2.7μs 10.5μs 8.33 0 0 56.39 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.99μs 10.2ns 46.8ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.53μs 3.43ns 13.3ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.7μs 2.41ns 9.34ns 0.147 0.0134 0 995 B
#7067 ExecuteNonQuery net6.0 1.98μs 2.65ns 9.17ns 0 0 0 1.03 KB
#7067 ExecuteNonQuery netcoreapp3.1 2.5μs 2.23ns 8.63ns 0 0 0 1.02 KB
#7067 ExecuteNonQuery net472 2.74μs 3.39ns 13.1ns 0.15 0.0136 0 995 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.83μs 5.85ns 22.7ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.3μs 11.8ns 56.6ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.55μs 3.05ns 11.8ns 0.159 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.81μs 3.89ns 14.5ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.35μs 7.26ns 28.1ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.84μs 3.19ns 12.4ns 0.169 0 0 1.11 KB
#7067 CallElasticsearch net6.0 1.8μs 9.5ns 46.6ns 0 0 0 1.04 KB
#7067 CallElasticsearch netcoreapp3.1 2.27μs 7.77ns 26.9ns 0 0 0 1.04 KB
#7067 CallElasticsearch net472 3.63μs 5.58ns 21.6ns 0.166 0 0 1.05 KB
#7067 CallElasticsearchAsync net6.0 1.83μs 6.21ns 22.4ns 0 0 0 1.02 KB
#7067 CallElasticsearchAsync netcoreapp3.1 2.41μs 9.63ns 37.3ns 0 0 0 1.09 KB
#7067 CallElasticsearchAsync net472 3.85μs 1.61ns 6.03ns 0.173 0 0 1.11 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.78μs 9.27ns 43.5ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.29μs 7.45ns 25.8ns 0 0 0 960 B
master ExecuteAsync net472 2.58μs 1.78ns 6.88ns 0.143 0 0 923 B
#7067 ExecuteAsync net6.0 1.82μs 1.16ns 4.33ns 0 0 0 960 B
#7067 ExecuteAsync netcoreapp3.1 2.3μs 9.38ns 36.3ns 0 0 0 960 B
#7067 ExecuteAsync net472 2.58μs 0.921ns 3.57ns 0.141 0 0 923 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 7.1μs 10.4ns 39ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.69μs 14.1ns 54.6ns 0 0 0 2.9 KB
master SendAsync net472 12.5μs 10.3ns 38.5ns 0.498 0 0 3.19 KB
#7067 SendAsync net6.0 7.1μs 8.49ns 32.9ns 0 0 0 2.37 KB
#7067 SendAsync netcoreapp3.1 8.49μs 11.8ns 45.5ns 0 0 0 2.9 KB
#7067 SendAsync net472 12.5μs 10.7ns 41.6ns 0.501 0 0 3.19 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7067

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 1.230 413,700.00 508,850.00

More allocations ⚠️ in #7067

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 57.34 KB 73.73 KB 16.38 KB 28.57%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 259.96 KB 276.5 KB 16.54 KB 6.36%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 42.64 KB 43.19 KB 552 B 1.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 274.93 KB 277.29 KB 2.36 KB 0.86%

Fewer allocations 🎉 in #7067

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.83 KB 43.1 KB -728 B -1.66%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 286.72 KB 278.53 KB -8.19 KB -2.86%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 44.6μs 212ns 1.6μs 0 0 0 43.83 KB
master StringConcatBenchmark netcoreapp3.1 47.2μs 224ns 838ns 0 0 0 42.64 KB
master StringConcatBenchmark net472 56.8μs 259ns 968ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 458μs 1.08μs 3.89μs 0 0 0 259.96 KB
master StringConcatAspectBenchmark netcoreapp3.1 447μs 6.44μs 63.7μs 0 0 0 274.93 KB
master StringConcatAspectBenchmark net472 410μs 2.07μs 9.27μs 0 0 0 286.72 KB
#7067 StringConcatBenchmark net6.0 47.3μs 233ns 1.38μs 0 0 0 43.1 KB
#7067 StringConcatBenchmark netcoreapp3.1 46.5μs 225ns 983ns 0 0 0 43.19 KB
#7067 StringConcatBenchmark net472 56.9μs 244ns 947ns 0 0 0 73.73 KB
#7067 StringConcatAspectBenchmark net6.0 504μs 2.32μs 9.28μs 0 0 0 276.5 KB
#7067 StringConcatAspectBenchmark netcoreapp3.1 513μs 2.28μs 8.53μs 0 0 0 277.29 KB
#7067 StringConcatAspectBenchmark net472 406μs 2.16μs 15.1μs 0 0 0 278.53 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.63μs 2.6ns 10.1ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.44μs 4.38ns 17ns 0 0 0 1.76 KB
master EnrichedLog net472 4.05μs 4.83ns 18.7ns 0.265 0 0 1.69 KB
#7067 EnrichedLog net6.0 2.59μs 12.3ns 47.7ns 0 0 0 1.76 KB
#7067 EnrichedLog netcoreapp3.1 3.51μs 6.32ns 24.5ns 0 0 0 1.76 KB
#7067 EnrichedLog net472 4.08μs 3.98ns 15.4ns 0.267 0 0 1.69 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 145ns 563ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 126μs 330ns 1.24μs 0 0 0 4.37 KB
master EnrichedLog net472 167μs 161ns 603ns 0 0 0 4.57 KB
#7067 EnrichedLog net6.0 122μs 125ns 483ns 0 0 0 4.37 KB
#7067 EnrichedLog netcoreapp3.1 127μs 181ns 627ns 0 0 0 4.37 KB
#7067 EnrichedLog net472 167μs 260ns 1.01μs 0 0 0 4.57 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.94μs 19.3ns 74.6ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.77μs 22.2ns 79.9ns 0 0 0 2.32 KB
master EnrichedLog net472 7.45μs 8.04ns 31.1ns 0.335 0 0 2.14 KB
#7067 EnrichedLog net6.0 4.99μs 15.8ns 61ns 0 0 0 2.32 KB
#7067 EnrichedLog netcoreapp3.1 6.64μs 3.73ns 14ns 0 0 0 2.32 KB
#7067 EnrichedLog net472 7.5μs 3.99ns 14.9ns 0.337 0 0 2.14 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.04μs 10.5ns 49.4ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.53μs 11.8ns 47.4ns 0 0 0 1.21 KB
master SendReceive net472 3.28μs 2.71ns 10.5ns 0.178 0 0 1.21 KB
#7067 SendReceive net6.0 2.01μs 9.56ns 40.6ns 0 0 0 1.21 KB
#7067 SendReceive netcoreapp3.1 2.51μs 8.33ns 32.3ns 0 0 0 1.21 KB
#7067 SendReceive net472 3.18μs 2.67ns 10.3ns 0.191 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.18μs 0.92ns 3.44ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.64μs 23ns 89.3ns 0 0 0 1.69 KB
master EnrichedLog net472 6.67μs 7.28ns 27.3ns 0.298 0 0 2.08 KB
#7067 EnrichedLog net6.0 4.31μs 5.17ns 19.3ns 0 0 0 1.64 KB
#7067 EnrichedLog netcoreapp3.1 5.59μs 6.42ns 24.8ns 0 0 0 1.69 KB
#7067 EnrichedLog net472 6.53μs 6.89ns 26.7ns 0.325 0 0 2.08 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 751ns 3.97ns 22.4ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 953ns 4.42ns 17.7ns 0 0 0 584 B
master StartFinishSpan net472 914ns 0.809ns 3.13ns 0.0912 0 0 586 B
master StartFinishScope net6.0 918ns 0.479ns 1.73ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.15μs 6.24ns 34.2ns 0 0 0 704 B
master StartFinishScope net472 1.09μs 0.174ns 0.652ns 0.104 0 0 666 B
#7067 StartFinishSpan net6.0 748ns 3.85ns 18.1ns 0 0 0 584 B
#7067 StartFinishSpan netcoreapp3.1 929ns 4.94ns 24.7ns 0 0 0 584 B
#7067 StartFinishSpan net472 914ns 0.223ns 0.864ns 0.092 0 0 586 B
#7067 StartFinishScope net6.0 895ns 4.29ns 17.1ns 0 0 0 704 B
#7067 StartFinishScope netcoreapp3.1 1.18μs 1.27ns 4.76ns 0 0 0 705 B
#7067 StartFinishScope net472 1.12μs 0.14ns 0.506ns 0.101 0 0 666 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.02μs 5.36ns 26.8ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.38μs 2.28ns 8.82ns 0 0 0 704 B
master RunOnMethodBegin net472 1.36μs 0.141ns 0.544ns 0.102 0 0 666 B
#7067 RunOnMethodBegin net6.0 1.03μs 4.74ns 17.7ns 0 0 0 704 B
#7067 RunOnMethodBegin netcoreapp3.1 1.33μs 3.27ns 12.7ns 0 0 0 704 B
#7067 RunOnMethodBegin net472 1.38μs 0.395ns 1.53ns 0.104 0 0 666 B

@bouwkast
Copy link
Collaborator

bouwkast commented Oct 3, 2025

Closing as stale, likely something to consider though in the future if we want to pin system-tests but I'm not terribly motivated to do so at the time being

@bouwkast bouwkast closed this Oct 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area:builds project files, build scripts, pipelines, versioning, releases, packages

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@TonyCTHsu @andrewlock @bouwkast