Avoid creating new tasks on wait #7777

kr-igor · 2025-11-04T18:24:02Z

Summary of changes

Avoid creating extra tasks/timer. DSM is not use a dedicated background thread.

Reason for change

See this ticket.

Implementation details

Based on the information we have it seems like the main reason for extra overhead comes from the "Finalizer" thread which calls CloseHandle too frequently. It's not clear what causes the kernel resource allocation/disposal and why it only happens in customer's environment.

dd-trace-dotnet-ci-bot · 2025-11-04T19:07:27Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (7777) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	75.39 ± (75.04 - 75.82) ms	75.06 ± (75.14 - 76.09) ms	-0.4%	✅
.NET Framework 4.8 - Bailout
duration	79.26 ± (79.00 - 79.47) ms	79.74 ± (79.60 - 80.26) ms	+0.6%	✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1108.38 ± (1111.88 - 1122.66) ms	1108.11 ± (1109.33 - 1121.34) ms	-0.0%	✅
.NET Core 3.1 - Baseline
process.internal_duration_ms	22.91 ± (22.83 - 22.98) ms	22.90 ± (22.83 - 22.96) ms	-0.0%	✅
process.time_to_main_ms	87.06 ± (86.78 - 87.34) ms	86.57 ± (86.19 - 86.94) ms	-0.6%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.88 ± (10.88 - 10.88) MB	10.86 ± (10.85 - 10.86) MB	-0.2%	✅
runtime.dotnet.threads.count	12 ± (12 - 12)	12 ± (12 - 12)	+0.0%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	22.87 ± (22.81 - 22.93) ms	22.93 ± (22.86 - 22.99) ms	+0.3%	✅⬆️
process.time_to_main_ms	88.78 ± (88.49 - 89.07) ms	89.40 ± (89.00 - 89.79) ms	+0.7%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.92 ± (10.92 - 10.92) MB	10.90 ± (10.90 - 10.90) MB	-0.2%	✅
runtime.dotnet.threads.count	13 ± (13 - 13)	13 ± (13 - 13)	+0.0%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	219.93 ± (218.58 - 221.29) ms	224.07 ± (222.74 - 225.40) ms	+1.9%	✅⬆️
process.time_to_main_ms	548.73 ± (547.60 - 549.86) ms	552.73 ± (551.49 - 553.98) ms	+0.7%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	52.60 ± (52.57 - 52.62) MB	52.94 ± (52.92 - 52.97) MB	+0.7%	✅⬆️
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.2%	✅⬆️
.NET 6 - Baseline
process.internal_duration_ms	21.60 ± (21.53 - 21.67) ms	21.95 ± (21.88 - 22.02) ms	+1.6%	✅⬆️
process.time_to_main_ms	75.20 ± (74.85 - 75.55) ms	77.03 ± (76.74 - 77.32) ms	+2.4%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.59 ± (10.58 - 10.59) MB	10.58 ± (10.58 - 10.59) MB	-0.0%	✅
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 6 - Bailout
process.internal_duration_ms	21.56 ± (21.48 - 21.64) ms	21.85 ± (21.80 - 21.91) ms	+1.4%	✅⬆️
process.time_to_main_ms	77.09 ± (76.76 - 77.43) ms	77.82 ± (77.46 - 78.18) ms	+0.9%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.64 ± (10.63 - 10.64) MB	10.64 ± (10.64 - 10.64) MB	+0.0%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	205.68 ± (204.42 - 206.94) ms	208.66 ± (207.27 - 210.04) ms	+1.4%	✅⬆️
process.time_to_main_ms	510.72 ± (509.74 - 511.69) ms	515.32 ± (514.16 - 516.47) ms	+0.9%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	51.78 ± (51.74 - 51.82) MB	52.13 ± (52.10 - 52.15) MB	+0.7%	✅⬆️
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.3%	✅⬆️
.NET 8 - Baseline
process.internal_duration_ms	19.76 ± (19.69 - 19.82) ms	20.08 ± (20.01 - 20.15) ms	+1.6%	✅⬆️
process.time_to_main_ms	75.24 ± (74.90 - 75.58) ms	76.61 ± (76.29 - 76.92) ms	+1.8%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.61 ± (7.61 - 7.62) MB	7.61 ± (7.61 - 7.62) MB	-0.0%	✅
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 8 - Bailout
process.internal_duration_ms	19.58 ± (19.53 - 19.63) ms	20.11 ± (20.04 - 20.17) ms	+2.7%	✅⬆️
process.time_to_main_ms	75.46 ± (75.13 - 75.80) ms	78.36 ± (78.00 - 78.72) ms	+3.8%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.67 ± (7.66 - 7.67) MB	7.69 ± (7.68 - 7.70) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	188.91 ± (187.99 - 189.83) ms	194.48 ± (193.66 - 195.31) ms	+3.0%	✅⬆️
process.time_to_main_ms	488.40 ± (487.39 - 489.41) ms	491.01 ± (489.96 - 492.05) ms	+0.5%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	38.86 ± (38.83 - 38.90) MB	39.35 ± (39.30 - 39.39) MB	+1.2%	✅⬆️
runtime.dotnet.threads.count	27 ± (27 - 27)	27 ± (27 - 27)	+0.2%	✅⬆️

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	191.55 ± (191.38 - 192.05) ms	192.48 ± (192.53 - 193.25) ms	+0.5%	✅⬆️
.NET Framework 4.8 - Bailout
duration	194.96 ± (194.79 - 195.34) ms	195.73 ± (195.51 - 195.97) ms	+0.4%	✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1206.94 ± (1197.95 - 1208.71) ms	1169.55 ± (1170.74 - 1180.98) ms	-3.1%	✅
.NET Core 3.1 - Baseline
process.internal_duration_ms	200.58 ± (200.09 - 201.06) ms	188.32 ± (188.02 - 188.63) ms	-6.1%	✅
process.time_to_main_ms	87.03 ± (86.78 - 87.29) ms	80.67 ± (80.49 - 80.85) ms	-7.3%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	15.97 ± (15.95 - 15.99) MB	16.05 ± (16.02 - 16.08) MB	+0.5%	✅⬆️
runtime.dotnet.threads.count	20 ± (20 - 20)	20 ± (20 - 20)	-0.5%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	202.57 ± (202.12 - 203.01) ms	188.11 ± (187.76 - 188.45) ms	-7.1%	✅
process.time_to_main_ms	89.15 ± (88.91 - 89.39) ms	82.17 ± (82.00 - 82.33) ms	-7.8%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	15.99 ± (15.97 - 16.01) MB	16.10 ± (16.07 - 16.13) MB	+0.7%	✅⬆️
runtime.dotnet.threads.count	21 ± (20 - 21)	21 ± (20 - 21)	-0.2%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	400.87 ± (398.26 - 403.48) ms	393.96 ± (391.60 - 396.32) ms	-1.7%	✅
process.time_to_main_ms	532.09 ± (530.75 - 533.42) ms	514.34 ± (513.76 - 514.93) ms	-3.3%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	63.40 ± (63.27 - 63.53) MB	63.01 ± (62.85 - 63.16) MB	-0.6%	✅
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 29)	+0.0%	✅⬆️
.NET 6 - Baseline
process.internal_duration_ms	194.11 ± (193.65 - 194.58) ms	191.38 ± (191.07 - 191.69) ms	-1.4%	✅
process.time_to_main_ms	70.60 ± (70.41 - 70.79) ms	69.55 ± (69.43 - 69.67) ms	-1.5%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.35 ± (16.32 - 16.38) MB	16.00 ± (15.85 - 16.15) MB	-2.1%	✅
runtime.dotnet.threads.count	19 ± (19 - 19)	18 ± (18 - 18)	-3.9%	✅
.NET 6 - Bailout
process.internal_duration_ms	194.30 ± (193.83 - 194.77) ms	190.76 ± (190.49 - 191.02) ms	-1.8%	✅
process.time_to_main_ms	72.12 ± (71.95 - 72.29) ms	70.59 ± (70.51 - 70.67) ms	-2.1%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.39 ± (16.32 - 16.45) MB	16.13 ± (15.98 - 16.27) MB	-1.6%	✅
runtime.dotnet.threads.count	20 ± (20 - 20)	19 ± (19 - 19)	-3.2%	✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	420.35 ± (417.48 - 423.21) ms	412.21 ± (408.85 - 415.57) ms	-1.9%	✅
process.time_to_main_ms	498.03 ± (496.84 - 499.21) ms	484.58 ± (484.04 - 485.11) ms	-2.7%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	62.46 ± (62.33 - 62.59) MB	61.94 ± (61.78 - 62.09) MB	-0.8%	✅
runtime.dotnet.threads.count	29 ± (29 - 30)	29 ± (29 - 29)	-0.1%	✅
.NET 8 - Baseline
process.internal_duration_ms	192.85 ± (192.40 - 193.29) ms	189.47 ± (189.16 - 189.79) ms	-1.7%	✅
process.time_to_main_ms	70.22 ± (70.04 - 70.41) ms	69.20 ± (69.05 - 69.35) ms	-1.5%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.67 ± (11.65 - 11.69) MB	11.74 ± (11.72 - 11.77) MB	+0.6%	✅⬆️
runtime.dotnet.threads.count	18 ± (18 - 18)	18 ± (18 - 18)	-0.6%	✅
.NET 8 - Bailout
process.internal_duration_ms	191.73 ± (191.36 - 192.11) ms	189.07 ± (188.81 - 189.34) ms	-1.4%	✅
process.time_to_main_ms	71.47 ± (71.30 - 71.64) ms	70.26 ± (70.17 - 70.36) ms	-1.7%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.74 ± (11.72 - 11.76) MB	11.83 ± (11.80 - 11.86) MB	+0.7%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 19)	19 ± (19 - 19)	-0.5%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	361.21 ± (359.95 - 362.48) ms	359.82 ± (358.36 - 361.27) ms	-0.4%	✅
process.time_to_main_ms	468.46 ± (467.76 - 469.15) ms	459.45 ± (458.84 - 460.05) ms	-1.9%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	50.51 ± (50.48 - 50.54) MB	50.47 ± (50.43 - 50.51) MB	-0.1%	✅
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 29)	+0.1%	✅⬆️

Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts

FakeDbCommand (.NET Framework 4.8)

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7777) - mean (76ms)  : 68, 83
    master - mean (75ms)  : 70, 81

    section Bailout
    This PR (7777) - mean (80ms)  : 75, 85
    master - mean (79ms)  : 76, 83

    section CallTarget+Inlining+NGEN
    This PR (7777) - mean (1,115ms)  : 1021, 1209
    master - mean (1,117ms)  : 1037, 1197

FakeDbCommand (.NET Core 3.1)

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7777) - mean (117ms)  : 109, 124
    master - mean (117ms)  : 112, 122

    section Bailout
    This PR (7777) - mean (120ms)  : 113, 127
    master - mean (119ms)  : 114, 124

    section CallTarget+Inlining+NGEN
    This PR (7777) - mean (815ms)  : 776, 854
    master - mean (806ms)  : 764, 847

FakeDbCommand (.NET 6)

gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7777) - mean (106ms)  : 100, 112
    master - mean (104ms)  : 96, 111

    section Bailout
    This PR (7777) - mean (107ms)  : 101, 113
    master - mean (106ms)  : 99, 112

    section CallTarget+Inlining+NGEN
    This PR (7777) - mean (757ms)  : 717, 797
    master - mean (745ms)  : 714, 777

FakeDbCommand (.NET 8)

gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7777) - mean (105ms)  : 98, 113
    master - mean (103ms)  : 96, 111

    section Bailout
    This PR (7777) - mean (107ms)  : 100, 114
    master - mean (103ms)  : 96, 110

    section CallTarget+Inlining+NGEN
    This PR (7777) - mean (720ms)  : 694, 745
    master - mean (707ms)  : 669, 746

HttpMessageHandler (.NET Framework 4.8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7777) - mean (193ms)  : 189, 197
    master - mean (192ms)  : 189, 195

    section Bailout
    This PR (7777) - mean (196ms)  : 194, 198
    master - mean (195ms)  : 192, 198

    section CallTarget+Inlining+NGEN
    This PR (7777) - mean (1,176ms)  : 1098, 1254
    master - mean (1,203ms)  : 1127, 1280

HttpMessageHandler (.NET Core 3.1)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7777) - mean (277ms)  : 273, 282
    master - mean (297ms)  : 288, 306

    section Bailout
    This PR (7777) - mean (278ms)  : 274, 282
    master - mean (301ms)  : 294, 308

    section CallTarget+Inlining+NGEN
    This PR (7777) - mean (945ms)  : 898, 993
    master - mean (977ms)  : 931, 1023

HttpMessageHandler (.NET 6)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7777) - mean (269ms)  : 265, 274
    master - mean (273ms)  : 267, 279

    section Bailout
    This PR (7777) - mean (269ms)  : 267, 272
    master - mean (275ms)  : 267, 283

    section CallTarget+Inlining+NGEN
    This PR (7777) - mean (927ms)  : 874, 980
    master - mean (958ms)  : 897, 1019

HttpMessageHandler (.NET 8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7777) - mean (269ms)  : 263, 274
    master - mean (273ms)  : 264, 281

    section Bailout
    This PR (7777) - mean (269ms)  : 265, 273
    master - mean (273ms)  : 266, 279

    section CallTarget+Inlining+NGEN
    This PR (7777) - mean (850ms)  : 834, 867
    master - mean (862ms)  : 834, 890

pr-commenter · 2025-11-04T19:21:32Z

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7777 compared to master:

2 benchmarks are faster, with geometric mean 1.162
6 benchmarks have fewer allocations
7 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7777

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472	6.07 KB	6.11 KB	43 B	0.71%
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1	5.67 KB	5.7 KB	37 B	0.65%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	11.1μs	61.4ns	389ns	0	0	0	5.52 KB
master	`StartStopWithChild`	netcoreapp3.1	13.6μs	70.6ns	339ns	0	0	0	5.67 KB
master	`StartStopWithChild`	net472	22.6μs	128ns	914ns	1.06	0.319	0.106	6.07 KB
#7777	`StartStopWithChild`	net6.0	11.1μs	59.4ns	309ns	0	0	0	5.51 KB
#7777	`StartStopWithChild`	netcoreapp3.1	13.8μs	65.7ns	255ns	0	0	0	5.7 KB
#7777	`StartStopWithChild`	net472	22.4μs	116ns	546ns	1.08	0.323	0.108	6.11 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7777

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472	3.35 KB	3.31 KB	-46 B	-1.37%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	941μs	277ns	961ns	2.71 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.03ms	94.7ns	367ns	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	1.21ms	603ns	2.26μs	3.35 KB
#7777	`WriteAndFlushEnrichedTraces`	net6.0	941μs	173ns	669ns	2.7 KB
#7777	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.03ms	319ns	1.24μs	2.7 KB
#7777	`WriteAndFlushEnrichedTraces`	net472	1.22ms	3.44μs	13.3μs	3.31 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`AllCycleSimpleBody`	net6.0	1.09μs	5.65ns	27.1ns	0	1.22 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	1.45μs	7.9ns	46ns	0	1.2 KB
master	`AllCycleSimpleBody`	net472	1.03μs	0.667ns	2.41ns	0.192	1.23 KB
master	`AllCycleMoreComplexBody`	net6.0	7.01μs	34.1ns	145ns	0	4.72 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	9.05μs	46ns	200ns	0	4.62 KB
master	`AllCycleMoreComplexBody`	net472	7.69μs	4.72ns	16.4ns	0.731	4.74 KB
master	`ObjectExtractorSimpleBody`	net6.0	324ns	1.8ns	11.1ns	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	391ns	2.21ns	14.3ns	0	272 B
master	`ObjectExtractorSimpleBody`	net472	301ns	0.108ns	0.405ns	0.044	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	6.27μs	32.1ns	151ns	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.88μs	35ns	136ns	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	6.74μs	3.45ns	13.3ns	0.573	3.8 KB
#7777	`AllCycleSimpleBody`	net6.0	1.1μs	6.37ns	55.5ns	0	1.22 KB
#7777	`AllCycleSimpleBody`	netcoreapp3.1	1.43μs	7.15ns	31.2ns	0	1.2 KB
#7777	`AllCycleSimpleBody`	net472	1.03μs	0.665ns	2.57ns	0.193	1.23 KB
#7777	`AllCycleMoreComplexBody`	net6.0	7.1μs	35.9ns	172ns	0	4.72 KB
#7777	`AllCycleMoreComplexBody`	netcoreapp3.1	8.97μs	44ns	187ns	0	4.62 KB
#7777	`AllCycleMoreComplexBody`	net472	7.69μs	6.65ns	25.7ns	0.729	4.74 KB
#7777	`ObjectExtractorSimpleBody`	net6.0	326ns	0.111ns	0.414ns	0	280 B
#7777	`ObjectExtractorSimpleBody`	netcoreapp3.1	394ns	2.23ns	14.9ns	0	272 B
#7777	`ObjectExtractorSimpleBody`	net472	302ns	0.0873ns	0.338ns	0.0441	281 B
#7777	`ObjectExtractorMoreComplexBody`	net6.0	6.32μs	32.4ns	159ns	0	3.78 KB
#7777	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.91μs	31.9ns	119ns	0	3.69 KB
#7777	`ObjectExtractorMoreComplexBody`	net472	6.71μs	0.957ns	3.71ns	0.57	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EncodeArgs`	net6.0	76.4μs	266ns	997ns	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	96.6μs	157ns	609ns	0	32.4 KB
master	`EncodeArgs`	net472	109μs	21.1ns	81.7ns	4.94	32.51 KB
master	`EncodeLegacyArgs`	net6.0	143μs	208ns	806ns	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	196μs	112ns	388ns	0	2.14 KB
master	`EncodeLegacyArgs`	net472	264μs	47.7ns	185ns	0	2.16 KB
#7777	`EncodeArgs`	net6.0	76μs	241ns	901ns	0	32.4 KB
#7777	`EncodeArgs`	netcoreapp3.1	98.4μs	104ns	401ns	0	32.4 KB
#7777	`EncodeArgs`	net472	109μs	21.3ns	79.5ns	4.92	32.51 KB
#7777	`EncodeLegacyArgs`	net6.0	144μs	124ns	480ns	0	2.15 KB
#7777	`EncodeLegacyArgs`	netcoreapp3.1	196μs	252ns	975ns	0	2.14 KB
#7777	`EncodeLegacyArgs`	net472	262μs	15.9ns	57.2ns	0	2.17 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`RunWafRealisticBenchmark`	net6.0	430μs	1.17μs	4.55μs	4.56 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	411μs	169ns	609ns	4.48 KB
master	`RunWafRealisticBenchmark`	net472	457μs	48.3ns	187ns	4.66 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	309μs	74ns	277ns	2.24 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	296μs	109ns	421ns	2.22 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	335μs	40.7ns	158ns	2.29 KB
#7777	`RunWafRealisticBenchmark`	net6.0	401μs	152ns	588ns	4.55 KB
#7777	`RunWafRealisticBenchmark`	netcoreapp3.1	410μs	68ns	245ns	4.48 KB
#7777	`RunWafRealisticBenchmark`	net472	429μs	71.2ns	266ns	4.66 KB
#7777	`RunWafRealisticBenchmarkWithAttack`	net6.0	288μs	44ns	165ns	2.24 KB
#7777	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	297μs	78.7ns	305ns	2.22 KB
#7777	`RunWafRealisticBenchmarkWithAttack`	net472	313μs	37.6ns	146ns	2.29 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`SendRequest`	net6.0	60.7μs	62.6ns	234ns	14.52 KB
master	`SendRequest`	netcoreapp3.1	71μs	118ns	440ns	17.42 KB
master	`SendRequest`	net472	0.00285ns	0.000927ns	0.00359ns	0 b
#7777	`SendRequest`	net6.0	61.3μs	84.8ns	317ns	14.52 KB
#7777	`SendRequest`	netcoreapp3.1	71.9μs	114ns	396ns	17.42 KB
#7777	`SendRequest`	net472	0.00559ns	0.00219ns	0.00849ns	0 b

Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7777

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0	3 B	4 B	1 B	33.33%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0	6 B	7 B	1 B	16.67%

Fewer allocations 🎉 in #7777

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑netcoreapp3.1	1 B	0 b	-1 B	-100.00%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`OriginalCharSlice`	net6.0	1.95ms	249ns	962ns	0	640.01 KB
master	`OriginalCharSlice`	netcoreapp3.1	2.07ms	6.45μs	25μs	0	640 KB
master	`OriginalCharSlice`	net472	2.74ms	241ns	931ns	100	641.95 KB
master	`OptimizedCharSlice`	net6.0	1.39ms	651ns	2.52μs	0	6 B
master	`OptimizedCharSlice`	netcoreapp3.1	1.64ms	381ns	1.47μs	0	1 B
master	`OptimizedCharSlice`	net472	1.97ms	248ns	960ns	0	73 B
master	`OptimizedCharSliceWithPool`	net6.0	800μs	31.8ns	123ns	0	3 B
master	`OptimizedCharSliceWithPool`	netcoreapp3.1	852μs	34.9ns	135ns	0	1 B
master	`OptimizedCharSliceWithPool`	net472	1.14ms	73ns	273ns	0	47 B
#7777	`OriginalCharSlice`	net6.0	1.92ms	350ns	1.36μs	0	640.01 KB
#7777	`OriginalCharSlice`	netcoreapp3.1	2.08ms	10.4μs	44.3μs	0	640 KB
#7777	`OriginalCharSlice`	net472	2.65ms	211ns	791ns	100	641.95 KB
#7777	`OptimizedCharSlice`	net6.0	1.47ms	76.4ns	296ns	0	7 B
#7777	`OptimizedCharSlice`	netcoreapp3.1	1.7ms	354ns	1.37μs	0	1 B
#7777	`OptimizedCharSlice`	net472	2ms	364ns	1.41μs	0	73 B
#7777	`OptimizedCharSliceWithPool`	net6.0	858μs	48.7ns	189ns	0	4 B
#7777	`OptimizedCharSliceWithPool`	netcoreapp3.1	805μs	86.3ns	334ns	0	0 b
#7777	`OptimizedCharSliceWithPool`	net472	1.13ms	59.1ns	221ns	0	47 B

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #7777

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	1.183	729,000.00	616,322.71

More allocations ⚠️ in #7777

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	41.75 KB	41.97 KB	222 B	0.53%

Fewer allocations 🎉 in #7777

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	42.14 KB	41.74 KB	-400 B	-0.95%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	727μs	3.77μs	17.3μs	0	42.14 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	718μs	4.17μs	34.6μs	0	41.75 KB
master	`WriteAndFlushEnrichedTraces`	net472	892μs	3.83μs	14.8μs	4.46	55.96 KB
#7777	`WriteAndFlushEnrichedTraces`	net6.0	659μs	401ns	1.39μs	0	41.74 KB
#7777	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	616μs	626ns	2.42μs	0	41.97 KB
#7777	`WriteAndFlushEnrichedTraces`	net472	927μs	3.75μs	14.5μs	4.46	55.78 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.88μs	1.54ns	5.97ns	0	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	2.62μs	12.1ns	46.8ns	0	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.87μs	4.85ns	17.5ns	0.144	0.0144	987 B
#7777	`ExecuteNonQuery`	net6.0	1.9μs	1.19ns	4.13ns	0	0	1.02 KB
#7777	`ExecuteNonQuery`	netcoreapp3.1	2.66μs	7.88ns	29.5ns	0	0	1.02 KB
#7777	`ExecuteNonQuery`	net472	2.89μs	4.22ns	16.3ns	0.156	0.0142	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.69μs	8.99ns	44.9ns	0	1.03 KB
master	`CallElasticsearch`	netcoreapp3.1	2.21μs	11.2ns	51.5ns	0	1.03 KB
master	`CallElasticsearch`	net472	3.59μs	1.43ns	5.36ns	0.162	1.04 KB
master	`CallElasticsearchAsync`	net6.0	1.8μs	0.962ns	3.6ns	0	1.01 KB
master	`CallElasticsearchAsync`	netcoreapp3.1	2.33μs	11.3ns	46.6ns	0	1.08 KB
master	`CallElasticsearchAsync`	net472	3.63μs	1.7ns	6.57ns	0.163	1.1 KB
#7777	`CallElasticsearch`	net6.0	1.77μs	8.6ns	35.4ns	0	1.03 KB
#7777	`CallElasticsearch`	netcoreapp3.1	2.25μs	11ns	45.3ns	0	1.03 KB
#7777	`CallElasticsearch`	net472	3.52μs	2.26ns	8.46ns	0.158	1.04 KB
#7777	`CallElasticsearchAsync`	net6.0	1.84μs	8.74ns	36ns	0	1.01 KB
#7777	`CallElasticsearchAsync`	netcoreapp3.1	2.38μs	11.3ns	46.6ns	0	1.08 KB
#7777	`CallElasticsearchAsync`	net472	3.63μs	3.34ns	12.9ns	0.163	1.1 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.97μs	1.77ns	6.84ns	0	952 B
master	`ExecuteAsync`	netcoreapp3.1	2.44μs	9.56ns	37ns	0	952 B
master	`ExecuteAsync`	net472	2.68μs	7.96ns	30.8ns	0.134	915 B
#7777	`ExecuteAsync`	net6.0	1.88μs	7.64ns	29.6ns	0	952 B
#7777	`ExecuteAsync`	netcoreapp3.1	2.42μs	5.2ns	19.5ns	0	952 B
#7777	`ExecuteAsync`	net472	2.68μs	4.26ns	16.5ns	0.134	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	7.06μs	19.8ns	76.7ns	0	2.36 KB
master	`SendAsync`	netcoreapp3.1	8.87μs	24.7ns	92.4ns	0	2.9 KB
master	`SendAsync`	net472	12.6μs	10.7ns	41.5ns	0.504	3.18 KB
#7777	`SendAsync`	net6.0	6.76μs	14.9ns	57.9ns	0	2.36 KB
#7777	`SendAsync`	netcoreapp3.1	8.56μs	8.03ns	29ns	0	2.9 KB
#7777	`SendAsync`	net472	12.5μs	10.7ns	41.5ns	0.498	3.18 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #7777

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0	1.141	47,300.00	41,450.00

More allocations ⚠️ in #7777

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	256.27 KB	273.58 KB	17.31 KB	6.76%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	260.24 KB	277.32 KB	17.08 KB	6.56%

Fewer allocations 🎉 in #7777

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1	43.14 KB	42.73 KB	-416 B	-0.96%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472	286.72 KB	278.53 KB	-8.19 KB	-2.86%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0	45.34 KB	43.44 KB	-1.9 KB	-4.20%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	48.1μs	306ns	2.95μs	45.34 KB
master	`StringConcatBenchmark`	netcoreapp3.1	50.9μs	391ns	3.71μs	43.14 KB
master	`StringConcatBenchmark`	net472	58.3μs	117ns	438ns	57.34 KB
master	`StringConcatAspectBenchmark`	net6.0	458μs	1.33μs	4.78μs	260.24 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	508μs	2.28μs	8.83μs	256.27 KB
master	`StringConcatAspectBenchmark`	net472	399μs	1.7μs	6.58μs	286.72 KB
#7777	`StringConcatBenchmark`	net6.0	41.6μs	230ns	1.34μs	43.44 KB
#7777	`StringConcatBenchmark`	netcoreapp3.1	49.8μs	278ns	2.32μs	42.73 KB
#7777	`StringConcatBenchmark`	net472	57.4μs	186ns	696ns	57.34 KB
#7777	`StringConcatAspectBenchmark`	net6.0	469μs	1.79μs	6.44μs	277.32 KB
#7777	`StringConcatAspectBenchmark`	netcoreapp3.1	516μs	2.28μs	8.55μs	273.58 KB
#7777	`StringConcatAspectBenchmark`	net472	407μs	2.21μs	17.2μs	278.53 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.65μs	0.571ns	2.21ns	0	1.7 KB
master	`EnrichedLog`	netcoreapp3.1	3.65μs	14.5ns	54.2ns	0	1.7 KB
master	`EnrichedLog`	net472	3.81μs	3.36ns	13ns	0.248	1.64 KB
#7777	`EnrichedLog`	net6.0	2.69μs	13.1ns	58.5ns	0	1.7 KB
#7777	`EnrichedLog`	netcoreapp3.1	3.6μs	14.6ns	56.5ns	0	1.7 KB
#7777	`EnrichedLog`	net472	4.13μs	8.34ns	32.3ns	0.248	1.64 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`EnrichedLog`	net6.0	128μs	606ns	2.5μs	4.31 KB
master	`EnrichedLog`	netcoreapp3.1	132μs	463ns	1.79μs	4.31 KB
master	`EnrichedLog`	net472	175μs	444ns	1.72μs	4.51 KB
#7777	`EnrichedLog`	net6.0	125μs	42.5ns	147ns	4.32 KB
#7777	`EnrichedLog`	netcoreapp3.1	130μs	148ns	552ns	4.31 KB
#7777	`EnrichedLog`	net472	170μs	166ns	643ns	4.52 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.99μs	6.8ns	26.3ns	0	2.26 KB
master	`EnrichedLog`	netcoreapp3.1	6.87μs	8.36ns	32.4ns	0	2.26 KB
master	`EnrichedLog`	net472	7.63μs	5.06ns	18.9ns	0.304	2.08 KB
#7777	`EnrichedLog`	net6.0	4.93μs	5.02ns	17.4ns	0	2.26 KB
#7777	`EnrichedLog`	netcoreapp3.1	6.93μs	15.4ns	59.7ns	0	2.26 KB
#7777	`EnrichedLog`	net472	7.66μs	8.86ns	34.3ns	0.305	2.08 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	2.06μs	6.65ns	25.8ns	0	1.2 KB
master	`SendReceive`	netcoreapp3.1	2.63μs	12.5ns	48.5ns	0	1.2 KB
master	`SendReceive`	net472	3.03μs	6.35ns	24.6ns	0.18	1.2 KB
#7777	`SendReceive`	net6.0	1.93μs	9.97ns	49.9ns	0	1.2 KB
#7777	`SendReceive`	netcoreapp3.1	2.53μs	0.859ns	3.21ns	0	1.2 KB
#7777	`SendReceive`	net472	3.12μs	4.24ns	15.9ns	0.186	1.2 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.27μs	15.3ns	57.1ns	0	1.58 KB
master	`EnrichedLog`	netcoreapp3.1	5.83μs	14.6ns	56.4ns	0	1.63 KB
master	`EnrichedLog`	net472	6.63μs	7.92ns	30.7ns	0.297	2.03 KB
#7777	`EnrichedLog`	net6.0	4.1μs	16ns	61.8ns	0	1.58 KB
#7777	`EnrichedLog`	netcoreapp3.1	5.54μs	7.88ns	30.5ns	0	1.63 KB
#7777	`EnrichedLog`	net472	6.88μs	8.98ns	34.8ns	0.307	2.03 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	772ns	3.4ns	12.7ns	0	576 B
master	`StartFinishSpan`	netcoreapp3.1	998ns	4.54ns	18.2ns	0	576 B
master	`StartFinishSpan`	net472	958ns	0.106ns	0.396ns	0.0911	578 B
master	`StartFinishScope`	net6.0	953ns	1.49ns	5.79ns	0	696 B
master	`StartFinishScope`	netcoreapp3.1	1.22μs	0.707ns	2.74ns	0	696 B
master	`StartFinishScope`	net472	1.17μs	2.6ns	10.1ns	0.1	658 B
#7777	`StartFinishSpan`	net6.0	784ns	4.23ns	22.4ns	0	576 B
#7777	`StartFinishSpan`	netcoreapp3.1	963ns	2.96ns	11.5ns	0	576 B
#7777	`StartFinishSpan`	net472	963ns	0.21ns	0.787ns	0.087	578 B
#7777	`StartFinishScope`	net6.0	931ns	4.65ns	19.2ns	0	696 B
#7777	`StartFinishScope`	netcoreapp3.1	1.19μs	6.49ns	36.7ns	0	696 B
#7777	`StartFinishScope`	net472	1.15μs	0.237ns	0.917ns	0.103	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	1.07μs	5.12ns	21.7ns	0	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	1.4μs	0.489ns	1.9ns	0	696 B
master	`RunOnMethodBegin`	net472	1.44μs	0.501ns	1.87ns	0.101	658 B
#7777	`RunOnMethodBegin`	net6.0	1.06μs	0.351ns	1.31ns	0	696 B
#7777	`RunOnMethodBegin`	netcoreapp3.1	1.43μs	5.77ns	21.6ns	0	696 B
#7777	`RunOnMethodBegin`	net472	1.52μs	0.852ns	3.3ns	0.0981	658 B

bouwkast

LGTM seems like it should be a lot less allocations now.

Were you able to reproduce this or benchmark at all or is this just an educated guess?

bouwkast · 2025-11-05T20:07:21Z

tracer/src/Datadog.Trace/DataStreamsMonitoring/DataStreamsWriter.cs

-            var tcs = new TaskCompletionSource<bool>();
-            using (new Timer(s => ((TaskCompletionSource<bool>)s!).SetResult(true), tcs, _waitTimeSpan, Timeout.InfiniteTimeSpan))
-            {
-                await Task.WhenAny(_processExit.Task, tcs.Task).ConfigureAwait(false);


We aren't responding / reacting to _processExit.Task anymore but I don't think a big deal

We aren't responding / reacting to _processExit.Task anymore but I don't think a big deal

Yeah, given it's only a 15ms wait, I think that's fine, but it does make me wonder... a 15ms wait period makes this is a pretty fast spinning loop, no? 😕For context, we flush traces every 1000ms (1s)... On low-core (1) machines would this not potentially be taking up a lot of thread/CPU time?

What's more, the _resetEvent.Wait() that's never triggered is effectively a Thread.Sleep(). And doing a Thread.Sleep on a thread pool thread is generally not a good idea, as it blocks anything else doing work. @tonyredondo did a bunch of work to remove all the Thread.Sleep() from our async code for exactly this reason. If we do want to dedicate a thread to this, I think we need to turn this loop into a sync loop and create a dedicated long-running thread, but that might be more work than we'd like.

Overall, I don't know if we have any evidence that the TCS and Tasks here are the source of the perf issue yet? and I wonder if it's the potentially small value of _waitTimeSpan that's causing the issue due to this loop spinning too fast? 🤔 Have we considered alternatives like increasing the buffering wait time? Or taking an approach similar to the agent writer where we have a longer buffering period, with dedicated flushing when the buffers get to a certain "fullness"? It'll be a trade-off obviously, but I'm not sure which is best

andrewlock · 2025-11-06T08:23:00Z

tracer/src/Datadog.Trace/DataStreamsMonitoring/DataStreamsWriter.cs

-            var tcs = new TaskCompletionSource<bool>();
-            using (new Timer(s => ((TaskCompletionSource<bool>)s!).SetResult(true), tcs, _waitTimeSpan, Timeout.InfiniteTimeSpan))
-            {
-                await Task.WhenAny(_processExit.Task, tcs.Task).ConfigureAwait(false);


We aren't responding / reacting to _processExit.Task anymore but I don't think a big deal

Yeah, given it's only a 15ms wait, I think that's fine, but it does make me wonder... a 15ms wait period makes this is a pretty fast spinning loop, no? 😕For context, we flush traces every 1000ms (1s)... On low-core (1) machines would this not potentially be taking up a lot of thread/CPU time?

What's more, the _resetEvent.Wait() that's never triggered is effectively a Thread.Sleep(). And doing a Thread.Sleep on a thread pool thread is generally not a good idea, as it blocks anything else doing work. @tonyredondo did a bunch of work to remove all the Thread.Sleep() from our async code for exactly this reason. If we do want to dedicate a thread to this, I think we need to turn this loop into a sync loop and create a dedicated long-running thread, but that might be more work than we'd like.

Overall, I don't know if we have any evidence that the TCS and Tasks here are the source of the perf issue yet? and I wonder if it's the potentially small value of _waitTimeSpan that's causing the issue due to this loop spinning too fast? 🤔 Have we considered alternatives like increasing the buffering wait time? Or taking an approach similar to the agent writer where we have a longer buffering period, with dedicated flushing when the buffers get to a certain "fullness"? It'll be a trade-off obviously, but I'm not sure which is best

kr-igor · 2025-11-06T15:41:22Z

Overall, I don't know if we have any evidence that the TCS and Tasks here are the source of the perf issue yet?

@andrewlock From the .nettrace it seems like a bunch of OS resources get created / destroyed in this loop. I was not able to reproduce this in any of my setups. Tried various environments / thread pool configurations. Maybe this is related to the underlying OS architecture and differences in actual thread scheduling in low resource scenarios.

My guess is that it comes from this code. Tasks wait due to thread pool starvation, causing ManualResetEventSlim to allocate a wait handle here.

I didn't know that _resetEvent.wait is equivalent of thread.sleep, my assumption was that is uses some Spin logic under the hood. I'll dig a bit more and will try to find a better way.

andrewlock · 2025-11-06T16:24:50Z

I didn't know that _resetEvent.wait is equivalent of thread.sleep

To be clear, I haven't looked into what it's actually like, but it's fundamentally not releasing the thread which is what I was getting at 🙂

Avoid creating new tasks on wait

a884295

github-actions bot added area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) area:data-streams-monitoring labels Nov 4, 2025

kr-igor and others added 5 commits November 4, 2025 13:21

Use manual reset event

5892de6

Cleanup

464897e

Merge branch 'master' into kr-igor/apms-17551

2dc59ab

Temprary revert for testing

077b880

Final update

8389632

kr-igor marked this pull request as ready for review November 5, 2025 18:20

kr-igor requested review from a team as code owners November 5, 2025 18:20

bouwkast reviewed Nov 5, 2025

View reviewed changes

bouwkast approved these changes Nov 5, 2025

View reviewed changes

andrewlock requested changes Nov 6, 2025

View reviewed changes

Use a dedicate thread for flushing DSM data

021e1e6

This comment has been minimized.

Sign in to view

Updated shutdown logic

d5a3564

kr-igor requested a review from a team as a code owner November 7, 2025 20:18

kr-igor and others added 6 commits November 7, 2025 15:22

Fixed background loop

2b8cc00

Minor fixes

1ae4166

Merge branch 'master' into kr-igor/apms-17551

4db5067

Simplified final flush logic

96cb743

Mark the thread as background

a7365f2

Increased sleep interval

e0ce96d

Avoid creating new tasks on wait #7777

Are you sure you want to change the base?

Avoid creating new tasks on wait #7777

Conversation

kr-igor commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Uh oh!

dd-trace-dotnet-ci-bot bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

FakeDbCommand

HttpMessageHandler

Uh oh!

pr-commenter bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmark details

More allocations ⚠️ in #7777

Raw results

Fewer allocations 🎉 in #7777

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #7777

Fewer allocations 🎉 in #7777

Raw results

Faster 🎉 in #7777

More allocations ⚠️ in #7777

Fewer allocations 🎉 in #7777

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #7777

More allocations ⚠️ in #7777

Fewer allocations 🎉 in #7777

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Uh oh!

bouwkast left a comment

Choose a reason for hiding this comment

Uh oh!

bouwkast Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

andrewlock Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

andrewlock Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

kr-igor commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrewlock commented Nov 6, 2025

Uh oh!

This comment has been minimized.

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kr-igor commented Nov 4, 2025 •

edited

Loading

dd-trace-dotnet-ci-bot bot commented Nov 4, 2025 •

edited

Loading

pr-commenter bot commented Nov 4, 2025 •

edited

Loading

kr-igor commented Nov 6, 2025 •

edited

Loading