Fix ad-hoc use of `TracerSettings` to use `MutableSettings` where appropriate #7543

andrewlock · 2025-09-22T14:27:08Z

Summary of changes

Fix usages of Tracer.Instance.Settings to use Tracer.Instance.CurrentTracerSettings.Settings where appropriate

Reason for change

This PR "fixes" the places that were previously grabbing the environment/service etc from TracerSettings to use the MutableSettings exposed via CurrentTracerSettings instead, as the location where these settings will ultimately exist.

This is effectively still just a refactoring, but prepares for the point where these settings aren't exposed on TracerSettings at all. The updates in this PR are for cases where you don't have long-lived services, and rather need to do ad-hoc Tracer.Instance grabbing of the setting values in a global context. Note too that many of these places could be updated in the future to subscribe to changes if that provides performance benefits. Also note that I elected not to change most calls to IsIntegrationEnabled() etc in this PR as there are hundreds of locations. The follow up PR handles that

Also found a few cases that were incorrectly assuming that these values cannot change. Marked them with 'TODO: Subscribe to changes in settings'

Implementation details

Mostly find and replace to use CurrentTracerSettings.Settings
Occasional extraction of a variable where it makes sense to avoid repeated access
Functionally identical currently (where MutableSettings is replaced on TracerSettings) but will be a required change once we stop replacing TracerManager.

Test coverage

Covered by existing tests

Other details

https://datadoghq.atlassian.net/browse/LANGPLAT-819

Part of a config stack

## Summary of changes - Extract a `MutableSettings` type from `TracerSettings` - Extract a `Raw` type from `ExporterSettings` ## Reason for change We're working on refactoring how we handle dynamic/remote/config in code settings i.e. settings which can change at runtime. As a first step, this PR extracts those settings to their own type, called `MutableSettings` because they mutable during the lifetime of the app. > Feel free to suggest other names for this type, or we can alternatively bikeshed it later. Additionally, extracted the "raw" settings from `ExporterSettings`. These are the values which are read from config sources. The actual values of `ExporterSettings` are set based on these values, using a highly convoluted (backward compatible) series of methods, but the idea is: if the `Raw` settings haven't changed, the `ExporterSettings` haven't changed. > This isn't _strictly_ true due to the `File.Exists` call we have, but I believe this is good enough for our purposes. ## Implementation details - Create `MutableSettings` and move all the properties from `TracerSettings` that _can_ change to it. - Update `TracerSettings` to create an instance of `MutableSettings`, and simply pass-through properties to it. - This should mean existing functionality is unaffected by this PR - Implement `IEquatable<MutableSettings>` for future comparisons between `MutableSettings` instances - Unfortunately, can't use a `record` here or auto-gen the implementation, because we need to handle equivalence of the dictionaries. - Extract the "raw" setting reading to an `ExporterSettings` nested-type - Opted for nested here, because unlike `MutableSettings` (which will eventually live separately from `TracerSettings`) we won't expose `Raw` to consumers - they'll still use `ExporterSettings` ## Test coverage This is just a refactoring, so it's covered by existing tests. Additionally, I added a test for the `IEquatable` implementation (which is similar to the test we have for `ImmutableDynamicSettings`) to ensure the implementation is updated if we add more properties. ## Other details https://datadoghq.atlassian.net/browse/LANGPLAT-819 Part of a config stack - #7522 👈 - #7525 - #7530 - #7532 - #7543 - #7544

dd-trace-dotnet-ci-bot · 2025-09-23T16:54:50Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7543) - mean (72ms)  : 71, 73
     .   : milestone, 72,
    master - mean (71ms)  : 71, 72
     .   : milestone, 71,

    section Baseline
    This PR (7543) - mean (68ms)  : 65, 72
     .   : milestone, 68,
    master - mean (68ms)  : 66, 70
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7543) - mean (1,052ms)  : 986, 1118
     .   : milestone, 1052,
    master - mean (1,048ms)  : 1003, 1093
     .   : milestone, 1048,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7543) - mean (106ms)  : 105, 107
     .   : milestone, 106,
    master - mean (106ms)  : 104, 107
     .   : milestone, 106,

    section Baseline
    This PR (7543) - mean (105ms)  : 102, 107
     .   : milestone, 105,
    master - mean (105ms)  : 102, 108
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (7543) - mean (738ms)  : 711, 766
     .   : milestone, 738,
    master - mean (744ms)  : 713, 774
     .   : milestone, 744,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7543) - mean (101ms)  : 99, 102
     .   : milestone, 101,
    master - mean (101ms)  : 100, 102
     .   : milestone, 101,

    section Baseline
    This PR (7543) - mean (100ms)  : 97, 102
     .   : milestone, 100,
    master - mean (99ms)  : 97, 101
     .   : milestone, 99,

    section CallTarget+Inlining+NGEN
    This PR (7543) - mean (774ms)  : 733, 814
     .   : milestone, 774,
    master - mean (781ms)  : 747, 814
     .   : milestone, 781,

gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7543) - mean (93ms)  : 91, 94
     .   : milestone, 93,
    master - mean (92ms)  : 91, 94
     .   : milestone, 92,

    section Baseline
    This PR (7543) - mean (92ms)  : 89, 95
     .   : milestone, 92,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (7543) - mean (657ms)  : 643, 671
     .   : milestone, 657,
    master - mean (660ms)  : 650, 670
     .   : milestone, 660,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7543) - mean (196ms)  : 193, 198
     .   : milestone, 196,
    master - mean (195ms)  : 193, 198
     .   : milestone, 195,

    section Baseline
    This PR (7543) - mean (192ms)  : 189, 196
     .   : milestone, 192,
    master - mean (193ms)  : 188, 197
     .   : milestone, 193,

    section CallTarget+Inlining+NGEN
    This PR (7543) - mean (1,183ms)  : 1097, 1268
     .   : milestone, 1183,
    master - mean (1,172ms)  : 1106, 1239
     .   : milestone, 1172,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7543) - mean (278ms)  : 273, 283
     .   : milestone, 278,
    master - mean (277ms)  : 273, 280
     .   : milestone, 277,

    section Baseline
    This PR (7543) - mean (277ms)  : 271, 282
     .   : milestone, 277,
    master - mean (276ms)  : 271, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (7543) - mean (942ms)  : 904, 981
     .   : milestone, 942,
    master - mean (946ms)  : 906, 986
     .   : milestone, 946,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7543) - mean (281ms)  : 274, 287
     .   : milestone, 281,
    master - mean (279ms)  : 275, 283
     .   : milestone, 279,

    section Baseline
    This PR (7543) - mean (280ms)  : 275, 285
     .   : milestone, 280,
    master - mean (279ms)  : 274, 284
     .   : milestone, 279,

    section CallTarget+Inlining+NGEN
    This PR (7543) - mean (996ms)  : 965, 1027
     .   : milestone, 996,
    master - mean (997ms)  : 965, 1030
     .   : milestone, 997,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7543) - mean (268ms)  : 264, 272
     .   : milestone, 268,
    master - mean (269ms)  : 265, 274
     .   : milestone, 269,

    section Baseline
    This PR (7543) - mean (267ms)  : 263, 272
     .   : milestone, 267,
    master - mean (269ms)  : 265, 273
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7543) - mean (851ms)  : 828, 873
     .   : milestone, 851,
    master - mean (854ms)  : 837, 871
     .   : milestone, 854,

bouwkast

There's a lot of build errors here in the tests
But I think just a function change

bouwkast · 2025-09-23T19:07:57Z

...rofiler/AutoInstrumentation/Logging/Log4Net/LogsInjection/AppenderAttachedImplIntegration.cs

-                loggingEvent.Properties[CorrelationIdentifier.ServiceKey] = tracer.DefaultServiceName ?? string.Empty;
-                loggingEvent.Properties[CorrelationIdentifier.VersionKey] = tracer.Settings.ServiceVersion ?? string.Empty;
-                loggingEvent.Properties[CorrelationIdentifier.EnvKey] = tracer.Settings.Environment ?? string.Empty;
+                loggingEvent.Properties[CorrelationIdentifier.ServiceKey] = mutableSettings.DefaultServiceName;


Are we omitting ?? string.Empty because it is never null?

Yup, exactly that

bouwkast · 2025-09-23T19:14:37Z

....Trace/ClrProfiler/AutoInstrumentation/Logging/NLog/LogsInjection/DiagnosticContextHelper.cs

-                    new KeyValuePair<string, object>(CorrelationIdentifier.ServiceKey, tracer.DefaultServiceName ?? string.Empty),
-                    new KeyValuePair<string, object>(CorrelationIdentifier.VersionKey, tracer.Settings.ServiceVersion ?? string.Empty),
-                    new KeyValuePair<string, object>(CorrelationIdentifier.EnvKey, tracer.Settings.Environment ?? string.Empty),
+                    new KeyValuePair<string, object>(CorrelationIdentifier.ServiceKey, mutableSettings.DefaultServiceName),


copy/paste comment
Are we omitting ?? string.Empty because it is never null?

Same below too
It must be never null 😅

bouwkast · 2025-09-23T19:16:17Z

tracer/src/Datadog.Trace/Configuration/GitMetadataTagsProvider.cs

+    public GitMetadataTagsProvider(TracerSettings immutableTracerSettings, MutableSettings settings, IScopeManager scopeManager, ITelemetryController telemetry)
    {
        _immutableTracerSettings = immutableTracerSettings;
+        // These never change, even though they are exposed on MutableSettings, so we can safely grab them once here


What if they do change (or is the "never" a "can't")? Will that break anything here if they change?

The "never" here is a "can't" because there's no mechanism for them to change. So why are they in the mutable settings, you ask? Well...

DD_TAGS can be used to set some other env vars. That includes service, env, and also these git tags

We strip those values from the tags initially, and set them on the "target" properties

DD_TAGS can also be set in code and via remote config, therefore it needs to live on MutableSettings

However, setting the other properties via DD_TAGS is not permitted in code or via remote config (we strip the disallowed values)

Therefore, git commit sha etc never change - they can be set either as tags directly, or using DD_TAGS, or via other static sources - but they cannot ever actually change

Hope that makes sense, it's kind of annoying they have to live there 😅

## Summary of changes Don't pass a `Tracer` instance to `PerTraceSettings.GetServiceName()` (as it's not required) ## Reason for change Cleanup/simplify the code. #7530 exposes `MutableSettings` and `DefaultServiceName` on `PerTraceSettings`. Reading `DefaultServiceName` was the only reason `GetServiceName()` took a `Tracer`, so this is now uneccessary. ## Implementation details - Use `MutableSettings.DefaultServiceName` directly inside `PerTraceSettings.GetServiceName()` - Stop passing in `Tracer` instance - Cleanup usages ## Test coverage Covered by existing, just a refactoring ## Other details Could have done it as part of #7530 but trying to keep things small! https://datadoghq.atlassian.net/browse/LANGPLAT-819 Part of a config stack - #7522 - #7525 - #7530 - #7532 👈 - #7543 - #7544

This "fixes" the places that were previously grabbing the environment/service etc from TracerSettings to use CurrentTracerSettings instead. This is effectively still just a refactoring, but prepares for the point where these settings aren't replaced on TracerSettings. The updates in this PR are for cases where you don't have long-lived services, and rather need to do ad-hoc Tracer.Instance grabbing of the setting values in a global context. Note too that many of these places can be updated in the future to subscribe to changes if that provides performance benefits. Also note that I elected not to change must calls to `IsIntegrationEnabled` in this PR as there are hundreds of locations, Also found a few cases that were incorrectly assuming that these values cannot change. Marked them with 'TODO: Subscribe to changes in settings'

NachoEchevarria

LGTM. Thanks!

…tIntegrationAnalyticsSampleRate()` (#7544) ## Summary of changes Fix usages of `IsIntegrationEnabled()`, `IsErrorStatusCode()`, and `GetIntegrationAnalyticsSampleRate()` to use `MutableSettings` instead of `TracerSettings` ## Reason for change These functions are dependent on `MutableSettings`, and are exposed there, so making sure we call the methods there, and remove the delegation from `TracerSettings` entirely. ## Implementation details - Find and replace usages - Remove the old delegating methods ## Test coverage Just a refactor, so covered by existing tests ## Other details https://datadoghq.atlassian.net/browse/LANGPLAT-819 Part of a config stack - #7522 - #7525 - #7530 - #7532 - #7543 - #7544 👈 - #7695

…ndows (#7721) ## Summary of changes Enforces that you can't _change_ the `AgentUri` to be a UDS Uri if you're on Windows ## Reason for change The trace exporter doesn't work with UDS on Windows, so we have a check in `TracerSettings` that disables the pipeline if we find this scenario. However, user's can still _change_ the agent URI at runtime in code (😭). We currently assume that the data pipeline won't be toggled at runtime (we _do_ allow for reconfiguring it in general, but not for completely removing or reintroducing). Changing this to allow the scenario would be a pain, so instead this PR blocks you from setting a UDS URI in code if you're on Windows. The good news is that as far as I can tell, noone does this today, so while _technically_ it could be considered a breaking change, I think it's ok. ## Implementation details - Throw an `ArgumentException` in the Datadog.Trace.Manual library, if you're on Windows (or .NET FX) and you try to set a UDS agent URI (using the same "detection" we do in `ExporterSettings`. - Add a check in the Instrumentation of `Tracer.Configure()` to make sure it hasn't slipped through. This could happen if a customer was using an old version of the Datadog.Trace NuGet package with a newer version of auto instrumentation. Note that this adds two additional framework references for .NET Core 3.1+, to check if we're on Windows. ## Test coverage Added an extra step to the manual instrumentation integration test to confirm we throw ## Other details https://datadoghq.atlassian.net/browse/LANGPLAT-819 Part of a config stack - #7522 - #7525 - #7530 - #7532 - #7543 - #7544 - #7721 👈 - #7722 - #7695 - #7723 - #7724

## Summary of changes - Fix ASM benchmarks by not returning `null` from `EmptyDatadogTracer.PerTraceSettings` - Stop re-initializing `TracerSettings` for every execution of the ASM benchmark ## Reason for change The ASM benchmarks have been broken since #7543 (I think). This is because the `EmptyDatadogTracer` stub used in the benchmark returns `null` from `PerTraceSettings` (which can't happen in practice). Additionally, noticed that the benchmark is repeatedly creating a new `TracerSettings` object with every execution, which will add noise and be much more expensive than real life. ## Implementation details - Ensure `EmptyDatadogTracer.PerTraceSettings` returns a "real" value - Stop rebuilding `TracerSettings` with every execution ## Test coverage This is the test

## Summary of changes Add a helper for comparing `ReadOnlyDictionary<>` instances ## Reason for change As part of the config work, we need to detect if tags have changed when customers do a manual/remote config update. This helper makes it easy ## Implementation details Added a `SequenceEqual` extension method. Note that I used `SequenceEqual` because it _already_ exists in _System.Linq_, but I could see an argument that it's too easy to use the wrong one, and instead we could use a different name? `IsSameAs(other)`? Also, I only wrote this for `ReadOnlyDictionary<>` because that's all we need, it's what we use for all our setting dictionaries, and it will be (a tiny bit) faster than making it `IDictionary<>`, but happy to change if people feel strongly. ## Test coverage Added unit tests ## Other details https://datadoghq.atlassian.net/browse/LANGPLAT-819 Part of a config stack - #7522 - #7525 - #7530 - #7532 - #7543 - #7544 - #7721 - #7722 👈 - #7695 - #7723 - #7724 --------- Co-authored-by: Steven Bouwkamp <[email protected]>

andrewlock added area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) type:refactor labels Sep 22, 2025

This comment has been minimized.

Sign in to view

andrewlock force-pushed the andrew/settings/4a-fix-adhoc-accessing branch from d582594 to 5c72ac5 Compare September 23, 2025 15:03

andrewlock force-pushed the andrew/settings/3d-simplify-per-trace-settings branch from 83b1b8c to a2951da Compare September 23, 2025 15:03

andrewlock marked this pull request as ready for review September 23, 2025 15:04

andrewlock requested review from a team as code owners September 23, 2025 15:04

andrewlock requested review from link04 and removed request for a team September 23, 2025 15:04

bouwkast reviewed Sep 23, 2025

View reviewed changes

andrewlock force-pushed the andrew/settings/4a-fix-adhoc-accessing branch from 5c72ac5 to f9243ce Compare September 24, 2025 07:20

andrewlock requested a review from a team as a code owner September 24, 2025 07:20

andrewlock force-pushed the andrew/settings/3d-simplify-per-trace-settings branch from a2951da to df2544b Compare September 24, 2025 07:20

andrewlock force-pushed the andrew/settings/4a-fix-adhoc-accessing branch from af21397 to 364c696 Compare September 24, 2025 14:05

link04 approved these changes Oct 8, 2025

View reviewed changes

andrewlock force-pushed the andrew/settings/4a-fix-adhoc-accessing branch from 1a187b8 to 191d5e2 Compare October 14, 2025 16:41

tonyredondo approved these changes Oct 16, 2025

View reviewed changes

Base automatically changed from andrew/settings/3d-simplify-per-trace-settings to master October 16, 2025 08:59

andrewlock added 8 commits October 16, 2025 10:00

nit: simplify PerTraceSettings to remove unecessary field

9d425d4

Fix API Security Tests

226e7d4

Fix AppSec unit tests (and remove uses of static Security instance)

1bf5ef9

Fix broken build

4785ffd

Introduce StubDatadogTracer to avoid using Mock<>

0aae0a8

More test fixes

9835a3a

More test fixes

c758157

andrewlock force-pushed the andrew/settings/4a-fix-adhoc-accessing branch from b3dd98e to c758157 Compare October 16, 2025 09:00

lucaspimentel approved these changes Oct 20, 2025

View reviewed changes

NachoEchevarria approved these changes Oct 22, 2025

View reviewed changes

andrewlock merged commit a2cc41a into master Oct 22, 2025
154 checks passed

andrewlock deleted the andrew/settings/4a-fix-adhoc-accessing branch October 22, 2025 09:34

github-actions bot added this to the vNext-v3 milestone Oct 22, 2025

andrewlock mentioned this pull request Oct 22, 2025

Create SettingsManager for managing mutable settings and ExporterSettings #7695

Open

andrewlock added the area:configuration label Nov 3, 2025

andrewlock mentioned this pull request Nov 3, 2025

Fix Asm benchmark and stop-reinitializing tracer #7750

Merged

andrewlock mentioned this pull request Nov 10, 2025

Add fix for not re-reporting telemetry when there are "empty" dynamic config changes #7796

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix ad-hoc use of `TracerSettings` to use `MutableSettings` where appropriate #7543

Fix ad-hoc use of `TracerSettings` to use `MutableSettings` where appropriate #7543

Uh oh!

andrewlock commented Sep 22, 2025 •

edited

Loading

Uh oh!

This comment has been minimized.

dd-trace-dotnet-ci-bot bot commented Sep 23, 2025 •

edited

Loading

Uh oh!

bouwkast left a comment

Uh oh!

bouwkast Sep 23, 2025

Uh oh!

andrewlock Sep 24, 2025

Uh oh!

bouwkast Sep 23, 2025

Uh oh!

bouwkast Sep 23, 2025

Uh oh!

andrewlock Sep 24, 2025

Uh oh!

NachoEchevarria left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Fix ad-hoc use of TracerSettings to use MutableSettings where appropriate #7543

Fix ad-hoc use of TracerSettings to use MutableSettings where appropriate #7543

Uh oh!

Conversation

andrewlock commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

Uh oh!

This comment has been minimized.

dd-trace-dotnet-ci-bot bot commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

Uh oh!

bouwkast left a comment

Choose a reason for hiding this comment

Uh oh!

bouwkast Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

andrewlock Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

bouwkast Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

bouwkast Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

andrewlock Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

NachoEchevarria left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Fix ad-hoc use of `TracerSettings` to use `MutableSettings` where appropriate #7543

Fix ad-hoc use of `TracerSettings` to use `MutableSettings` where appropriate #7543

andrewlock commented Sep 22, 2025 •

edited

Loading

dd-trace-dotnet-ci-bot bot commented Sep 23, 2025 •

edited

Loading