Skip to content

fix(openfeature): isolate invalid flags [backport v2.9] - #5105

Merged
darccio merged 3 commits into
release-v2.9.xfrom
agent/backport-ufe-invalid-flag-v2.9
Aug 6, 2026
Merged

fix(openfeature): isolate invalid flags [backport v2.9]#5105
darccio merged 3 commits into
release-v2.9.xfrom
agent/backport-ufe-invalid-flag-v2.9

Conversation

@leoromanovsky

Copy link
Copy Markdown
Contributor

Motivation

A malformed flag can reject the complete UFC configuration. This makes unrelated flags return defaults.

Backport the exact merged main commit 6d96ff5424 to release-v2.9.x. A compatibility commit adapts tests to the older evaluator signature.

Changes and Decisions

  • Parse and validate flags independently.
  • Reject only the malformed flag.
  • Keep valid neighboring flags available for evaluation.
  • Use the shared FFE fixture corpus.

Use the shared FFE fixture corpus. This prevents copied Go fixtures from drifting from other SDKs.

The same migration is merged in [Java](DataDog/dd-trace-java#11355) and [libdatadog](DataDog/libdatadog#1979). Related migrations are open for [Python](DataDog/dd-trace-py#19390), [JavaScript](DataDog/dd-trace-js#8441), [Ruby](DataDog/dd-trace-rb#5742), and [.NET](DataDog/dd-trace-dotnet#8616).

- Add `DataDog/ffe-system-test-data` as an OpenFeature test submodule.
- Read all canonical configuration and evaluation cases from the submodule.
- Parse and validate flags independently so one invalid flag does not reject valid neighbors.
- Assert canonical values and reasons, including temporal defaults and missing-flag errors.
- Initialize submodules in Go test workflows and configure weekly Dependabot updates.

Co-authored-by: leo.romanovsky <leo.romanovsky@datadoghq.com>
@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jul 31, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: No data available
Overall Coverage: 62.04% (+0.03%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 28ec9e5 | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jul 31, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-08-05 20:59:08

Comparing candidate commit 28ec9e5 in PR branch agent/backport-ufe-invalid-flag-v2.9 with baseline commit f5f7068 in branch release-v2.9.x.

Found 4 performance improvements and 0 performance regressions! Performance is the same for 269 metrics, 2 unstable metrics, 1 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:BenchmarkOTLPProtoSize/1000spans

  • 🟩 execution_time [-18.230µs; -17.207µs] or [-5.612%; -5.297%]

scenario:BenchmarkOTLPProtoSize/100spans

  • 🟩 execution_time [-1.865µs; -1.751µs] or [-5.744%; -5.393%]

scenario:BenchmarkOTLPProtoSize/10spans

  • 🟩 execution_time [-232.893ns; -223.907ns] or [-6.941%; -6.673%]

scenario:BenchmarkOTLPProtoSize/1span

  • 🟩 execution_time [-28.721ns; -26.739ns] or [-7.076%; -6.588%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

Known flaky benchmarks without significant changes:

  • scenario:BenchmarkOTLPTraceWriterFlush

@darccio
darccio marked this pull request as ready for review August 3, 2026 13:42
@darccio
darccio requested review from a team as code owners August 3, 2026 13:42
@darccio
darccio requested review from pavlokhrebto and sameerank and removed request for a team August 3, 2026 13:42

@darccio darccio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving on behalf of @DataDog/dd-trace-go-guild as the original PR was reviewed and approved.

@darccio
darccio force-pushed the agent/backport-ufe-invalid-flag-v2.9 branch from d6a9172 to 28ec9e5 Compare August 5, 2026 20:33
@darccio

darccio commented Aug 6, 2026

Copy link
Copy Markdown
Member

Failures are unrelated.

@darccio
darccio merged commit 75602a2 into release-v2.9.x Aug 6, 2026
199 of 205 checks passed
@darccio
darccio deleted the agent/backport-ufe-invalid-flag-v2.9 branch August 6, 2026 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants