Skip to content

Conversation

@mahendrabishnoi2
Copy link
Member

@mahendrabishnoi2 mahendrabishnoi2 commented Oct 11, 2025

Fixes #7014

This PR adds support for below self observability metrics for stdoutmetric exporter

  • otel.sdk.exporter.metric_data_point.inflight
  • otel.sdk.exporter.metric_data_point.exported
  • otel.sdk.exporter.operation.duration

These metrics are experimental and hence behind a feature flag OTEL_GO_X_OBSERVABILITY.
Definition of above metrics is available at https://github.com/open-telemetry/semantic-conventions/blob/v1.36.0/docs/otel/sdk-metrics.md

Observability Implementation Checklist

Observability Implementation Checklist

Based on the project Observability guidelines, ensure the following are completed:

Environment Variable Activation

  • Observability features are disabled by default
  • Features are activated through the OTEL_GO_X_OBSERVABILITY environment variable
  • Use consistent pattern with x.Observability.Enabled() check 1
  • Follow established experimental feature pattern 23

Encapsulation

  • Instrumentation is encapsulated within a dedicated struct (e.g., Instrumentation)
  • Instrumentation is not mixed into the instrumented component
  • Instrumentation code is in its own file or package if complex/reused
  • Instrumentation setup doesn't bloat the main component code

Initialization

  • Initialization is only done when observability is enabled
  • Setup is explicit and side-effect free
  • Return errors from initialization when appropriate
  • Use the global Meter provider (e.g., otel.GetMeterProvider())
  • Include proper meter configuration with:
    • Instrumentation package name is used for the Meter
    • Instrumentation version (e.g. Version)
    • Schema URL (e.g. SchemaURL)

Performance

  • Little to no overhead when observability is disabled
  • Expensive operations are only executed when observability is enabled
  • When enabled, instrumentation code paths are optimized to reduce allocation/computation overhead

Attribute and Option Allocation Management

  • Use sync.Pool for attribute slices and options with dynamic attributes
  • Pool objects are properly reset before returning to pool
  • Pools are scoped for maximum efficiency while ensuring correctness

Caching

  • Static attribute sets known at compile time are pre-computed and cached
  • Common attribute combinations use lookup tables/maps

Benchmarking

  • Benchmarks provided for all instrumentation code
  • Benchmark scenarios include both enabled and disabled observability
  • Benchmark results show impact on allocs/op, B/op, and ns/op (use b.ReportAllocs() in benchmarks)

Error Handling and Robustness

  • Errors are reported back to caller when possible
  • Partial failures are handled gracefully
  • Use partially initialized components when available
  • Return errors to caller instead of only using otel.Handle()
  • Use otel.Handle() only when component cannot report error to user

Context Propagation

  • Observability measurements receive the context from the function being measured (don't break context propagation by using context.Background())

Semantic Conventions Compliance

  • All metrics follow OpenTelemetry Semantic Conventions
  • Use the otelconv convenience package for metric semantic conventions
  • Component names follow semantic conventions
  • Use package path scope type as stable identifier for non-standard components
  • Component names are stable unique identifiers
  • Use global counter for uniqueness if necessary
  • Component ID counter is resettable for deterministic testing

Testing

  • Use deterministic testing with isolated state
  • Restore previous state after tests (t.Cleanup())
  • Isolate meter provider for testing
  • Use t.Setenv() for environment variable testing
  • Reset component ID counter for deterministic component names
  • Test order doesn't affect results

Footnotes

  1. https://github.com/open-telemetry/opentelemetry-go/blob/e4ab3141123d0811125a69823dbbe4d9ec5a9b8f/exporters/stdout/stdouttrace/internal/observ/instrumentation.go#L101-L103

  2. https://github.com/open-telemetry/opentelemetry-go/blob/e4ab3141123d0811125a69823dbbe4d9ec5a9b8f/exporters/stdout/stdouttrace/internal/x/x.go

  3. https://github.com/open-telemetry/opentelemetry-go/blob/e4ab3141123d0811125a69823dbbe4d9ec5a9b8f/sdk/internal/x/x.go

mahendrabishnoi2 and others added 30 commits August 4, 2025 22:18
…etrics

1. otel.sdk.exporter.metric_data_point.inflight
2. otel.sdk.exporter.metric_data_point.exported
3. otel.sdk.exporter.operation.duration
- use pool to amortize slice allocation
- pass actual context
- use t.Cleanup instead of defer in tests
- improve readability by returning without using err var
- use metricdatatest for comparision in testcase
Copy link
Member

@flc1125 flc1125 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be continued

@MrAlias MrAlias added this to the v1.39.0 milestone Oct 16, 2025
@mahendrabishnoi2
Copy link
Member Author

Updated Benchmarks

exporter_1.txt (commit=c106988)
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	 2128890	       657.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2040018	       593.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2096229	       583.6 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2097553	       591.9 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2071342	       598.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 1933758	       595.3 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2066118	       584.8 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2086310	       652.5 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2051514	       576.4 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2041674	       582.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1635116	       734.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1599358	       825.2 ns/op	    1442 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1554500	       738.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631379	       744.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1628688	       743.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1623816	       731.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631787	       740.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1652600	       739.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1646541	       733.1 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1642182	       736.2 ns/op	    1443 B/op	      13 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability	38.504s
exporter_3.txt (commit=545ab7e)
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	39108700	        29.91 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	59554951	        40.01 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	31668709	        42.51 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	31591450	        45.11 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	30063759	        43.24 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	30614202	        40.06 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	32252391	        46.24 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	46108329	        36.79 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	48092174	        43.25 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	34851410	        47.61 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/WithError-8       	 6605668	       182.8 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6558997	       186.4 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6475551	       190.7 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6609228	       186.8 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6534050	       188.7 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6294747	       190.5 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6340726	       194.8 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6104074	       198.2 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6362425	       197.6 ns/op	     240 B/op	       3 allocs/op
BenchmarkTrackExport/WithError-8       	 6397458	       200.4 ns/op	     240 B/op	       3 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ	33.281s
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
                        │ exporter_1.txt │            exporter_3.txt            │
                        │     sec/op     │    sec/op     vs base                │
TrackExport/Success-8      592.55n ± 10%   42.88n ± 14%  -92.76% (p=0.000 n=10)
TrackExport/WithError-8     739.2n ±  1%   190.6n ±  4%  -74.22% (p=0.000 n=10)
geomean                     661.8n         90.40n        -86.34%

                        │ exporter_1.txt │              exporter_3.txt               │
                        │      B/op      │     B/op      vs base                     │
TrackExport/Success-8       1.157Ki ± 0%   0.000Ki ± 0%  -100.00% (p=0.000 n=10)
TrackExport/WithError-8      1443.0 ± 0%     240.0 ± 0%   -83.37% (p=0.000 n=10)
geomean                     1.277Ki                      ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

                        │ exporter_1.txt │             exporter_3.txt              │
                        │   allocs/op    │ allocs/op   vs base                     │
TrackExport/Success-8         13.00 ± 0%    0.00 ± 0%  -100.00% (p=0.000 n=10)
TrackExport/WithError-8      13.000 ± 0%   3.000 ± 0%   -76.92% (p=0.000 n=10)
geomean                       13.00                    ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

@MrAlias MrAlias changed the title stdoutmetric: self-observability: exporter metrics stdoutmetric exporter observability Oct 23, 2025
@MrAlias MrAlias added the pkg:exporter:stdout Related to the stdout exporter package label Oct 23, 2025
@mahendrabishnoi2
Copy link
Member Author

mahendrabishnoi2 commented Oct 28, 2025

Updated Benchmarks

exporter_1.txt (commit=c106988)
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	 2128890	       657.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2040018	       593.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2096229	       583.6 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2097553	       591.9 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2071342	       598.0 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 1933758	       595.3 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2066118	       584.8 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2086310	       652.5 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2051514	       576.4 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/Success-8         	 2041674	       582.2 ns/op	    1185 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1635116	       734.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1599358	       825.2 ns/op	    1442 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1554500	       738.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631379	       744.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1628688	       743.6 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1623816	       731.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1631787	       740.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1652600	       739.9 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1646541	       733.1 ns/op	    1443 B/op	      13 allocs/op
BenchmarkTrackExport/WithError-8       	 1642182	       736.2 ns/op	    1443 B/op	      13 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/selfobservability	38.504s
exporter_4.txt (commit=39c5f62)
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
BenchmarkTrackExport/Success-8         	36423651	        27.62 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	69032962	        24.43 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	52642642	        25.75 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	65783911	        25.83 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	58209097	        26.04 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	70442344	        20.41 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	45047722	        24.51 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	63353791	        25.05 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	50422885	        22.71 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/Success-8         	67147107	        21.94 ns/op	       0 B/op	       0 allocs/op
BenchmarkTrackExport/WithError-8       	 6784684	       170.4 ns/op	     216 B/op	       2 allocs/op
BenchmarkTrackExport/WithError-8       	 6851582	       173.4 ns/op	     216 B/op	       2 allocs/op
BenchmarkTrackExport/WithError-8       	 6904780	       171.1 ns/op	     216 B/op	       2 allocs/op
BenchmarkTrackExport/WithError-8       	 7078076	       195.2 ns/op	     216 B/op	       2 allocs/op
BenchmarkTrackExport/WithError-8       	 5842893	       187.0 ns/op	     216 B/op	       2 allocs/op
BenchmarkTrackExport/WithError-8       	 6889072	       184.7 ns/op	     216 B/op	       2 allocs/op
BenchmarkTrackExport/WithError-8       	 7094217	       174.1 ns/op	     216 B/op	       2 allocs/op
BenchmarkTrackExport/WithError-8       	 7073342	       178.3 ns/op	     216 B/op	       2 allocs/op
BenchmarkTrackExport/WithError-8       	 7034476	       173.2 ns/op	     216 B/op	       2 allocs/op
BenchmarkTrackExport/WithError-8       	 7085778	       176.4 ns/op	     216 B/op	       2 allocs/op
PASS
ok  	go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ	30.051s
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
                        │ exporter_1.txt │            exporter_4.txt            │
                        │     sec/op     │    sec/op     vs base                │
TrackExport/Success-8      592.55n ± 10%   24.78n ± 11%  -95.82% (p=0.000 n=10)
TrackExport/WithError-8     739.2n ±  1%   175.2n ±  7%  -76.29% (p=0.000 n=10)
geomean                     661.8n         65.90n        -90.04%

                        │ exporter_1.txt │              exporter_4.txt               │
                        │      B/op      │     B/op      vs base                     │
TrackExport/Success-8       1.157Ki ± 0%   0.000Ki ± 0%  -100.00% (p=0.000 n=10)
TrackExport/WithError-8      1443.0 ± 0%     216.0 ± 0%   -85.03% (p=0.000 n=10)
geomean                     1.277Ki                      ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

                        │ exporter_1.txt │             exporter_4.txt              │
                        │   allocs/op    │ allocs/op   vs base                     │
TrackExport/Success-8         13.00 ± 0%    0.00 ± 0%  -100.00% (p=0.000 n=10)
TrackExport/WithError-8      13.000 ± 0%   2.000 ± 0%   -84.62% (p=0.000 n=10)
geomean                       13.00                    ?                       ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean

Benchmark vs exporter_3.txt (Before review from @MrAlias)

goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/exporters/stdout/stdoutmetric/internal/observ
cpu: Apple M1 Pro
                        │ exporter_3.txt │            exporter_4.txt            │
                        │     sec/op     │    sec/op     vs base                │
TrackExport/Success-8       42.88n ± 14%   24.78n ± 11%  -42.20% (p=0.000 n=10)
TrackExport/WithError-8     190.6n ±  4%   175.2n ±  7%   -8.05% (p=0.002 n=10)
geomean                     90.40n         65.90n        -27.10%

                        │ exporter_3.txt │            exporter_4.txt            │
                        │      B/op      │    B/op     vs base                  │
TrackExport/Success-8       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
TrackExport/WithError-8     240.0 ± 0%     216.0 ± 0%  -10.00% (p=0.000 n=10)
geomean                                ²                -5.13%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                        │ exporter_3.txt │            exporter_4.txt            │
                        │   allocs/op    │ allocs/op   vs base                  │
TrackExport/Success-8       0.000 ± 0%     0.000 ± 0%        ~ (p=1.000 n=10) ¹
TrackExport/WithError-8     3.000 ± 0%     2.000 ± 0%  -33.33% (p=0.000 n=10)
geomean                                ²               -18.35%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

Copy link
Member

@flc1125 flc1125 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition, we are missing a benchmark test case and its results based on the exporter.


// Version is the current release version of the OpenTelemetry stdoutmetric
// exporter in use.
const Version = "1.38.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add configurations such as the following:

go.opentelemetry.io/otel/exporters/stdout/stdouttrace:
version-refs:
- ./exporters/stdout/stdouttrace/internal/version.go

em := newExp(b)
b.ResetTimer()
b.ReportAllocs()
b.RunParallel(func(pb *testing.PB) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmarking can be verified without using concurrent testing.

em := newExp(b)
b.ResetTimer()
b.ReportAllocs()
b.RunParallel(func(pb *testing.PB) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as: Benchmarking can be verified without using concurrent testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pkg:exporter:stdout Related to the stdout exporter package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Metrics SDK observability - stdoutmetric exporter metrics

4 participants