feat(storage): Add SigV4 authentication support for Elasticsearch/OpenSearch storage backends #7611

SoumyaRaikwar · 2025-10-22T21:39:20Z

This PR enables Jaeger to use AWS Managed Elasticsearch/OpenSearch for trace and metrics storage by adding SigV4 HTTP authentication support to Elasticsearch and OpenSearch backends.

Summary of changes

Configuration

Add jaeger_storage.backends.<name>.<elasticsearch|opensearch>.auth_extension.authenticator to reference an OpenTelemetry HTTP authenticator extension by name
Add jaeger_storage.metric_backends.<name>.<elasticsearch|opensearch>.auth_extension.authenticator for metric storage backends

Elasticsearch/OpenSearch backends

Thread the resolved HTTP authenticator through the factory chain (v1/v2 trace storage and metrics storage)
Wrap the HTTP RoundTripper used by ES/OS clients with the extension's RoundTripper (applies SigV4 signing when using sigv4authextension )
Updated GetHTTPRoundTripper() to accept and apply the HTTP authenticator

Configuration example

extensions:
sigv4auth:
region: us-east-1
service: es # or 'aoss' for OpenSearch Serverless
# credentials/assume-role configuration per the extension's documentation

service:
extensions: [sigv4auth]

jaeger_storage:
backends:
es-aws:
elasticsearch:
servers: ["https://my-domain.us-east-1.es.amazonaws.com/"]
auth_extension:
authenticator: sigv4auth
indices:
spans:
shards: 5
replicas: 1

metric_backends:
es-metrics:
elasticsearch:
servers: ["https://my-domain.us-east-1.es.amazonaws.com/"]
auth_extension:
authenticator: sigv4auth

Implementation

ES/OS backends now support optional HTTP authenticators via auth_extension.authenticator
The extension's RoundTripper wraps the base transport for SigV4 signing
Supports trace and metrics storage for Elasticsearch 7.x/8.x and OpenSearch

Scope

Adds authentication support to:
- Elasticsearch trace storage (v1 and v2)
- OpenSearch trace storage (v1 and v2)
- Elasticsearch metrics storage
- OpenSearch metrics storage
Backward compatible - authentication is optional

Related issue

Part of #7468

SoumyaRaikwar · 2025-10-22T22:44:54Z

@yurishkuro This PR adds SigV4 authentication support for ES/OS backends (both trace and metric storage) by reusing the existing HTTP RoundTripper pattern. The smaller diff compared to #7520 is because ES/OS already has auth infrastructure, so I just threaded the httpAuth parameter through the factory chain. Let me know if you'd like any additional test coverage or have questions about the approach.

cmd/jaeger/internal/extension/jaegerstorage/extension.go

internal/storage/metricstore/elasticsearch/factory.go

codecov · 2025-10-23T10:01:26Z

Codecov Report

❌ Patch coverage is 74.57627% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.47%. Comparing base (39ef71e) to head (5dfb4ea).

Files with missing lines	Patch %	Lines
...eger/internal/extension/jaegerstorage/extension.go	72.97%	5 Missing and 5 partials ⚠️
internal/storage/elasticsearch/config/config.go	61.53%	4 Missing and 1 partial ⚠️

❌ Your patch check has failed because the patch coverage (74.57%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7611      +/-   ##
==========================================
- Coverage   96.55%   96.47%   -0.09%     
==========================================
  Files         384      384              
  Lines       19473    19497      +24     
==========================================
+ Hits        18803    18810       +7     
- Misses        486      496      +10     
- Partials      184      191       +7

Flag	Coverage Δ
badger_v1	`8.82% <0.00%> (-0.01%)`	⬇️
badger_v2	`1.72% <0.00%> (-0.01%)`	⬇️
cassandra-4.x-v1-manual	`12.54% <0.00%> (-0.01%)`	⬇️
cassandra-4.x-v2-auto	`1.71% <0.00%> (-0.01%)`	⬇️
cassandra-4.x-v2-manual	`1.71% <0.00%> (-0.01%)`	⬇️
cassandra-5.x-v1-manual	`12.54% <0.00%> (-0.01%)`	⬇️
cassandra-5.x-v2-auto	`1.71% <0.00%> (-0.01%)`	⬇️
cassandra-5.x-v2-manual	`1.71% <0.00%> (-0.01%)`	⬇️
clickhouse	`1.65% <0.00%> (-0.01%)`	⬇️
elasticsearch-6.x-v1	`16.74% <40.00%> (-0.01%)`	⬇️
elasticsearch-7.x-v1	`16.77% <40.00%> (-0.01%)`	⬇️
elasticsearch-8.x-v1	`16.92% <55.00%> (-0.01%)`	⬇️
elasticsearch-8.x-v2	`1.72% <0.00%> (-0.01%)`	⬇️
elasticsearch-9.x-v2	`1.72% <0.00%> (-0.01%)`	⬇️
grpc_v1	`10.75% <0.00%> (-0.01%)`	⬇️
grpc_v2	`1.72% <0.00%> (-0.01%)`	⬇️
kafka-3.x-v1	`10.25% <0.00%> (-0.01%)`	⬇️
kafka-3.x-v2	`1.72% <0.00%> (-0.01%)`	⬇️
memory_v2	`1.72% <0.00%> (-0.01%)`	⬇️
opensearch-1.x-v1	`16.81% <40.00%> (-0.01%)`	⬇️
opensearch-2.x-v1	`16.81% <40.00%> (-0.01%)`	⬇️
opensearch-2.x-v2	`1.72% <0.00%> (-0.01%)`	⬇️
opensearch-3.x-v2	`1.72% <0.00%> (-0.01%)`	⬇️
query	`1.72% <0.00%> (-0.01%)`	⬇️
tailsampling-processor	`0.49% <0.00%> (-0.01%)`	⬇️
unittests	`95.38% <74.57%> (-0.09%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2025-10-23T10:02:08Z

Metrics Comparison Summary

Total changes across all snapshots: 106

Detailed changes per snapshot

summary_metrics_snapshot_cassandra

📊 Metrics Diff Summary

Total Changes: 53

🆕 Added: 0 metrics
❌ Removed: 53 metrics
🔄 Modified: 0 metrics

❌ Removed Metrics

http_server_request_body_size_bytes (18 variants)

View diff sample

-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...

- `http_server_request_duration_seconds` (17 variants)

View diff sample

-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.005",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.01",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.025",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.05",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.075",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.1",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...

- `http_server_response_body_size_bytes` (18 variants)

View diff sample

-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...

### summary_metrics_snapshot_cassandra ## 📊 Metrics Diff Summary

Total Changes: 53

🆕 Added: 0 metrics
❌ Removed: 53 metrics
🔄 Modified: 0 metrics

❌ Removed Metrics

http_server_request_body_size_bytes (18 variants)

View diff sample

-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...

- `http_server_request_duration_seconds` (17 variants)

View diff sample

-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.005",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.01",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.025",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.05",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.075",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.1",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...

- `http_server_response_body_size_bytes` (18 variants)

View diff sample

-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...

➡️ View full metrics file

- Add AuthExtension field to ES/OS configuration - Implement authenticator resolution in jaegerstorage extension - Update factory chain to pass httpAuth parameter - Add support for both trace and metric storage - Update all tests to handle new httpAuth parameter Fixes jaegertracing#7468 Signed-off-by: SoumyaRaikwar <[email protected]>

- Add AuthExtension field to Elasticsearch/OpenSearch configuration - Implement getAuthenticator method in jaegerstorage extension - Update factory chain (v1/v2/metrics) to accept httpAuth parameter - Support authentication for both trace and metric storage - Add test coverage for authenticator resolution - Update all existing tests to handle new httpAuth parameter This enables using OpenTelemetry HTTP authenticator extensions (e.g., sigv4auth) with Elasticsearch and OpenSearch backends. The authenticator is resolved from the host and wrapped into the HTTP transport layer for secure AWS authentication. Fixes jaegertracing#7468 Signed-off-by: SoumyaRaikwar <[email protected]>

Co-authored-by: Yuri Shkuro <[email protected]> Signed-off-by: Soumya Raikwar <[email protected]>

@yurishkuro

- Add resolveAuthenticator helper to reduce code duplication - Simplifies ES/OS backend initialization in extension.go Addresses review feedback from @yurishkuro Signed-off-by: SoumyaRaikwar <[email protected]>

Signed-off-by: SoumyaRaikwar <[email protected]>

- Test nil config handling (returns nil auth) - Test empty authenticator string (returns nil auth) - Test valid authenticator resolution - Test authenticator not found error case - Improves test coverage for authentication code Signed-off-by: SoumyaRaikwar <[email protected]>

graphite-app · 2025-10-23T10:45:14Z

internal/storage/v1/elasticsearch/factory_v1.go

 		cfg.UseReadWriteAliases = true
 	}
-	coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger)
+	coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger, nil)


The nil parameter passed to NewFactoryBase() should be replaced with the HTTP authenticator. The function signature has been updated to accept an authenticator parameter, but this call site is still passing nil, which means SigV4 authentication won't work when using this factory.

Consider updating this to pass the authenticator through, similar to how it's done in other factory implementations in this PR.

Suggested change

coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger, nil)

coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger, authenticator)

Spotted by Graphite Agent

Is this helpful? React 👍 or 👎 to let us know.

- Test configuration with auth extension - Test configuration without auth extension - Test configuration with empty authenticator - Improves coverage for auth extension config Signed-off-by: SoumyaRaikwar <[email protected]>

- Add TestNewFactoryWithAuthenticator for metricstore factory - Add TestElasticsearchFactoryBaseWithAuthenticator for v1 factory - Tests verify factory initialization with HTTP authenticator - Improves coverage for authenticator integration paths Signed-off-by: SoumyaRaikwar <[email protected]>

SoumyaRaikwar · 2025-10-23T11:11:47Z

@yurishkuro Done! Introduced resolveAuthenticator() helper to DRY up the ES/OS code, removed unnecessary comments, added unit tests, could you please re-review

…kends - Add auth_extension config support for ES/OS trace and metric backends - Implement resolveAuthenticator helper to reduce code duplication - Add comprehensive unit tests for authenticator resolution - Add factory creation tests with HTTP authenticators - Fix error handling and variable shadowing issues Signed-off-by: SoumyaRaikwar <[email protected]>

cmd/jaeger/internal/extension/jaegerstorage/extension.go

internal/storage/elasticsearch/config/config.go

internal/storage/v1/elasticsearch/factory.go

internal/storage/elasticsearch/config/config.go

- Change Authentication struct to use direct Authenticator field - Remove nested AuthExtension config for cleaner API - Match OpenTelemetry Collector's standard auth pattern - Update all tests and references accordingly Signed-off-by: SoumyaRaikwar <[email protected]>

internal/storage/elasticsearch/config/config.go

Co-authored-by: Yuri Shkuro <[email protected]> Signed-off-by: Soumya Raikwar <[email protected]>

@yurishkuro

- Embed configauth.Config in Authentication struct as per @yurishkuro's suggestion - Initialize embedded Config in test literals explicitly - Add configauth import to config_test.go Fixes the simpler authentication configuration pattern. Signed-off-by: SoumyaRaikwar <[email protected]>

SoumyaRaikwar · 2025-10-25T23:15:30Z

Hi @yurishkuro,

I've refactored the implementation following your suggestion to embed configauth.Config in the Authentication struct. The changes are now much cleaner:

Changes made:

Embedded configauth.Config directly in the Authentication struct using the squash mapstructure tag
Removed the custom AuthExtensionConfig type entirely
Updated extension.go to access AuthenticatorID via cfg.Authentication.AuthenticatorID
Fixed test initialization to properly initialize the embedded struct

This follows the same pattern used in other OTel collector components and keeps the code simple.
could you please review again?

cmd/jaeger/internal/extension/jaegerstorage/extension.go

internal/storage/elasticsearch/config/config.go

- Remove AuthExtensionConfig type alias - Pass Authentication struct directly to resolveAuthenticator - Update Prometheus auth to use Authentication struct - Update all tests to match new signature Follows OTEL pattern by embedding configauth.Config with squash tag Signed-off-by: SoumyaRaikwar <[email protected]>

SoumyaRaikwar · 2025-10-26T13:06:27Z

@yurishkuro Changes implemented!

Removed the AuthExtensionConfig and simplified the code as requested:

Pass Authentication struct directly to resolveAuthenticator
Eliminated all boilerplate wrapping code

The embedded configauth.Config with mapstructure:",squash" ensures the config structure matches OTEL pattern.

internal/storage/elasticsearch/config/config_test.go

cmd/jaeger/internal/extension/jaegerstorage/extension.go

- Removed TestAuthExtensionConfig from ES config tests (no actual validation) - Changed PrometheusConfiguration to use escfg.Authentication directly - Removed custom AuthConfig wrapper and boilerplate conversion code - Updated config_test to access embedded Configuration.ServerURL Both Elasticsearch and Prometheus now use identical config structures following OpenTelemetry patterns with configauth.Config. Signed-off-by: SoumyaRaikwar <[email protected]>

Signed-off-by: SoumyaRaikwar <[email protected]>

Made Prometheus metrics backend authentication configuration identical to Elasticsearch and OpenSearch for consistency: - Removed custom AuthConfig wrapper in Prometheus config - Now uses escfg.Authentication directly (same as ES/OpenSearch) - Eliminated boilerplate authentication conversion code - Added ctx parameter to NewFactoryWithConfig() for proper context propagation - Maintains Prometheus-specific jsquash wrapper for config parsing This ensures consistent authentication patterns across all storage backends while enabling proper context propagation for authentication and cancellation. Signed-off-by: SoumyaRaikwar <[email protected]>

Signed-off-by: SoumyaRaikwar <[email protected]>

SoumyaRaikwar · 2025-10-27T19:03:16Z

@yurishkuro could you please review again when you have time .

Signed-off-by: Soumya Raikwar <[email protected]>

Signed-off-by: SoumyaRaikwar <[email protected]>

SoumyaRaikwar · 2025-11-04T21:13:34Z

@yurishkuro, Codecov shows 74% for the auth paths. Should I add tests to increase coverage or is this acceptable?
could you please review again

SoumyaRaikwar requested a review from a team as a code owner October 22, 2025 21:39

SoumyaRaikwar requested a review from joe-elliott October 22, 2025 21:39

dosubot bot added area/storage enhancement storage/elasticsearch labels Oct 22, 2025

yurishkuro reviewed Oct 23, 2025

View reviewed changes

cmd/jaeger/internal/extension/jaegerstorage/extension.go Outdated Show resolved Hide resolved

yurishkuro reviewed Oct 23, 2025

View reviewed changes

internal/storage/metricstore/elasticsearch/factory.go Outdated Show resolved Hide resolved

yurishkuro reviewed Oct 23, 2025

View reviewed changes

internal/storage/metricstore/elasticsearch/factory.go Outdated Show resolved Hide resolved

SoumyaRaikwar force-pushed the elasticsearch-opensearch-authenticator branch from 7279cf9 to c0d9da3 Compare October 23, 2025 09:51

SoumyaRaikwar force-pushed the elasticsearch-opensearch-authenticator branch from c0d9da3 to 397a84b Compare October 23, 2025 10:07

SoumyaRaikwar and others added 5 commits October 23, 2025 15:42

Update internal/storage/metricstore/elasticsearch/factory.go

355862f

Co-authored-by: Yuri Shkuro <[email protected]> Signed-off-by: Soumya Raikwar <[email protected]>

refactor: DRY up authenticator resolution boilerplate

b68d4ed

- Add resolveAuthenticator helper to reduce code duplication - Simplifies ES/OS backend initialization in extension.go Addresses review feedback from @yurishkuro Signed-off-by: SoumyaRaikwar <[email protected]>

removed unnecessary comments

d6f9241

Signed-off-by: SoumyaRaikwar <[email protected]>

SoumyaRaikwar force-pushed the elasticsearch-opensearch-authenticator branch from 397a84b to d6f9241 Compare October 23, 2025 10:15

graphite-app bot reviewed Oct 23, 2025

View reviewed changes

SoumyaRaikwar added 2 commits October 23, 2025 16:18

test: Add tests for AuthExtensionConfig validation

6c235e9

- Test configuration with auth extension - Test configuration without auth extension - Test configuration with empty authenticator - Improves coverage for auth extension config Signed-off-by: SoumyaRaikwar <[email protected]>

SoumyaRaikwar requested a review from yurishkuro October 23, 2025 11:11

SoumyaRaikwar and others added 2 commits October 23, 2025 23:49

Merge branch 'main' into elasticsearch-opensearch-authenticator

31c3cd5

yurishkuro reviewed Oct 24, 2025

View reviewed changes

cmd/jaeger/internal/extension/jaegerstorage/extension.go Outdated Show resolved Hide resolved

yurishkuro reviewed Oct 24, 2025

View reviewed changes

internal/storage/elasticsearch/config/config.go Outdated Show resolved Hide resolved

yurishkuro reviewed Oct 24, 2025

View reviewed changes

internal/storage/v1/elasticsearch/factory.go Outdated Show resolved Hide resolved

Merge branch 'main' into elasticsearch-opensearch-authenticator

04e9084

SoumyaRaikwar requested a review from yurishkuro October 25, 2025 20:59

yurishkuro reviewed Oct 25, 2025

View reviewed changes

internal/storage/elasticsearch/config/config.go Outdated Show resolved Hide resolved

SoumyaRaikwar requested a review from yurishkuro October 25, 2025 22:02

yurishkuro reviewed Oct 25, 2025

View reviewed changes

internal/storage/elasticsearch/config/config.go Outdated Show resolved Hide resolved

Update internal/storage/elasticsearch/config/config.go

c9b8821

Co-authored-by: Yuri Shkuro <[email protected]> Signed-off-by: Soumya Raikwar <[email protected]>

SoumyaRaikwar requested a review from yurishkuro October 25, 2025 23:02

yurishkuro reviewed Oct 26, 2025

View reviewed changes

cmd/jaeger/internal/extension/jaegerstorage/extension.go Outdated Show resolved Hide resolved

yurishkuro reviewed Oct 26, 2025

View reviewed changes

internal/storage/elasticsearch/config/config.go Outdated Show resolved Hide resolved

SoumyaRaikwar and others added 2 commits October 26, 2025 18:35

Merge branch 'main' into elasticsearch-opensearch-authenticator

3833de7

SoumyaRaikwar requested a review from yurishkuro October 26, 2025 13:10

Merge branch 'main' into elasticsearch-opensearch-authenticator

80a1bd7

yurishkuro reviewed Oct 26, 2025

View reviewed changes

internal/storage/elasticsearch/config/config_test.go Outdated Show resolved Hide resolved

yurishkuro reviewed Oct 26, 2025

View reviewed changes

cmd/jaeger/internal/extension/jaegerstorage/extension.go Outdated Show resolved Hide resolved

SoumyaRaikwar and others added 3 commits October 27, 2025 22:46

Merge branch 'main' into elasticsearch-opensearch-authenticator

6a2c037

fixed indentation

913aa31

Signed-off-by: SoumyaRaikwar <[email protected]>

SoumyaRaikwar requested a review from yurishkuro October 27, 2025 17:23

SoumyaRaikwar added 3 commits October 27, 2025 22:55

deleted blank line

b8ca574

Signed-off-by: SoumyaRaikwar <[email protected]>

fixed linting error

841dea2

Signed-off-by: SoumyaRaikwar <[email protected]>

SoumyaRaikwar and others added 2 commits November 4, 2025 20:08

Merge branch 'main' into elasticsearch-opensearch-authenticator

4fadb8a

Signed-off-by: Soumya Raikwar <[email protected]>

add HTTPClient parameter to new client calls in tests

5dfb4ea

Signed-off-by: SoumyaRaikwar <[email protected]>

	coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger, nil)
	coreFactory, err := NewFactoryBase(context.Background(), *cfg, metricsFactory, logger, authenticator)

feat(storage): Add SigV4 authentication support for Elasticsearch/OpenSearch storage backends #7611

Are you sure you want to change the base?

feat(storage): Add SigV4 authentication support for Elasticsearch/OpenSearch storage backends #7611

Uh oh!

Conversation

SoumyaRaikwar commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Configuration example

Implementation

Scope

Related issue

Uh oh!

SoumyaRaikwar commented Oct 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Metrics Comparison Summary

summary_metrics_snapshot_cassandra

📊 Metrics Diff Summary

❌ Removed Metrics

❌ Removed Metrics

Uh oh!

graphite-app bot Oct 23, 2025

Choose a reason for hiding this comment

Uh oh!

SoumyaRaikwar commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SoumyaRaikwar commented Oct 25, 2025

Uh oh!

Uh oh!

Uh oh!

SoumyaRaikwar commented Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SoumyaRaikwar commented Oct 27, 2025

Uh oh!

SoumyaRaikwar commented Nov 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SoumyaRaikwar commented Oct 22, 2025 •

edited

Loading

codecov bot commented Oct 23, 2025 •

edited

Loading

github-actions bot commented Oct 23, 2025 •

edited

Loading

SoumyaRaikwar commented Oct 23, 2025 •

edited

Loading

SoumyaRaikwar commented Oct 26, 2025 •

edited

Loading