Prometheus 3 Native Histogram and OpenTelemetry Exponential Histogram #7356
Description
What type of enhancement is this?
API improvement
What does the enhancement do?
Prometheus Native Histogram
Native histogram becomes a stable feature of Prometheus since 3.8
General structure
Similar to a classic histogram, a native histogram has a field for the count of observations and a field for the sum of observations. While the count of observation is generally non-negative (with the only exception being intermediate results in PromQL), the sum of observations might have any float64 value.
In addition, a native histogram contains the following components, which are described in detail in dedicated sections below:
- A schema to identify the method of determining the boundaries of any given bucket with an index i.
- A sparse representation of indexed buckets, mirrored for positive and negative observations.
- A zero bucket to count observations close to zero.
- A (possibly empty) list of custom values.
Exemplars.
More context: https://prometheus.io/docs/specs/native_histograms/
Not sure if it's required to support Remote Write 2.0 first. #4765
OpenTelemetry Exponential Histogram
OpenTelemetry Exponential Histogram is the OpenTelemetry equivalent to Prometheus Native Histogram. We can use the data structure of Prometheus to support this Otel data type.
https://opentelemetry.io/docs/specs/otel/metrics/data-model/#exponentialhistogram
Required changes
- Remote write interface update to support histogram message. We will need to support the new remote write 2.0 first Prometheus remote write 2.0 #4765 And new histogram message is available here https://github.com/prometheus/prometheus/blob/main/prompb/io/prometheus/write/v2/types.proto#L165
- Update: native histogram is also available from v1 remote write protobuf definition: https://github.com/prometheus/prometheus/blob/main/prompb/types.proto#L63
- Data modelling for new histogram data structure
- There will be only 1 time-series for each histogram
- Histogram will carry a struct for its data, instead of traditional float. General struct
- PromQL adoption for new histogram data
- Implement OpenTelemetry exponential histogram for Prometheus data modelling and PromQL, just like an alternative input format than remote write.
Implementation challenges
No response