-
Notifications
You must be signed in to change notification settings - Fork 703
feat: add sampling attributes for jaeger remote sampler #8073
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
cc code owner: @yurishkuro PTAL |
dmathieu
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those attributes don't match anything defined in semantic conventions.
yurishkuro
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest writing a test in a different package like jaegerremote_test, to make sure you got the visibility of new functions / types correctly
samplers/jaegerremote/sampler.go
Outdated
|
|
||
| type samplerOptionFunc func(*samplerOptions) | ||
|
|
||
| func withAttributesOn() samplerOptionFunc { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would make then on by default, and have this option to turn off. The function needs to be public in order to be used from a different package.
| func withAttributesOn() samplerOptionFunc { | |
| func WithAttributesDisabled() samplerOptionFunc { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't sure that we want to change current default behavior - jaegerremote does not set any attributes.
This is supposed to be an internal option for creating and updating samplers which constructors also aren't exported.
For external usage I added WithAttributesOn option.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do want to change the current behavior. The current behavior is useless, the sampler does not actually work with Jaeger's adaptive backend. And the change itself is additive, it's backwards compatible (especially since the new attributes are prefixed with jaeger.).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need an option to disable these attributes in that case? If not implementation will be simple and no need for internal sampler options too
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Personally I would argue you don't even need the opt-out option, OTEL SDK is already extremely verbose in terms of how many attributes it captures, having two extra ones on root span only seems irrelevant. But for 100% backwards compat you would provide an option to opt-out.
samplers/jaegerremote/sampler.go
Outdated
| samplingRate float64 | ||
| sampler trace.Sampler | ||
| attributes []attribute.KeyValue | ||
| attributesOn bool |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this flag is only used at initialization, you don't really need to store it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When probabilisticSampler or ratelimitngSampler are being updated its init method is also called, so I not sure if we can omit this flag.
samplers/jaegerremote/sampler.go
Outdated
| samplerTypeKey = "sampler.type" | ||
| samplerTypeValueProbabilistic = "probabilistic" | ||
| samplerTypeValueRateLimiting = "ratelimiting" | ||
| samplerParamKey = "sampler.param" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would make sense to prefix with jaeger., This may require small change in the backend to recognize, but it separates custom attributes into clear namespace.
| samplerTypeKey = "sampler.type" | |
| samplerTypeValueProbabilistic = "probabilistic" | |
| samplerTypeValueRateLimiting = "ratelimiting" | |
| samplerParamKey = "sampler.param" | |
| samplerTypeKey = "jaeger.sampler.type" | |
| samplerParamKey = "jaeger.sampler.param" | |
| samplerTypeValueProbabilistic = "probabilistic" | |
| samplerTypeValueRateLimiting = "ratelimiting" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, this is reasonable but I see some difficulties when using in mixed environments (jaeger and otel services) where will be different sampler attributes in that case. Not sure if backend can support more that one key.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is a Jaeger-specific functionality and a Jaeger-specific sampler. Since it's not setting any attributes now then nothing is depending on them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So can we keep canonical keys? I did small research and using new keys will require some overhead on collector to transform these attributes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since those are jaeger specific, prefixing them with jaeger definitely makes sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added prefix
|
Thank you for this PR! Originally when I looked at this problem there was no way in the API to pass back attributes (I think I even had an option issue about that). |
Can we do anything about it? Those attributes provide back-compatibility with jaeger-remote-sampler and can be turned only by an option. So yeah it is kinda jaeger attributes. |
|
@dmathieu @yurishkuro If you’re okay with it, I could try a different approach - add two new options - functions that create attributes for probabilistic/ratelimiting samplers, so user can define their own attributes to solve this issue. |
|
Sorry, my assessment was wrong. As these are jaeger specific, they shouldn't need semantic conventions. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8073 +/- ##
=======================================
+ Coverage 80.9% 81.8% +0.8%
=======================================
Files 194 194
Lines 13073 13380 +307
=======================================
+ Hits 10588 10949 +361
+ Misses 2107 2033 -74
- Partials 378 398 +20
🚀 New features to boost your workflow:
|
|
@yurishkuro I added new test in different package and answered on all comments. Review pls |
We can fix the CI errors before entering the review to reduce some trivial checks. 😁 |
9cc7fce to
8e6fd09
Compare
|
@etilite please add a CHANGELOG |
|
@etilite There are currently some conflicts; please fix them. |
|
@flc1125 resolved conflicts but now ci/cd failing. Probably need just to retry these jobs, can't do it myself. |
samplers/jaegerremote/sampler.go
Outdated
| samplerTypeValueRateLimiting = "ratelimiting" | ||
| ) | ||
|
|
||
| type samplerOptions struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is redundant. The package already has the functional options mechanism
func New(
serviceName string,
opts ...Option,
) *Sampler {
options := newConfig(opts...)
So WithAttributesOn() should be implemented as Option just like other options, and should set a flag on the config struct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the idea behind these internal options is to have different behavior when creating samplers without need to change every test to add new argument in constructor.
I can use flag arg in sampler's constructors if it is ok with you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
without need to change every test to add new argument in constructor.
a) why do you need to change every test? do they all compare attributes?
b) even if you need to, it seems like a trivial task for a Copilot
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a) cause a lot of tests call these constructors
b) yeah but it is harder to review a lot of small changes
nvm I'll change to flag argument 👍
samplers/jaegerremote/sampler.go
Outdated
|
|
||
| type samplerOptionFunc func(*samplerOptions) | ||
|
|
||
| func withAttributesOn() samplerOptionFunc { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do want to change the current behavior. The current behavior is useless, the sampler does not actually work with Jaeger's adaptive backend. And the change itself is additive, it's backwards compatible (especially since the new attributes are prefixed with jaeger.).
| // Sampler invokes the updaters while holding a lock on the main sampler. | ||
| type samplerUpdater interface { | ||
| Update(sampler trace.Sampler, strategy any) (modified trace.Sampler, err error) | ||
| Update(sampler trace.Sampler, strategy any, attributesDisabled bool) (modified trace.Sampler, err error) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is not necessary to change this interface to pass attributesDisabled. When individual updaters are created the value of this flag is known and can be captured in the updaters themselves. This way we separate configuration information from the execution information (only sampler and strategy are variable at runtime)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point! Can I remove withUpdaters option then? Cause it makes harder to properly create updaters and pass attributesDisabled to them. Anyway withUpdaters is not needed for tests now (can be safely removed) and seems to be an old arfifact
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
withUpdaters is private, safe to remove if not needed. These APIs used to be public for extensibility.
Which problem is this PR solving?
Resolves #3653
Resolves #691
Description of the changes
WithAttributesOnwhich addsjaeger.sampler.paramandjaeger.sampler.typeattributes for sampled spans