Skip to content

Emit metric when task count is capped by maxTasks#997

Open
khandelwal-ayush wants to merge 1 commit intolinkedin:masterfrom
khandelwal-ayush:inc-max-task
Open

Emit metric when task count is capped by maxTasks#997
khandelwal-ayush wants to merge 1 commit intolinkedin:masterfrom
khandelwal-ayush:inc-max-task

Conversation

@khandelwal-ayush
Copy link
Collaborator

Summary

  • Add a numTasksCappedByMaxTasks meter to StickyPartitionAssignmentStrategy that fires whenever the computed task count exceeds maxTasks and gets capped
  • Enables alerting on datastreams that are persistently hitting their maxTasks ceiling, which indicates they may need a higher limit or capacity adjustment

Details

When elastic task assignment computes a task count higher than the configured maxTasks, the strategy silently caps it and logs a warning. There was no metric to alert on this condition. This change adds a
per-datastream meter (StickyPartitionAssignmentStrategy..numTasksCappedByMaxTasks) so we can set up alerts for sustained capping, which signals that a datastream is under-provisioned.

Testing Done

Will deploy and test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants