Feature Request: Intelligent Polling with Backoff for Windows Event Log Receiver

[jsirianni]

Component(s)

receiver/windowseventlog

Is your feature request related to a problem? Please describe.

Yes. The Windows Event Log receiver currently uses a fixed polling interval (poll_interval), which results in inefficient resource usage. High volume systems require collector config tuning in order to operate efficiently.

1. When there are many events available, the receiver reads up to max_reads events, then waits for the full poll_interval before checking again. This creates unnecessary delays in processing high-volume event streams.

2. When there are few or no events, the receiver continues polling at the fixed interval.

Describe the solution you'd like

Implement intelligent polling with exponential backoff that adapts to event volume:

1. If a read operation returns the maximum number of configured events (max_reads), immediately perform another read without any delay. This ensures high-volume event streams are processed as quickly as possible.

2. If a read operation returns fewer than max_reads events, begin exponential backoff before the next read attempt.

3. The maximum backoff interval should be capped at the configured poll_interval.**

The function readOnInterval is structured in a way that would easily allow backoff to be introduced using the github.com/cenkalti/backoff package.

**The goal is to keep the user interface the same. I am open to discussion here, we could implement new config options that allow users to opt into the new backoff behavior.

Describe alternatives you've considered

The alternative is to configure the poll interval to a very low value, ensuring the configuration works well on high volume and low volume systems.

Additional context

I am happy to contribute this change if the proposal is accepted.

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[github-actions]

Pinging code owners:

• receiver/windowseventlog: @armstrmi @pjanotti

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[VihasMakwana]

Looks like a useful enhancement! I have a question:

1. What would be the starting period between the poll cycles, in case of low volume reads?

[jsirianni]

Looks like a useful enhancement! I have a question:

1. What would be the starting period between the poll cycles, in case of low volume reads?

It would be configurable. I believe it would be okay to default to the backoff package defaults, with the exception of a max elapsed time, this would be set to 0 (infinte),

const (
	DefaultInitialInterval     = 500 * time.Millisecond
	DefaultRandomizationFactor = 0.5
	DefaultMultiplier          = 1.5
	DefaultMaxInterval         = 60 * time.Second
)

https://github.com/cenkalti/backoff/blob/v5/exponential.go#L56-L62

The max interval would be the current poll interval, or we introduce a new set of configuration that overrides poll interval, allowing the user to opt into the backoff behavior.

[pjanotti]

This should be fixed via #43195

cc @MrAnno