A note for the community
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Problem
Filing this issue as requested from this comment, with some extra color.
We have a lot of servers, all of them sends system logs by http-sink to our service "logs-collector". Logs-collector has many IPv6 addresses and these addresses are issued in random order.
The vector instances sort the DNS output and sent logs to the first IPv6 address ("----:----:----:----:----:----:2f48:5fc" in example ), as a result one of instances logs-collector is very busy and all the other are idle.
We have a similar configuration, but using IPv4, and see the same behavior.
I've had to dig into this in a Java application recently, and what I discovered is that this behavior is caused by getaddrinfo respecting RFC3484. In short, rule 9 is hitting here. Addresses in the same subnet/prefix as the client (host) will be first in getaddrinfo's response, sorted by proximity to the client. As a result, if any of the destination addresses are in the same subnet/prefix as the host, Vector will display this behavior of getting "stuck" to the same destination.
I haven't dug in far enough to understand if this is fixable in vector. At a glance, it looks like hyper::client allows one to use a custom resolver (docs implying this). If so, there's two options here:
- Implement a resolver that does not use getaddrinfo and just returns the list of addresses directly. This is what the other commenter is asking for, and probably the "best" solution. More effort, though.
- (Optionally?) shuffle the addresses coming back from getaddrinfo. This is less effort (though may need a config option), but if someone has DNS servers returning addresses in some preferred order, it won't work for them.
Thoughts? Happy to provide more details if needed.
Configuration
Version
0.47.0
Debug Output
Example Data
No response
Additional Context
No response
References
No response
A note for the community
Problem
Filing this issue as requested from this comment, with some extra color.
We have a similar configuration, but using IPv4, and see the same behavior.
I've had to dig into this in a Java application recently, and what I discovered is that this behavior is caused by
getaddrinforespecting RFC3484. In short, rule 9 is hitting here. Addresses in the same subnet/prefix as the client (host) will be first in getaddrinfo's response, sorted by proximity to the client. As a result, if any of the destination addresses are in the same subnet/prefix as the host, Vector will display this behavior of getting "stuck" to the same destination.I haven't dug in far enough to understand if this is fixable in vector. At a glance, it looks like
hyper::clientallows one to use a custom resolver (docs implying this). If so, there's two options here:Thoughts? Happy to provide more details if needed.
Configuration
Version
0.47.0
Debug Output
Example Data
No response
Additional Context
No response
References
No response