The right way to profile vector

[nabokihms]

Hello, Vector team!

In Kubernetes clusters under my control, some installations of Vector consume a lot of CPU and memory (3 cores and 7Gb RAM top). Probably, the problem is with the configuration, but I want to find exactly where.

Do you have any tips or tricks how I can shed more light on why vector has such an appetite? It would be nice if there is the way to extract the date from a running process, but not required.

[pront]

There are a few useful ways to narrow this down:

• Memory by component: start Vector with --allocation-tracing, then collect the internal_metrics source. The allocation metrics include component_allocated_bytes for current tracked allocations and component_allocated_bytes_total for cumulative allocations, tagged by component. This requires restarting with tracing enabled.
• CPU by transform: enable measure_cpu_usage: true on the transforms you want to inspect. It exposes component_cpu_usage_ns_total through the same internal metrics source.
• CPU call stacks from a running process: our profiling guide covers attaching Linux perf to Vector. You'll need the appropriate profiling permissions and debug symbols for useful stack traces.

I verified the allocation and transform CPU metrics locally on Vector 0.60.0. The pipeline debugging guide also covers checking component throughput and inspecting events with vector top and vector tap.