Whether the Address field can be configured with multiple instances

[uglyliu]

My config as follewers

---

data_dir: /data/svr/vector/data
timezone: "local"
api:
enabled: false
address: "[::]:8686"
playground: false
sources:
file1:
include:
- "/data/log/**/*.log"
type: file
fingerprint:
strategy: "device_and_inode"
ignore_checkpoints: true
read_from: end
internal_metrics:
type: internal_metrics
sinks:
prom_exporter:
type: prometheus_exporter
inputs:
- internal_metrics
address: "[::]:9090"
vector1:
type: vector
inputs:
- file1
address: "10.35.2.213:8688"
version: "2"

[jszwedko]

I'm not sure what question you are asking here 🤔 Could you restate it?

[uglyliu]

I'm not sure what question you are asking here 🤔 Could you restate it?

For example, I have deployed multiple vector instances and I want to configure multiple IPs in adress. Can I do this?

address: "10.35.2.213:8688"

[jszwedko]

Ah I see, no, unfortunately you can only configure one address. The typical recommendation is to put a load balancer in-between. You could also use DNS to load balance. #18249 is tracking enhancements in this area.

[uglyliu]

Ah I see, no, unfortunately you can only configure one address. The typical recommendation is to put a load balancer in-between. You could also use DNS to load balance. #18249 is tracking enhancements in this area.

Hmm, yes, I currently have an nlb load balancing installed in front。In this case, the logs are randomly placed on the disk of the vector server. if I mount remote storage, the logs of multiple services will be stored in one file. Will there be a risk of losing logs when writing? I don’t know how the vector placement works.

[pront]

This is supported now. On Vector 0.60.0, the vector sink can load-balance requests across multiple endpoints:

sinks:
  vector1:
    type: vector
    inputs: [file1]
    routing:
      endpoints:
        - http://10.35.2.213:8688
        - http://10.35.2.214:8688
      strategy: load_balance

Use your aggregators' actual addresses. routing replaces address; the two settings are mutually exclusive.

For your shared-storage question, I would give each aggregator its own output path, such as /shared/logs/aggregator-a.jsonl and /shared/logs/aggregator-b.jsonl. Load balancing does not coordinate writes to a shared file, and the safety of concurrent appends depends on the filesystem. A shared directory can contain separate files from each writer.

I tested this with two Vector receivers writing separate local files: all 100 events arrived, split 49/51 between them. The remote-storage behavior was not part of that test.