Skip to content

feat: Support a list of concurrencies for bench configs - #58

Merged
khluu merged 6 commits into
vllm-project:mainfrom
micah-wil:micah/bench-concurrency-list
Aug 13, 2026
Merged

feat: Support a list of concurrencies for bench configs#58
khluu merged 6 commits into
vllm-project:mainfrom
micah-wil:micah/bench-concurrency-list

Conversation

@micah-wil

@micah-wil micah-wil commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

This PR allows us to specify a list of concurrencies for bench configs in workloads. For example:

vllm_bench:
  configs:
    - name: 8k-in-1k-out
      backend: openai
      dataset: random
      input_len: 8192
      output_len: 1024
      num_prompts: 512
      max_concurrency: [1, 64, 128]

The above would run a an 8k/1k benchmark for each concurrency listed (1, 64, 128). You can also optionally change num_prompts to a list to correspond to each max_concurrency, e.g.:

vllm_bench:
  configs:
    - name: 8k-in-1k-out
      backend: openai
      dataset: random
      input_len: 8192
      output_len: 1024
      num_prompts: [10, 64, 1280]
      max_concurrency: [1, 64, 128]

In expand_bench_config, -conc-<value> is appended to each config's name for each conc, so in the above example the final names for the config would be 8k-in-1k-out-conc-1, 8k-in-1k-out-conc-64, 8k-in-1k-out-conc-128.

I've updated workloads/qwen3_5_mi355x.yaml as an initial example.

Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
…cy-list

Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
@khluu
khluu merged commit 0e12628 into vllm-project:main Aug 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants