Skip to content

Avoid native server port collisions - #56

Draft
khluu wants to merge 1 commit into
mainfrom
codex/fix-native-server-port-collision
Draft

Avoid native server port collisions#56
khluu wants to merge 1 commit into
mainfrom
codex/fix-native-server-port-collision

Conversation

@khluu

@khluu khluu commented Jul 31, 2026

Copy link
Copy Markdown
Member

What changed

  • derive a high per-job server port from BUILDKITE_JOB_ID, skipping ports already in use
  • require /v1/models to report the expected workload model before declaring the server healthy
  • bound server and log-follower shutdown so a failed benchmark cannot hang until the Buildkite timeout
  • add stdlib-only lifecycle regression tests for port selection, model-aware readiness, and cleanup escalation

Root cause

In nightly build 350, gpt_oss_120b-b200 and minimax_m3-b200 ran concurrently on dgxB200-14 with native servers using host networking and fixed port 8000. GPT-OSS became healthy at 06:12:31; three seconds later MiniMax's readiness probe accepted that server while MiniMax was still loading. MiniMax sent 512 requests for nvidia/MiniMax-M3-NVFP4 to the GPT-OSS server and received 512 HTTP 404 responses. Its EXIT cleanup then blocked on the log-follower pipeline until the two-hour job timeout.

Build: https://buildkite.com/vllm/perf-eval/builds/350

Validation

  • python3 .buildkite/test_server_lifecycle.py (4/4)
  • python3 .buildkite/test_generate_pipeline.py (6/6)
  • python3 -m py_compile .buildkite/test_server_lifecycle.py .buildkite/generate_pipeline.py lib/parse_workload.py
  • bash -n lib/server.sh lib/run.sh lib/run_vllm_bench.sh
  • shellcheck lib/server.sh lib/run.sh
  • MiniMax B200 workload parsing and targeted pipeline generation with BENCH_ONLY=1
  • git diff --check
  • Targeted B200 validation: https://buildkite.com/vllm/perf-eval/builds/357 — 512/512 successful requests, 0 failed

Signed-off-by: khluu <khluu000@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant