fix(onboard): warn on risky Spark vLLM servers

[HwangJohn]

Summary

Adds a DGX Spark-specific warning when onboarding selects an already-running vLLM server that appears to serve a large unquantized model. This gives operators headroom guidance before agent/tool load can surface as NVRM NV_ERR_NO_MEMORY, SSH loss, or a hard host freeze.

The warning is limited to bring-your-own vLLM servers and is suppressed for the managed Spark vLLM handoff, so NemoClaw's validated managed recipe does not warn on itself.

Changes

• Detect DGX Spark during existing vLLM onboarding and warn for large unquantized model IDs, including reported max_model_len when vLLM exposes it.
• Thread a managedInstall option through the managed vLLM flow so the BYO-server warning is not emitted after install-vllm succeeds.
• Document DGX Spark local vLLM headroom triage and the distinction between host freezes and agent tool-call hangs.
• Add targeted CLI tests for the Spark warning, managed-path suppression, quantized model exclusion, numeric large-model threshold behavior, and managed vLLM option handoff.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: self-reviewed as a warning-only onboarding/inference guardrail; no credential, policy, sandbox, or vLLM launch behavior changes. Targeted tests cover warning, suppression, and quantized exclusion.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

• PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable — npm run check:diff passed on DGX Spark Linux after building the plugin/CLI artifacts required by CLI typecheck.
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — npx vitest run --project cli src/lib/onboard/setup-nim-vllm.test.ts src/lib/onboard/setup-nim-flow.test.ts passed, 22 tests.
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only) — passed on DGX Spark Linux with 0 errors; Fern emitted only the existing upgrade notice.
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

Additional validation:

• npx @biomejs/biome check src/lib/onboard/setup-nim-vllm.ts src/lib/onboard/setup-nim-vllm.test.ts src/lib/onboard/setup-nim-flow.ts src/lib/onboard/setup-nim-flow.test.ts src/lib/onboard.ts passed.
• npm run typecheck:cli passed locally after build artifacts were present.
• npm run docs:check-agent-variants and npm run docs:check-routes passed locally; Fern check returned 0 errors.
• DGX Spark runtime evidence: a manually operated vLLM serving Qwen/Qwen3.6-35B-A3B with a very long context and high memory utilization left minimal unified-memory headroom and had prior NVRM NV_ERR_NO_MEMORY logs. Restarting with a smaller envelope allowed a smoke request to complete without freezing, which matches the warning/troubleshooting scope.

---

Signed-off-by: HwangJohn angelic805@gmail.com

Summary by CodeRabbit

• Documentation
  • Added DGX Spark troubleshooting guidance for host freezes / NVRM NV_ERR_NO_MEMORY from risky local vLLM setups, with recovery steps and advice to prefer the managed Spark path.
  • Clarified managed vLLM labeling and behavior when port 8000 is already in use.
• New Features
  • DGX Spark onboarding now warns about high-risk existing local vLLM instances and suppresses the warning for managed vLLM.
  • Managed vLLM onboarding is safer when a server is already running, including Spark-specific handling.
• Tests
  • Expanded coverage for DGX Spark warnings, suppression, managed-vs-existing detection, and port-conflict flows.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds DGX Spark-specific vLLM headroom warnings, managed-container ownership detection, protection against conflicting managed installs, expanded tests, and troubleshooting guidance for memory-pressure failures.

Changes

DGX Spark vLLM onboarding

Layer / File(s)	Summary
Managed vLLM ownership detection src/lib/inference/vllm.ts, src/lib/inference/vllm.test.ts	Labels the canonical vLLM container, detects managed ownership, protects replacement operations, and validates lifecycle behavior.
DGX Spark warning detection and emission src/lib/onboard/setup-nim-vllm.ts, src/lib/onboard/setup-nim-vllm.test.ts	Parses model metadata, identifies large or long-context configurations, emits warnings for unmanaged Spark servers, and covers suppression and threshold behavior.
Managed-install flow and port protection src/lib/onboard/setup-nim-flow.ts, src/lib/onboard.ts, src/lib/onboard/setup-nim-flow.test.ts	Propagates managed-install and Spark state, blocks managed installation when vLLM already occupies the port, and wires ownership detection into onboarding.
DGX Spark diagnostics and recovery docs/inference/set-up-vllm.mdx, docs/reference/troubleshooting.mdx	Documents memory-pressure symptoms, diagnostics, reduced vLLM settings, checkpoint alternatives, and managed-install recovery.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant SetupNim
  participant ExistingVllM
  participant SetupNimVllmHandler
  participant DockerInspector
  Operator->>SetupNim: select vLLM provider
  SetupNim->>ExistingVllM: check port availability
  ExistingVllM-->>SetupNim: return running status
  SetupNim->>SetupNimVllmHandler: pass managedInstall and sparkHost
  SetupNimVllmHandler->>ExistingVllM: read model metadata
  ExistingVllM-->>SetupNimVllmHandler: return model and context settings
  SetupNimVllmHandler->>DockerInspector: inspect managed container label
  DockerInspector-->>SetupNimVllmHandler: return ownership status
  SetupNimVllmHandler-->>Operator: emit headroom warning when applicable

Loading

Suggested labels: platform: dgx-spark, area: providers, bug-fix

Suggested reviewers: cjagwani, ericksoa, jyaunches

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: adding DGX Spark warnings for risky vLLM servers during onboarding.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

docs/inference/set-up-vllm.mdx (1)

45-50: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Cross-link the two DGX Spark headroom write-ups. The new Warning in set-up-vllm.mdx and the new troubleshooting section in troubleshooting.mdx describe the same risk (NVRM NV_ERR_NO_MEMORY, host freeze, same serve-flag reductions) but neither references the other, so a reader landing on one has no signposted path to the other.

• docs/inference/set-up-vllm.mdx#L45-L50: add a route-style link from the Warning (or the "Related Topics" list at lines 157-161) to the troubleshooting dgx-spark anchor for full diagnostics/recovery steps.
• docs/reference/troubleshooting.mdx#L2189-L2223: add a route-style link back to the set-up-vllm.mdx "Use an Existing Server" section for the initial onboarding-time warning context.

Resolve both links against the Fern published routes in docs/index.yml (not filesystem-relative paths) before adding them.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/inference/set-up-vllm.mdx` around lines 45 - 50, Cross-link the DGX
Spark guidance at both sites: in docs/inference/set-up-vllm.mdx lines 45-50, add
a route-style link from the Warning or Related Topics to the troubleshooting
dgx-spark anchor; in docs/reference/troubleshooting.mdx lines 2189-2223, add a
route-style link back to the set-up-vllm.mdx “Use an Existing Server” section.
Resolve both URLs using the Fern routes in docs/index.yml, not
filesystem-relative paths.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@docs/inference/set-up-vllm.mdx`:
- Around line 45-50: Cross-link the DGX Spark guidance at both sites: in
docs/inference/set-up-vllm.mdx lines 45-50, add a route-style link from the
Warning or Related Topics to the troubleshooting dgx-spark anchor; in
docs/reference/troubleshooting.mdx lines 2189-2223, add a route-style link back
to the set-up-vllm.mdx “Use an Existing Server” section. Resolve both URLs using
the Fern routes in docs/index.yml, not filesystem-relative paths.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f567cea9-7ec8-41e6-89f8-ffcd4b09ada8

📥 Commits

Reviewing files that changed from the base of the PR and between 3de1de6 and 7fcc0bf.

📒 Files selected for processing (7)

• docs/inference/set-up-vllm.mdx
• docs/reference/troubleshooting.mdx
• src/lib/onboard.ts
• src/lib/onboard/setup-nim-flow.test.ts
• src/lib/onboard/setup-nim-flow.ts
• src/lib/onboard/setup-nim-vllm.test.ts
• src/lib/onboard/setup-nim-vllm.ts

[coderabbitai]

🧹 Nitpick comments (1)

src/lib/onboard/setup-nim-vllm.ts (1)

32-32: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Large-model detection heuristic has narrow size coverage.

LARGE_MODEL_PATTERN only matches 35b, 70b, 120b, and super, so common large checkpoints (e.g. 34b, 40b, 72b, 180b, 405b) won't trigger the headroom warning, silently defeating the feature's purpose for those models.

♻️ Suggested generalization using a numeric threshold

-const LARGE_MODEL_PATTERN = /(?:^|[-_/])(?:35b|70b|120b|super)(?:$|[-_/])/i;
+const LARGE_MODEL_SIZE_PATTERN = /(?:^|[-_/])(\d+)b(?:$|[-_/])/i;
+const LARGE_MODEL_SIZE_THRESHOLD_B = 30;

Then in buildDgxSparkExistingVllmHeadroomWarning, replace the boolean test with a parsed comparison against LARGE_MODEL_SIZE_THRESHOLD_B (falling back to the super keyword check).

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/setup-nim-vllm.ts` at line 32, Generalize LARGE_MODEL_PATTERN
to recognize numeric model-size segments at or above the chosen threshold while
retaining the super keyword match. Add or use LARGE_MODEL_SIZE_THRESHOLD_B, and
update buildDgxSparkExistingVllmHeadroomWarning to parse the matched numeric
size and compare it against that threshold, falling back to the super check.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/onboard/setup-nim-vllm.ts`:
- Line 32: Generalize LARGE_MODEL_PATTERN to recognize numeric model-size
segments at or above the chosen threshold while retaining the super keyword
match. Add or use LARGE_MODEL_SIZE_THRESHOLD_B, and update
buildDgxSparkExistingVllmHeadroomWarning to parse the matched numeric size and
compare it against that threshold, falling back to the super check.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 185517bc-30dd-48a5-ac4f-f03611a4247a

📥 Commits

Reviewing files that changed from the base of the PR and between 7fcc0bf and d3bcc89.

📒 Files selected for processing (7)

• docs/inference/set-up-vllm.mdx
• docs/reference/troubleshooting.mdx
• src/lib/onboard.ts
• src/lib/onboard/setup-nim-flow.test.ts
• src/lib/onboard/setup-nim-flow.ts
• src/lib/onboard/setup-nim-vllm.test.ts
• src/lib/onboard/setup-nim-vllm.ts

🚧 Files skipped from review as they are similar to previous changes (6)

• src/lib/onboard/setup-nim-flow.ts
• src/lib/onboard.ts
• docs/inference/set-up-vllm.mdx
• src/lib/onboard/setup-nim-flow.test.ts
• src/lib/onboard/setup-nim-vllm.test.ts
• docs/reference/troubleshooting.mdx

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/setup-nim-vllm.test.ts`:
- Around line 147-154: Add a test case alongside the existing large-model
warning test that uses a 30B model and asserts
buildDgxSparkExistingVllmHeadroomWarning includes that model identifier. Keep
the assertion focused on the warning behavior so the inclusive threshold is
covered without locking in implementation details.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 63b4142b-76d5-4718-91fa-feb7d3774fd5

📥 Commits

Reviewing files that changed from the base of the PR and between 787e0bf and 849ebdd.

📒 Files selected for processing (7)

• docs/inference/set-up-vllm.mdx
• docs/reference/troubleshooting.mdx
• src/lib/onboard.ts
• src/lib/onboard/setup-nim-flow.test.ts
• src/lib/onboard/setup-nim-flow.ts
• src/lib/onboard/setup-nim-vllm.test.ts
• src/lib/onboard/setup-nim-vllm.ts

🚧 Files skipped from review as they are similar to previous changes (6)

• src/lib/onboard.ts
• docs/inference/set-up-vllm.mdx
• src/lib/onboard/setup-nim-flow.ts
• docs/reference/troubleshooting.mdx
• src/lib/onboard/setup-nim-flow.test.ts
• src/lib/onboard/setup-nim-vllm.ts

[HwangJohn]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[wscurran]

✨ Thanks for the fix, @HwangJohn. Adding a DGX Spark-specific warning for risky BYO vLLM servers should give operators clearer headroom guidance before memory pressure causes host issues. Ready for maintainer review.

[cjagwani]

Reviewed exact head 3fa3814. The standard checks are green, but the new Spark safety warning has four correctness blockers:

1. src/lib/onboard.ts:1093-1097 uses firmware-only detectNvidiaPlatform(), while the existing GPU path can classify firmware-unknown NVIDIA GB10 as Spark (src/lib/inference/nim.ts:583-613). Thread the already-detected gpu.platform/gpu.spark result into this handler and test empty firmware + GB10 through the real flow.

2. src/lib/onboard/setup-nim-vllm.ts:67-76 returns before evaluating max_model_len unless the model ID first looks large and unquantized. Long context is therefore not an independent trigger; the current NVFP4+262144 test codifies behavior that contradicts both updated docs. Trigger on longContext || riskyLargeModel and add 131071/131072, small-model, and quantized-model long-context boundaries.

3. src/lib/onboard/setup-nim-vllm.ts:32-36,51-69,123-146 treats /v1/models[].id as authoritative size/quantization metadata, but vLLM permits arbitrary served-model aliases and separate quantization configuration. Unknown or misleading aliases can suppress a safety warning, and the warning claims 'unquantized' without proof. Use stronger metadata where available, never treat an alias/suffix as proof of safety, phrase residual classification as heuristic, and test arbitrary aliases, misleading suffixes, root metadata, and config-based quantization.

4. src/lib/onboard/setup-nim-flow.ts:554-590 and provider-host-state.ts:110-115 define 'managed' only for this invocation. A NemoClaw-managed server becomes indistinguishable from BYO on re-onboard, and the remediation env is ignored while a server already occupies the endpoint. Determine ownership durably or remove ownership claims; tell users to stop the existing server before selecting managed install; test managed re-onboard/resume and occupied-port remediation.

After fixes, rerun required cloud-onboard, onboard-repair, and onboard-resume exact-head E2E; add the advised Spark BYO/managed-handoff/re-onboard/routing cleanup scenario.

[prekshivyas]

Addressing cjagwani's four blockers in commit f3554cd (two commits on HEAD). Notes on B3 and B4 where the implementation diverges from the review request:

B3 — alias as authoritative metadata

The QUANTIZED_MODEL_PATTERN suppression is retained. A warning that fires on NVFP4- or INT8-labeled models would be a false positive for the common safe case and would train operators to ignore the warning, defeating the feature. Explicit quantization markers in model IDs are a standard convention (NVIDIA model catalog, HuggingFace) and are a strong signal even if not a cryptographic proof. The concern is addressed in two ways: the code comment above QUANTIZED_MODEL_PATTERN now documents the heuristic nature explicitly, and the warning text now says "heuristically-classified" instead of asserting "unquantized." Long context is now an independent trigger (B2 fix), so even a model with an NVFP4 alias warns if max_model_len is very large — the KV cache exhaustion risk exists regardless of quantization. New tests cover arbitrary aliases and misleading suffixes. If the position is that the suppression must be removed entirely, please say so and we will update.

B4 — managed ownership durability across sessions

Durable ownership persistence (a marker file written on managed install and read on re-onboard) is not added in this PR. The warning is advisory, not a gate. On re-onboard, the operator explicitly selects the vLLM menu entry labeled "— running," signaling awareness that the server is active. A false-positive advisory warning on a NemoClaw-managed re-onboard is less harmful than a false-negative on a risky BYO server with an unrecognized alias. The re-onboard limitation is documented inline at the managedInstall handoff site. The occupied-port case (install-vllm selected while vLLM is already running) now logs a guidance message to stop the existing server first. If durable ownership is a hard requirement, we will scope it as a follow-up — it touches the state persistence layer and warrants its own PR.

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[prekshivyas]

/ok to test 86a63d2

[prekshivyas]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[prekshivyas]

/ok to test ba4511a

[prekshivyas]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[prekshivyas]

@coderabbitai resume

[coderabbitai]

✅ Action performed

Reviews resumed.

[prekshivyas]

@cjagwani ready for exact-head re-review at ba4511a6ff541849b3be4d04a459d0e21de9dfad.

All four requested correctness boundaries are addressed: the real flow threads the already-detected gpu.spark; long context is an independent inclusive trigger; served aliases are never treated as proof of size/quantization and stronger root/config metadata is used conservatively; and NemoClaw ownership is durable via a container label with managed re-onboard plus occupied-port remediation. A final ownership hardening pass also preserves foreign/unlabeled containers, fails closed on ambiguous inspection, rechecks immediately before teardown, and removes only the inspected managed container ID.

Exact-head regular CI is green, CodeRabbit has no unresolved threads, and both Advisor lanes are clean at 0/0/0. Focused ownership/onboarding coverage is 95/95 locally. Trusted E2E dispatch is currently blocked by the base-branch controller crash tracked in #6823; the failing gate executes broken main code.

[prekshivyas]

/ok to test 63cfe98

[prekshivyas]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[prekshivyas]

Exact-head re-review request for 63cfe98298e8246d27cfe0fbb6e92b3aa3a1e160 (base 7b1382924d29097e96aeecf34505c2e247d12c8d).

All four requested correctness areas are now covered: firmware-unknown GB10 uses the already-detected Spark classification; long context is an independent inclusive trigger; aliases are never treated as proof of size/quantization; and managed vLLM ownership is durable with fail-closed occupied-port handling and ID-scoped cleanup. The focused 95-test set and full check:diff pass. Exact-head CI is green, canonical GPT Advisor is 0/0/0, and CodeRabbit has zero unresolved threads.

The remaining machine gate is the expected fork safety exception: credential-bearing E2E jobs cannot execute fork code. Gate run 29305035858 requests a no-secret exception for this exact head/base. @cjagwani, please re-review the addressed findings. @cv @ericksoa, please approve e2e-no-secret-exception or use the typed resolve-fork fallback after reviewing this exact revision.

[prekshivyas]

New exact head after the main-only merge: 18a4eff0e017a67d9beafd0f30e47d59c88e44d1 on base 4f64b0ac19d321c6a85c88799c5675e33c3c1a7e. Standard CI is green, canonical GPT Advisor is 0/0/0, CodeRabbit is green, and no review threads remain. The Nemotron red is an Advisor protocol failure and is being rerun.

The remaining policy gate requires a maintainer fork exception for this exact revision: controller run 29308287855. @cv @ericksoa, please approve the protected environment or run the resolve-fork fallback with this head/base. @cjagwani, the feature diff is unchanged by the main merge; please complete the requested re-review.

Outdated review

[cjagwani]

Approved exact head 14d59fd after maintainer review. This is maintenance of the existing supported DGX Spark local/managed vLLM surface, not a new integration. Required CI and the exact-head/base E2E gate are green; DCO and all commits are Verified; ownership handling fails closed and is ID-scoped; focused onboarding/inference tests cover the risky paths; and no actionable CodeRabbit or advisor finding remains. The failed Terra advisor job is an adjudicated tool-protocol failure with zero published findings and is not a required check.