From e808e82b74ff444c58161e9125e80a1c9ab99d18 Mon Sep 17 00:00:00 2001 From: Miyoung Choi Date: Wed, 5 Aug 2026 12:12:02 -0700 Subject: [PATCH] docs: consolidate runtime documentation ownership --- .../set-up-vllm-on-two-dgx-sparks.mdx | 9 +-------- docs/manage-sandboxes/backup-restore.mdx | 19 +++++++++++-------- docs/manage-sandboxes/runtime-controls.mdx | 10 ++++++++++ docs/reference/architecture.mdx | 10 ++++++++++ 4 files changed, 32 insertions(+), 16 deletions(-) diff --git a/docs/inference/set-up-vllm-on-two-dgx-sparks.mdx b/docs/inference/set-up-vllm-on-two-dgx-sparks.mdx index 53848323a01..f3e86c1386d 100644 --- a/docs/inference/set-up-vllm-on-two-dgx-sparks.mdx +++ b/docs/inference/set-up-vllm-on-two-dgx-sparks.mdx @@ -169,13 +169,6 @@ The explicit target still must match the exact reciprocal rail topology. ## Understand the Distributed Runtime -The compiled managed inference catalog reads cluster size, profile selection, artifacts, runtime resources, serving arguments, preparation, and readiness checks from preset and recipe YAML. -Maintainers can add another compatible profile that uses the registered managed-cluster adapters by adding preset and recipe YAML. -The YAML `nodeCount` declares the required cardinality. -The TypeScript adapters validate the qualified topology against it and materialize every ranked topology node. -The build compiles those YAML definitions into the packaged runtime catalog automatically. -A profile that needs a different execution contract requires a registered adapter. - The compiled profile pins these artifacts: - The image is `ghcr.io/anemll/dspark-vllm-gx10@sha256:a83948492cf13df455170fb42885f5ef4db54fefe0feff0f841ecbff464ac9d8`. @@ -234,7 +227,7 @@ $$nemoclaw uninstall Full uninstall revalidates every exact container ID before it removes the containers. It then retires the receipt, copied SSH bindings, and exact temporary discovery claims recorded by that receipt. -If revalidation, removal, or safe claim retirement fails, uninstall stops before other cleanup and preserves the receipt for a retry. +If revalidation, removal, or retirement of the exact recorded temporary claims fails, uninstall stops before other cleanup and preserves the receipt for a retry. A gateway-scoped uninstall preserves the host-global cluster, receipt, SSH bindings, and API key while another gateway remains. Cluster cleanup does not delete the Hugging Face model cache. diff --git a/docs/manage-sandboxes/backup-restore.mdx b/docs/manage-sandboxes/backup-restore.mdx index 2d4c8de57f8..a2050906977 100644 --- a/docs/manage-sandboxes/backup-restore.mdx +++ b/docs/manage-sandboxes/backup-restore.mdx @@ -102,14 +102,17 @@ NemoClaw computes versions (`v1`, `v2`, through `vN`) from timestamp order, so ` `snapshot create` requires shields to be down. Snapshot creation and restore share the per-sandbox transition lock with the shields auto-restore timer. -If a timed shields-down window expires during snapshot work, auto-restore closes the per-sandbox lifecycle deadline gate. -The gate blocks new mutations and waits for the recorded live owner to release its exact lock generation. -NemoClaw does not signal the snapshot process. -If ownership becomes ambiguous, NemoClaw attempts to record durable containment and reports exact-generation recovery guidance. -If the containment commit fails, NemoClaw retains any exact lifecycle and deadline gates it already owns. -A state-directory failure that prevented gate publication also prevents normal mutation-lock acquisition. -Correct the reported state-directory write failure, then run `$$nemoclaw shields status` to resume recovery or receive exact-generation recovery guidance. -If the command reports exact-generation recovery guidance, stop all NemoClaw processes for the sandbox, then follow that guidance before you retry the snapshot. +If a timed shields-down window expires during snapshot work, the deadline gate blocks new mutations and waits for the exact snapshot owner to finish without signaling it. +Snapshot work does not bypass recovery for an expired shields-down window. + +Follow [Timed Shields Windows](../configure-sandboxes/understand-runtime-changes#timed-shields-windows) to correct state-directory failures or complete exact-generation recovery before you retry the snapshot. + + +If a state-directory failure prevents gate publication, normal mutation-lock acquisition also fails. +If recovery reports a state-directory failure, correct it and run `$$nemoclaw shields status`. +Durable containment or retained exact gates keep new mutations blocked until you complete exact-generation recovery. +If the command reports exact-generation recovery guidance, stop all NemoClaw processes for the sandbox and follow that guidance before you retry the snapshot. + Tag a snapshot with a human-readable label: diff --git a/docs/manage-sandboxes/runtime-controls.mdx b/docs/manage-sandboxes/runtime-controls.mdx index 8bb3d416720..e13e1417f71 100644 --- a/docs/manage-sandboxes/runtime-controls.mdx +++ b/docs/manage-sandboxes/runtime-controls.mdx @@ -110,31 +110,41 @@ Run `$$nemoclaw shields down` before the change, then restore lockdown wi NemoClaw serializes host-side gateway recovery, config and inference writes, snapshot mutation, sandbox destruction, and shields transitions for each sandbox. When `shields down --timeout` is active, each mutation binds to that exact timer generation so a replaced or expired timer cannot race a later command or a new sandbox that reuses the same name. +### Restore Lockdown After Expiration + If the timeout expires while a mutation is changing sandbox state, auto-restore closes the per-sandbox lifecycle deadline gate. The gate blocks new mutations and waits for the recorded live owner to release its exact lock generation. NemoClaw does not signal that process because portable process inspection cannot prove that all descendants stopped. After the owner releases the lock, auto-restore restores the restrictive policy and configuration posture. The ownership check includes both the process ID and process start identity so PID reuse does not grant control over an unrelated process. +### Complete Exact-Generation Recovery + An interactive command can take over an expired timer. Interactive recovery has separate transition-takeover and restoration phases. Each phase makes up to 7 attempts and waits 5 seconds between failures, for up to 30 seconds of retry delay per phase. Detached recovery uses one 7-attempt budget across deadline setup, main-generation publication, and restoration. The deadline gate remains closed during those attempts. + If restoration cannot commit, NemoClaw attempts to record durable containment. If that containment commit also fails, NemoClaw retains any exact lifecycle and deadline gates it already owns. A state-directory failure that prevented gate publication also prevents normal mutation-lock acquisition. Correct the reported state-directory write failure, then run `$$nemoclaw shields status` to resume recovery or receive exact-generation recovery guidance. When recovery cannot complete, an interactive command returns an error, or the detached timer exits with a failure status. + NemoClaw also records durable containment when an owner exits before it can prove that the owner's descendants stopped, or when ownership becomes ambiguous. Durable containment, retained exact gates, or the fail-closed state-directory error keeps new mutations blocked until you complete exact-generation operator recovery. + Stop all NemoClaw processes for the sandbox, then follow the paths, identities, tokens, and removal order in the reported error. Verify each recorded generation is unchanged, remove only the exact stale generations first, and remove the exact containment generation last. +### Preserve Managed MCP Policy + Before a manual Shields transition replaces a policy, NemoClaw requires exact Model Context Protocol (MCP) agreement among the sandbox registry, generated-policy record, and live gateway policy. `shields down` carries the proven managed MCP policy entries into the relaxed policy. Restoration removes snapshot-time managed MCP entries before it overlays current exact entries. If exact agreement is absent, a manual Shields transition refuses the replacement policy. + At an expired deadline, auto-restore omits unproven managed MCP policy entries, restores lockdown, and records the omission count in its audit entry. An MCP server removed during the shields-down window stays removed. A surviving server keeps its recorded endpoint and address pins while its policy ownership remains exact. diff --git a/docs/reference/architecture.mdx b/docs/reference/architecture.mdx index 508d21c978f..55c5cba4d48 100644 --- a/docs/reference/architecture.mdx +++ b/docs/reference/architecture.mdx @@ -342,6 +342,16 @@ The router selects from the configured pool, then calls the upstream NVIDIA endp Some model and provider combinations need agent-specific compatibility setup. NemoClaw keeps those declarations under `nemoclaw-blueprint/model-specific-setup//` so fixes for each supported agent can be tested and reviewed independently. +### Managed Inference Catalog + +Managed inference profiles use definitions under `managed-inference/presets/` and `managed-inference/recipes/`. +The catalog compiler validates those definitions and includes them in the packaged runtime catalog. + +For a managed-cluster recipe, the `nodeCount` field declares cluster cardinality. +Registered managed-cluster TypeScript adapters validate qualified topology and materialize each ranked node. +Contributors can define a compatible managed-cluster profile that uses an existing execution contract with preset and recipe YAML. +A managed-cluster profile that needs a different execution contract requires a registered adapter. + Refer to [Choose an Inference Provider](../inference/learn-and-choose/choose-inference-provider) for provider configuration details. ## Provider Credential Storage