Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ authors = ["NVIDIA Carbide Engineering <carbide-dev@exchange.nvidia.com>"]
[workspace.dependencies]
clap = { version = "4", features = ["derive", "env"] }
libredfish = { git = "https://github.com/NVIDIA/libredfish.git", tag = "v0.44.22" }
librms = { git = "https://github.com/NVIDIA/nv-rms-client.git", tag = "v0.9.1" }
librms = { git = "https://github.com/NVIDIA/nv-rms-client.git", tag = "v0.10.0" }
ansi-to-html = "0.2.2"

tokio = { version = "1", features = ["full", "tracing"] }
Expand Down
64 changes: 35 additions & 29 deletions book/src/configuration/configurability.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,20 +190,32 @@ explicitly enabled in the TOML.
| `[machine_identity]` | SPIFFE JWT-SVID issuance for machine (host) identity | Per-org JWT signing. See [Day 0 Machine Identity](../../../docs/getting-started/installation-options/day0-machine-identity.md) and [Machine Identity (Day 1)](../../../docs/configuration/machine_identity.md). |
| `[measured_boot_collector]` | TPM-based attestation metrics | |
| `[machine_validation_config]` | Pre-ingestion validation tests | |
| `[component_manager]` | Compute tray, NvLink switch, and power shelf management | RMS backends require rack profile data for node type resolution. |
| `[component_manager]` | Compute tray, NvLink switch, and power shelf management | RMS backends require rack profile data for node descriptors. |
| `[vmaas_config]` | VM system integration / VM-aware traffic intercept | Requires `public_prefixes`. |
| `[rms]` | Rack Manager Service (mTLS connectivity to external RMS) | |
| `[dpf]` | DPU Platform Framework — Kubernetes DPU workload deployment | Requires the DPF operator deployed in-cluster. |
| `rack_management_enabled` | Standalone infrastructure manager mode (GB200/GB300/VR144) | Top-level boolean, not a sub-section. |

For RMS component-manager backends, NICo resolves the RMS node type from the
rack profile. The rack profile provides two facts:
For RMS component-manager backends, NICo builds RMS node descriptors from rack
profiles. Each descriptor contains three attributes:

- Product family from `product_family`, which is required for RMS-backed
operations and currently accepts `gb200` or `gb300`.
- Role from the component-manager operation: `compute`, `switch`, or
`power_shelf`.
- Product family from `product_family`, which must be non-empty for RMS-backed
operations. NICo passes other non-empty product-family identifiers to RMS
without a local hardware mapping.
- Vendor from `rack_capabilities.<role>.vendor` for each role using an RMS
backend.

NICo always sends these attributes in descriptor-based RMS requests. For exact
role, vendor, and product-family combinations represented by the current RMS
`NodeType` enum, NICo also sends that enum and legacy firmware-filter entries
for compatibility with older RMS servers. Other combinations leave `NodeType`
unset and require RMS support for `NodeDescriptor`. This best-effort legacy
mapping does not participate in startup validation. In particular, VRNVL72
power shelves use their configured VRNVL72 descriptor because no matching
legacy `NodeType` exists.

NICo validates configured rack profiles at startup when any component-manager
backend is set to `rms`. The component-manager backend fields default to `rms`,
so deployments that only want one RMS role must explicitly set the other backend
Expand All @@ -213,22 +225,16 @@ the vendor fields for enabled RMS roles. For example, if only
values, then only `rack_capabilities.power_shelf.vendor` is required as a vendor
field.

Use these canonical vendor names in config:
NICo trims outer whitespace from `product_family` and vendor values and requires
both to be non-empty. It does not validate either value against a fixed list.
RMS determines whether each role/vendor/product-family combination is supported
when a request is made. See
[Supported RMS descriptor combinations](../../../docs/configuration/component-manager-rms.md#supported-rms-descriptor-combinations),
including VRNVL72.

| Role | Canonical values |
| --- | --- |
| Compute, when `compute_tray_backend = "rms"` | `NVIDIA`, `Lenovo` |
| Switch, when `nv_switch_backend = "rms"` | `NVIDIA` |
| Power shelf, when `power_shelf_backend = "rms"` | `LiteOn`, `Delta` |

The `product_family` value is not normalized. It must exactly match one of the
accepted lowercase values, such as `gb200` or `gb300`; values like `GB200` are
rejected. Vendor matching is more forgiving. Vendor values are trimmed,
case-insensitive, and ignore spaces, hyphens, and underscores, so `NVIDIA`,
`nvidia`, `LiteOn`, `liteon`, `Lite-On`, and `lite_on` all work. Common company
suffix text also works when the normalized value starts with the canonical
vendor, but the canonical values above are preferred for operator-supplied
config.
For product families other than `gb200` and `gb300`, the `GetRackProfile`
`product_family` enum is `UNSPECIFIED`. The configured string remains available
to descriptor-based RMS operations.

The examples below only show the component-manager and rack-profile fields.
Configure `[rms]` separately when NICo needs to call RMS.
Expand Down Expand Up @@ -306,16 +312,16 @@ still checked when an RMS operation runs.

| Field | Accepted values |
| --- | --- |
| `product_family`, when an RMS-backed operation uses the profile | Exact match: `gb200`, `gb300` |
| `rack_hardware_topology` | `gb200_nvl36r1_c2g4_topology`, `gb200_nvl72r1_c2g4_topology`, `gb300_nvl36r1_c2g4_topology`, `gb300_nvl72r1_c2g4_topology` |
| Compute profile vendor, when `compute_tray_backend = "rms"` | `nvidia`, `lenovo` after normalization |
| Switch profile vendor, when `nv_switch_backend = "rms"` | `nvidia` after normalization |
| Power shelf profile vendor, when `power_shelf_backend = "rms"` | `liteon`, `delta` after normalization |
| `product_family`, when an RMS-backed operation uses the profile | Non-empty string; RMS validates support at request time |
| `rack_hardware_topology` | `gb200_nvl36r1_c2g4_topology`, `gb200_nvl72r1_c2g4_topology`, `gb300_nvl36r1_c2g4_topology`, `gb300_nvl72r1_c2g4_topology`, `vr_nvl8r1_c2g4_rtf_topology`, `vr_nvl72r1_c2g4_topology` |
| Compute profile vendor, when `compute_tray_backend = "rms"` | Non-empty string; RMS validates support at request time |
| Switch profile vendor, when `nv_switch_backend = "rms"` | Non-empty string; RMS validates support at request time |
| Power shelf profile vendor, when `power_shelf_backend = "rms"` | Non-empty string; RMS validates support at request time |

The separate site-explorer machine-ingestion RMS slot/tray lookup also uses the
rack profile for RMS node type resolution. If that path is enabled for machines
with rack IDs, the profile also needs compute product-family and vendor data even
when `compute_tray_backend` is not `rms`.
rack profile to build a compute node descriptor. If that path is enabled for
machines with rack IDs, the profile also needs compute product-family and vendor
data even when `compute_tray_backend` is not `rms`.

### State-controller timing

Expand Down Expand Up @@ -1226,7 +1232,7 @@ on or off.
| Site Explorer machine auto-creation | siteConfig | `[site_explorer].create_machines` | on | Disable for manual-onboarding environments. |
| Site Explorer switch / power shelf auto-creation | siteConfig | `[site_explorer].create_switches` / `[site_explorer].create_power_shelves` | on | Ingests only declared hardware (`expected_switches` / `expected_power_shelves` records). Disable to pause switch or power shelf ingestion site-wide. |
| Firmware autoupdate | siteConfig | `[firmware_global].autoupdate` | off | Enable once the fleet's firmware baseline is stable. |
| Component Manager (compute trays / NvLink switches / power shelves) | siteConfig | `[component_manager]` present | off | GB200/GB300 sites with managed compute, power, and switch fabric. RMS backends require rack profile data for node type resolution. |
| Component Manager (compute trays / NvLink switches / power shelves) | siteConfig | `[component_manager]` present | off | GB200/GB300 sites with managed compute, power, and switch fabric. RMS backends require rack profile data for node descriptors. |
| Auto-repair plugin | siteConfig | `[auto_machine_repair_plugin]` | off | Enable per fault class as fleet maturity grows. |
| BOM / SKU validation | siteConfig | `[bom_validation]` present | off | Validate ingested hardware against expected BOM before `Ready`. |
| Network Security Groups | siteConfig | `[network_security_group]` | default | Touch only for non-default direction policy or scale-out limits. |
Expand Down
63 changes: 34 additions & 29 deletions crates/api-core/src/cfg/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,16 +122,28 @@ applicable.

---

### Component Manager RMS Node Type Resolution
### Component Manager RMS Node Descriptors

When `[component_manager]` uses RMS backends, NICo resolves RMS node types from
rack profiles. The rack profile provides two facts:
When `[component_manager]` uses RMS backends, NICo builds RMS node descriptors
from rack profiles. Each descriptor contains three attributes:

- Product family from `product_family`, which is required for RMS-backed
operations and currently accepts `gb200` or `gb300`.
- Role from the component-manager operation: `compute`, `switch`, or
`power_shelf`.
- Product family from `product_family`, which must be non-empty for RMS-backed
operations. NICo passes other non-empty product-family identifiers to RMS
without a local hardware mapping.
- Vendor from `rack_capabilities.<role>.vendor` for each role using an RMS
backend.

NICo always sends these attributes in descriptor-based RMS requests. For exact
role, vendor, and product-family combinations represented by the current RMS
`NodeType` enum, NICo also sends that enum and legacy firmware-filter entries
for compatibility with older RMS servers. Other combinations leave `NodeType`
unset and require RMS support for `NodeDescriptor`. This best-effort legacy
mapping does not participate in startup validation. In particular, VRNVL72
power shelves use their configured VRNVL72 descriptor because no matching
legacy `NodeType` exists.

NICo validates configured rack profiles at startup when any component-manager
backend is set to `rms`. The component-manager backend fields default to `rms`,
so deployments that only want one RMS role must explicitly set the other backend
Expand All @@ -141,22 +153,16 @@ the vendor fields for enabled RMS roles. For example, if only
values, then only `rack_capabilities.power_shelf.vendor` is required as a vendor
field.

Use these canonical vendor names in config:

| Role | Canonical values |
|------|------------------|
| Compute, when `compute_tray_backend = "rms"` | `NVIDIA`, `Lenovo` |
| Switch, when `nv_switch_backend = "rms"` | `NVIDIA` |
| Power shelf, when `power_shelf_backend = "rms"` | `LiteOn`, `Delta` |
NICo trims outer whitespace from `product_family` and vendor values and requires
both to be non-empty. It does not validate either value against a fixed list.
RMS determines whether each role/vendor/product-family combination is supported
when a request is made. See
[Supported RMS descriptor combinations](../../../../docs/configuration/component-manager-rms.md#supported-rms-descriptor-combinations),
including VRNVL72.

The `product_family` value is not normalized. It must exactly match one of the
accepted lowercase values, such as `gb200` or `gb300`; values like `GB200` are
rejected. Vendor matching is more forgiving. Vendor values are trimmed,
case-insensitive, and ignore spaces, hyphens, and underscores, so `NVIDIA`,
`nvidia`, `LiteOn`, `liteon`, `Lite-On`, and `lite_on` all work. Common company
suffix text also works when the normalized value starts with the canonical
vendor, but the canonical values above are preferred for operator-supplied
config.
For product families other than `gb200` and `gb300`, the `GetRackProfile`
`product_family` enum is `UNSPECIFIED`. The configured string remains available
to descriptor-based RMS operations.

The examples below only show the component-manager and rack-profile fields.
Configure `[rms]` separately when NICo needs to call RMS.
Expand Down Expand Up @@ -206,9 +212,8 @@ vendor = "delta"
```

Example: only the component-manager power shelf backend uses RMS. The compute
and switch component-manager backends are explicitly set to real non-RMS values
so component-manager startup validation only requires the power shelf vendor
field:
and switch component-manager backends are explicitly set to non-RMS values, so
component-manager startup validation only requires the power shelf vendor field:

```toml
[component_manager]
Expand All @@ -233,13 +238,13 @@ existing rack database rows, so missing or unknown per-rack profile IDs are
still checked when an RMS operation runs.

The separate site-explorer machine-ingestion RMS slot/tray lookup also uses the
rack profile for RMS node type resolution. If that path is enabled for machines
with rack IDs, the profile also needs compute product-family and vendor data even
when `compute_tray_backend` is not `rms`.
rack profile to build a compute node descriptor. If that path is enabled for
machines with rack IDs, the profile also needs compute product-family and vendor
data even when `compute_tray_backend` is not `rms`.

Supported RMS product-family values are exact-match `gb200` and `gb300`. The
optional `rack_hardware_topology` field remains available for topology-specific
flows.
NICo accepts non-empty product-family strings. RMS evaluates descriptor support
when an operation runs. The optional `rack_hardware_topology` field remains
available for topology-specific flows.

---

Expand Down
5 changes: 3 additions & 2 deletions crates/api-core/src/setup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2043,8 +2043,9 @@ mod tests {
let error = format!("{error:?}");

assert!(
error.contains("rack_capabilities.switch.vendor"),
"error message should name the missing vendor field: {error}"
error.contains("rack profile NVL72 cannot build RMS switch node descriptor")
&& error.contains("rack profile does not identify an RMS switch vendor"),
"error message should identify the rack profile and missing role vendor: {error}"
);

Ok(())
Expand Down
45 changes: 10 additions & 35 deletions crates/api-core/src/tests/rack_state_controller/handler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2435,14 +2435,8 @@ async fn test_configure_nmx_cluster_disable_scale_up_fabric_state_runs_on_all_sw
.as_slice();

assert_eq!(devices.len(), switch_ids.len());
for device in devices {
let host_endpoint = device
.host_endpoint
.as_ref()
.ok_or_else(|| eyre::eyre!("disable request should include host endpoints"))?;
assert!(devices.iter().all(|device| device.host_endpoint.is_some()));

assert!(host_endpoint.dangerously_accept_invalid_certs);
}
let node_ids = devices
.iter()
.map(|device| device.node_id.clone())
Expand Down Expand Up @@ -2603,13 +2597,7 @@ async fn test_configure_nmx_cluster_configure_selects_persists_and_configures_pr
.ok_or_else(|| eyre::eyre!("configure request should include a primary switch"))?;

assert_eq!(configure_node.node_id, primary_switch_id.to_string());
assert!(
configure_node
.host_endpoint
.as_ref()
.ok_or_else(|| eyre::eyre!("configure request should include a host endpoint"))?
.dangerously_accept_invalid_certs
);
assert!(configure_node.host_endpoint.is_some());

let mut txn = pool.acquire().await?;
let primary_switch = db_switch::find_by_id(&mut txn, &primary_switch_id)
Expand Down Expand Up @@ -2919,14 +2907,13 @@ async fn test_configure_nmx_cluster_runs_start_disable_configure_to_wait_for_fab
.as_slice();

assert_eq!(disable_devices.len(), switch_ids.len());
for device in disable_devices {
let host_endpoint = device
.host_endpoint
.as_ref()
.ok_or_else(|| eyre::eyre!("disable request should include host endpoints"))?;

assert!(host_endpoint.dangerously_accept_invalid_certs);
}
assert!(
disable_devices
.iter()
.all(|device| device.host_endpoint.is_some())
);

let disabled_node_ids = disable_devices
.iter()
.map(|device| device.node_id.clone())
Expand Down Expand Up @@ -2999,13 +2986,7 @@ async fn test_configure_nmx_cluster_runs_start_disable_configure_to_wait_for_fab
.ok_or_else(|| eyre::eyre!("configure request should include a primary switch"))?;

assert_eq!(configure_node.node_id, primary_switch_id.to_string());
assert!(
configure_node
.host_endpoint
.as_ref()
.ok_or_else(|| eyre::eyre!("configure request should include a host endpoint"))?
.dangerously_accept_invalid_certs
);
assert!(configure_node.host_endpoint.is_some());

let mut txn = pool.acquire().await?;
let primary_switch = db_switch::find_by_id(&mut txn, &primary_switch_id)
Expand Down Expand Up @@ -3359,13 +3340,7 @@ async fn test_configure_nmx_cluster_configure_failure_advances_to_wait_for_fabri
.ok_or_else(|| eyre::eyre!("configure request should include a primary switch"))?;

assert_eq!(configure_node.node_id, primary_switch_id.to_string());
assert!(
configure_node
.host_endpoint
.as_ref()
.ok_or_else(|| eyre::eyre!("configure request should include a host endpoint"))?
.dangerously_accept_invalid_certs
);
assert!(configure_node.host_endpoint.is_some());

let mut txn = pool.acquire().await?;
let primary_switch = db_switch::find_by_id(&mut txn, &primary_switch_id)
Expand Down
2 changes: 1 addition & 1 deletion crates/api-db/src/machine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2795,7 +2795,7 @@ impl<'r> FromRow<'r, PgRow> for _HealthReportWrapper {
}

/// RMS identity for a compute tray machine, including rack profile context for
/// node type resolution.
/// node descriptor construction.
#[derive(Debug, sqlx::FromRow)]
pub struct MachineRmsIdentity {
pub id: String,
Expand Down
4 changes: 2 additions & 2 deletions crates/api-db/src/power_shelf.rs
Original file line number Diff line number Diff line change
Expand Up @@ -523,8 +523,8 @@ pub async fn find_ids_by_bmc_macs(
.map_err(|err| DatabaseError::new("power_shelf::find_ids_by_bmc_macs", err))
}

/// RMS identity for a power shelf, including rack profile context for node type
/// resolution.
/// RMS identity for a power shelf, including rack profile context for node
/// descriptor construction.
#[derive(Debug, sqlx::FromRow)]
pub struct PowerShelfRmsIdentity {
pub id: String,
Expand Down
4 changes: 2 additions & 2 deletions crates/api-db/src/switch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -784,8 +784,8 @@ pub async fn find_ids_by_bmc_macs(
.map_err(|err| DatabaseError::new("switch::find_ids_by_bmc_macs", err))
}

/// RMS identity for a switch, including rack profile context for node type
/// resolution.
/// RMS identity for a switch, including rack profile context for node descriptor
/// construction.
#[derive(Debug, sqlx::FromRow)]
pub struct SwitchRmsIdentity {
pub id: String,
Expand Down
Loading
Loading