test(tmachine): add Docker CDI injection coverage and local DEB builds by elezar · Pull Request #4461 · NVIDIA/OpenShell

elezar · 2026-10-13T10:33:35Z

Summary

Add hardware-free Docker CDI injection coverage to tmachine and build local Debian artifacts for package-installed qualification. The Docker environment recreates dummy YAML CDI specs on each boot, and the feature suite checks environment and mount injection through the installed OpenShell CLI.

Stacked on #4529 (codex/cdi-name-parser). PR #4204 builds on this test harness.

Related Issue

Related to #4460. This PR supplies the initial CDI test infrastructure; the issue remains open to track follow-up coverage.

Changes

Build the CLI, gateway, prover, VM init, and VM driver with its embedded Linux runtime in build-artifacts-binaries; package staged inputs with build-artifacts-deb and include both steps in build-artifacts.
Refresh the verified Linux VM runtime archive checksums.
Generate two fake NVIDIA CDI devices plus an unrelated vendor spec, using file mounts and TMACHINE_CDI_* environment markers. No GPU hardware or device nodes are required.
Add eight tests for explicit/default/count selection, absence of injection without a GPU request, and invalid requests. Keep the existing reserved OPENSHELL_* environment behavior unchanged.
Publish installed-gateway test context and temporarily enable caller device selection in a qualification TOML, restoring it after the tests. Retain the package-owned service and client authentication.
Add Docker CDI coverage to branch and release feature-test matrices and document local commands.
Add ten explicitly selected tmachine.example/cdi-policy devices for read-only and writable mounts, library parents, device nodes, supplementary GIDs, and Unix sockets. Keep them outside the automatic GPU inventory. Supply persistent source data and a reboot-restored echo socket; these fixtures do not yet have scenario-specific assertions in the eight-test suite.

Testing

Checks appropriate to the affected code and behavior pass: Nix formatting/parsing, CDI Clippy and Rust formatting, Ansible syntax, and both commits' repository pre-commit hooks.
Eight integration tests added; all eight passed with nix run .#tmachine -- test ubuntu-docker-rootful deb cdi on local ARM64 using freshly built DEB and matching runtime images.
Separate build-artifacts-binaries and build-artifacts-deb commands passed; DEB packaging ran without invoking Cargo. Package architecture, version, and payload inspected.
test-packaging-assets.sh passed with Debian tooling available.

Local executable validation was ARM64; the x86_64 builder was evaluated but not executed locally. Real GPU execution, Podman, and Kubernetes CDI coverage are outside this change. Synthetic device-node fixtures are included; device-node behavior assertions remain follow-up work.

Checklist

Follows Conventional Commits
Commits are signed off (DCO)
Testing documentation updated; runtime architecture is unchanged

copy-pr-bot · 2026-10-13T10:33:40Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

github-actions · 2026-10-13T10:34:12Z

🌿 Preview your docs: https://nvidia-preview-pr-4461.docs.buildwithfern.com/openshell

github-actions · 2026-10-15T12:12:40Z

Compatibility review (advisory)

Needs review

No source-level SDK, CLI, configuration, policy, Helm, or state contract changes were found. Qualification remains incomplete because release VM drivers embed newly hashed Linux runtime archives whose semantic delta is unavailable in the supplied source.

Not inspected: api-behavior

1. api-behavior — NOT ACKED

The pinned x86_64 and aarch64 Linux VM runtime archive hashes changed, but the archive contents and semantic differences are not present in the supplied source.

Impact: Consumers of Linux VM sandboxes may receive changed libkrun, firmware, or umoci behavior that cannot be checked for compatibility from this repository.
Action: Provide an immutable provenance or source diff for both replacement archives and confirm runtime compatibility, or restore the baseline archives for this train.
PR: test(tmachine): add Docker CDI injection coverage and local DEB builds #4461

Review details

Execution: incomplete
Candidate: PR #4461 (782cf8bdcab7138914bfdd859ae6a3f904c4ead6)
Baseline: 360f6d5cc2ad707bf84cbcda239e08086a527cac (360f6d5cc2ad707bf84cbcda239e08086a527cac)
Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

SDAChess

Some questions, I'm surprised there are changes related to the VM driver, are these on purpose?

SDAChess · 2026-10-15T12:16:12Z

+const PROBE: &str = r#"
+set -eu
+test "${TMACHINE_CDI_SPEC_0:-}" = "${TMACHINE_CDI_DEVICE_0:+fixture}"
+test "${TMACHINE_CDI_SPEC_1:-}" = "${TMACHINE_CDI_DEVICE_1:+fixture}"
+for index in 0 1; do
+    eval "selected=\${TMACHINE_CDI_DEVICE_${index}:-}"
+    mount=/usr/share/openshell-cdi/device-$index
+    if [ "$selected" = "$index" ]; then
+        test "$(cat "$mount")" = "tmachine-cdi-$index"
+        printf 'device=%s\n' "$index"
+    else
+        test -z "$selected"
+        test ! -e "$mount"
+    fi
+done
+test -z "${TMACHINE_CDI_UNRELATED:-}"
+"#;


Let's be careful here about portability of this test. Not sure if running bash here is a good idea.

SDAChess · 2026-10-15T12:17:04Z

+async fn no_gpu_request_has_no_cdi_edits() {
+    check_case("cdi-none", None, None, r#"
+set -eu
+test -z "${TMACHINE_CDI_SPEC_0:-}${TMACHINE_CDI_SPEC_1:-}${TMACHINE_CDI_DEVICE_0:-}${TMACHINE_CDI_DEVICE_1:-}${TMACHINE_CDI_UNRELATED:-}"
+for index in 0 1; do
+    test ! -e /usr/share/openshell-cdi/device-$index
+done
+"#, Some(&[]), None).await;
+}


If there is no other way to check than to run some bash code then let's do that, but I would feel better about having these checks be Rust code.

SDAChess · 2026-10-15T12:19:06Z

      aarch64-linux = {
        platform = "linux-aarch64";
-        hash = "sha256-7FNqUtC6ixfw202EyoYCgZOLdyxJYfg/YzdlkTT/+sM=";
+        hash = "sha256-Yk5Rxn6JWxMu82Vkyx3eSAxh5/mnfUwcNDNeDIQJz4c=";


I wonder why this was updated. Did we push an update that was not used before? Interesting that we are not testing this anywhere in the CI, I guess we are now.

Created #4600 to track. The vm-runtime was added here because the debian package includes the VM driver.

This PR adds building debian which rebuilds the VM driver and that triggers this refresh. Similarly in #4598

Have the openshell client role publish the installed gateway's name, active TOML path, service scope, service user, and network, and add an openshell_test_gateway role that loads that context, resolves the CLI, restarts the gateway service, and collects its journal. A feature suite can then change the gateway's active configuration for its own run and restore it afterwards. Select the candidate runtime images for packaged installs through a qualification gateway.toml referenced by OPENSHELL_GATEWAY_CONFIG instead of OPENSHELL_* variables in the service environment file, so the gateway has a TOML file a suite can modify. Update the packaging asset check to match. Signed-off-by: Evan Lezar <elezar@nvidia.com>

Define each CDI scenario once in vars/scenarios.yaml: the host fixtures it needs (directories, files, a character device, a socket), the CDI edits that reference them, and the behavior the workload is expected to observe. The cdi_fixtures role creates the fixtures, renders one tmachine.example/cdi-scenario spec with a device per scenario, and publishes the expectations as scenarios.json. Install the spec into /var/run/cdi from systemd-tmpfiles on every boot so it survives tmachine's setup, installation, and test disk layers, and run the role as part of the ubuntu-docker-rootful setup. Signed-off-by: Evan Lezar <elezar@nvidia.com>

Add a cdi feature suite that creates a sandbox for each CDI scenario by selecting its device explicitly with --gpu 1 and the Docker cdi_devices driver option, then checks what the workload observes or that creation is rejected. The sandbox only reports facts; the test makes every assertion, so a failure names the check that disagreed. Each scenario is its own test. The expectations record the behavior the sandbox has today, loaded from the scenarios.json published by the fixtures role. A OPENSHELL_CDI_CAPTURE mode prints observations instead of asserting, to record that behavior for a new scenario. The test playbook enables caller driver configuration in the installed gateway's active TOML for the run and restores it afterwards. Build the suite's nextest archive in tests/artifacts.nix, expose it as build-artifacts-cdi-tests, and run the suite on the Docker environment with the deb installer in the branch and release matrices. Signed-off-by: Evan Lezar <elezar@nvidia.com>

elezar mentioned this pull request Oct 13, 2026

test(tmachine): add hardware-free Docker CDI qualification #4460

Open

13 tasks

elezar changed the base branch from main to codex/cdi-name-parser October 14, 2026 14:39

elezar mentioned this pull request Oct 14, 2026

feat(gpu): derive Docker sandbox requirements from CDI specs #4204

Draft

9 of 13 tasks

elezar marked this pull request as ready for review October 15, 2026 12:09

elezar requested review from a team, derekwaynecarr, mrunalp and sjenning as code owners October 15, 2026 12:09

elezar mentioned this pull request Oct 15, 2026

feat(tmachine): build local Debian package artifacts #4597

Closed

5 tasks

SDAChess reviewed Oct 15, 2026

View reviewed changes

elezar force-pushed the test/4460-tmachine-cdi/elezar branch from 844e44f to e9fa277 Compare October 15, 2026 14:51

elezar added 3 commits October 15, 2026 16:58

elezar force-pushed the codex/cdi-name-parser branch from 360f6d5 to 01b0b55 Compare October 15, 2026 15:01

elezar force-pushed the test/4460-tmachine-cdi/elezar branch from e9fa277 to fb39ac1 Compare October 15, 2026 15:01

Navigation Menu

Repository navigation

test(tmachine): add Docker CDI injection coverage and local DEB builds - #4461

test(tmachine): add Docker CDI injection coverage and local DEB builds#4461
elezar wants to merge 3 commits into
codex/cdi-name-parserfrom
test/4460-tmachine-cdi/elezar

elezar commented Oct 13, 2026 •

edited

Loading

Uh oh!

copy-pr-bot Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

SDAChess left a comment

Uh oh!

SDAChess Oct 15, 2026

Uh oh!

SDAChess Oct 15, 2026

Uh oh!

SDAChess Oct 15, 2026

Uh oh!

elezar Oct 15, 2026

Uh oh!

elezar Oct 15, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

elezar commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issue

Changes

Testing

Checklist

Uh oh!

copy-pr-bot Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Compatibility review (advisory)

1. api-behavior — NOT ACKED

Uh oh!

SDAChess left a comment

Choose a reason for hiding this comment

Uh oh!

SDAChess Oct 15, 2026

Choose a reason for hiding this comment

Uh oh!

SDAChess Oct 15, 2026

Choose a reason for hiding this comment

Uh oh!

SDAChess Oct 15, 2026

Choose a reason for hiding this comment

Uh oh!

elezar Oct 15, 2026

Choose a reason for hiding this comment

Uh oh!

elezar Oct 15, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

elezar commented Oct 13, 2026 •

edited

Loading