Skip to content

test(onboard): update sandbox identity fixtures - #9072

Merged
cv merged 2 commits into
mainfrom
codex/update-sandbox-identity-fixtures-9050
Aug 14, 2026
Merged

test(onboard): update sandbox identity fixtures#9072
cv merged 2 commits into
mainfrom
codex/update-sandbox-identity-fixtures-9050

Conversation

@rsliter

@rsliter rsliter commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

After #9064 pinned the durable sandbox identity, several integration harnesses still returned empty structured output for openshell sandbox get. That caused CI to stop before the scenarios reached their intended assertions. This follow-up makes those mocks return the same durable identities their existing capture mocks already advertise.

Related Issue

Follow-up to #9064 for #9050.

Changes

  • Return a durable sandbox ID from the structured runner mocks used by nine onboarding integration fixtures.
  • Preserve each scenario's existing sandbox lifecycle state and identity transitions.
  • Keep the fixtures branchless so the source-shape test budget does not grow.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior; justification:
  • Tests not applicable; justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable; justification: this changes only integration fixture responses. The user-visible behavior and troubleshooting guidance landed in fix(onboard): gate forwarding on recreated readiness #9064.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded; reviewer/approval link/justification:
  • Non-success, skipped, or missing CI check accepted by maintainer; check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: nine onboarding integration fixtures under test/ only; docs/reference/troubleshooting.mdx already owns the behavior through fix(onboard): gate forwarding on recreated readiness #9064
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above: nine affected integration files, 54 tests passed. The test conditional growth scanner passed.
  • Applicable broad gate passed; PR CI is the applicable broad gate. The local full integration lane is not supported on this macOS host and encountered unrelated Linux and Python 3.9 failures before timing out.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Rebecca Sliter 571084+rsliter@users.noreply.github.com

Summary by CodeRabbit

  • Tests
    • Improved onboarding test coverage for sandbox lookup and recreation workflows.
    • Added realistic sandbox identity data to test scenarios, including names and IDs.
    • Strengthened validation of lifecycle handling, deletion and recreation, registration timing, policy preservation, and terminal dashboard behavior.
    • Preserved command tracking and successful default responses across test fixtures.

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
@rsliter rsliter self-assigned this Aug 14, 2026
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: aa08f49c-9f2f-40d8-83dd-2edf688c1c2f

📥 Commits

Reviewing files that changed from the base of the PR and between ca587b2 and 7d751b7.

📒 Files selected for processing (2)
  • test/onboard-messaging.test.ts
  • test/onboard-sandbox-build.test.ts

📝 Walkthrough

Walkthrough

The onboarding test runner mocks now return sandbox names and IDs for matching sandbox get commands. Recreation scenarios preserve command tracking and lifecycle state while providing identity output.

Changes

Onboarding sandbox identity

Layer / File(s) Summary
Identity lookup mock responses
test/onboard-custom-dockerfile.test.ts, test/onboard-extra-provider-reconciliation.test.ts, test/onboard-installer-restore-intent.test.ts, test/onboard-prepared-build-context.test.ts, test/onboard-reservation-recreate.test.ts, test/onboard-terminal-dashboard.test.ts, test/onboard-messaging.test.ts, test/onboard-sandbox-build.test.ts
The test runner mocks return sandbox identity data for matching lookup commands and successful defaults for other commands.
Recreation lifecycle mock behavior
test/onboard-sandbox-recreation.test.ts
Recreation scenarios return sandbox identity data while preserving deletion, creation, download, command-recording, and event-tracking behavior.

Estimated code review effort: 2 (Simple) | ~15 minutes

Mergeability Score: 🟡 Moderate · up to 7d751

This change updates onboarding test fixtures to return durable sandbox identities, and the targeted tests pass. The required broad PR CI gate has not yet passed, so merge should wait for that check or explicit maintainer acceptance.

Possibly related PRs

  • NVIDIA/NemoClaw#8577: Updates onboarding tests for sandbox recreation and per-sandbox Hermes API-port behavior.
  • NVIDIA/NemoClaw#9013: Updates onboarding mocks for sandbox identity and readiness responses.
  • NVIDIA/NemoClaw#9064: Updates onboarding mocks for sandbox identity data in recreation flows.

Suggested labels: area: onboarding, area: sandbox

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the onboarding test fixture updates for sandbox identity.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/update-sandbox-identity-fixtures-9050

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 7d751b7 in the codex/update-sandbox... branch remains at 96%, unchanged from commit 5bb69ed in the main branch.


Updated August 14, 2026 01:20 UTC

@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · medium confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized terminology decisions match; normalized E2E selections differ; severity counts match.
2 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • onboard-repair: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • onboard-resume: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite for the commit under review.

Recommended E2E: None

1 optional E2E recommendation
  • openshell-gateway-upgrade

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
@cv
cv merged commit 91ccacf into main Aug 14, 2026
80 of 87 checks passed
@cv
cv deleted the codex/update-sandbox-identity-fixtures-9050 branch August 14, 2026 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants