Skip to content

test(runtime): preserve workspaces on MicroSandbox compatibility failures - #1475

Merged
skevetter merged 9 commits into
mainfrom
codex/microsandbox-compatibility-parity
Oct 13, 2026
Merged

skevetter merged 9 commits into
mainfrom
codex/microsandbox-compatibility-parity

Conversation

@skevetter

@skevetter skevetter commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Add two actual-VM compatibility scenarios, one for the built-in MicroSandbox provider and one for the published external provider. A test-owned CLI shim injects old, malformed and failing version responses while forwarding all other commands to the supported CLI. Each failed explicit recreation must consume the injected response, issue no mutating runtime command, and preserve the running VM generation, root and source data, workspace record and SSH files.

Restoring the supported CLI must permit explicit recreation, produce a new VM generation and a working SSH session, reset the old root marker and preserve workspace and source identity. Cleanup clears injected faults independently and targets only the fixture's recorded workspace or previously absent backend identity. These fault-injection scenarios do not claim behavior against an actual historical MicroSandbox release or complete migration readiness.

Validation: affected-package uncached race tests, vet, module verification, strict lint with zero issues, all applicable pre-commit hooks, actionlint and formatting/diff checks passed. The branch now integrates the merged safe-recreate/shared MicroSandbox foundation. Compatibility uses its shared session/provider selector and explicit provider labels; its two cases are the thirteenth dedicated CI row, bringing the native catalogue to 74 expected cases. The new selection-only dry run chose both provider cases; it did not start VMs. Fresh full committed local CodeRabbit review completed with zero findings and exact coverage of all three final files. Post-integration race/vet/module verification, strict lint (zero) and all applicable hooks passed.

The previous Windows run completed CLI and E2E builds and artifact uploads, then exceeded the unchanged job deadline while archiving the optional Go cache. Disable that cache only for Windows builds; Linux/macOS caching, all test assertions, catalogue counts and deadlines remain unchanged. Workflow hooks and full committed local review passed for that cache correction; its implementation CI completed with all 84 jobs successful, including all 74 actual MicroSandbox cases.

Greptile identified a valid gap in the no-mutation assertion: image imports through msb load were missing. The final guard includes load. A temporary regression against the actual assertion reproduced the old false acceptance and passed after the change, while read-only version/list/inspect calls remained allowed. Fresh affected-package race/vet, strict lint, all scoped hooks, module verification and full committed local CodeRabbit review (zero findings, all three files) passed. Current-head implementation CI passed all 84 jobs, including all 74 actual MicroSandbox cases (37 built-in and 37 external), with one attempt and no selected skips or dry runs. Fresh exact-head Greptile review completed at 5/5 with no actionable findings. Explicit full remote CodeRabbit review completed while draft with all three PR files covered, no actionable findings and minimal merge risk. Its private-helper docstring warning is dispositioned with source evidence. All actionable threads are resolved; all nine commits have valid GitHub signatures. The built-in provider remains available.

@netlify

netlify Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for devsydev canceled.

Name Link
🔨 Latest commit 5146b2e
🔍 Latest deploy log https://app.netlify.com/projects/devsydev/deploys/6ace81751e961e0b9163c5cb

@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2a95b275-89f0-4c70-ba49-8202ce81fc11
📥 Commits

Reviewing files that changed from the base of the PR and between 229b844 and 5146b2e.

📒 Files selected for processing: 3
  • .github/workflows/pr-ci.yml
  • e2e/tests/up/provider_microsandbox_compatibility.go
  • sites/docs-devsy-sh/content/docs/developing-providers/runtime-protocol.mdx

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

📝 Walkthrough

The pull request adds MicroSandbox compatibility E2E coverage for version-command faults and successful recreation. It also registers the suite in CI, documents its scope and limitations, and changes Go setup caching for Windows runners.

Changes: MicroSandbox compatibility E2E

Layer / File(s) Summary
Scenario, fixture, and CI registration
e2e/tests/up/provider_microsandbox_compatibility.go, .github/workflows/pr-ci.yml
The E2E spec sets up an isolated workspace for builtin and external providers. The CI matrix registers the compatibility suite.
Version faults and state preservation
e2e/tests/up/provider_microsandbox_compatibility.go
The test shim injects old, malformed, or failing msb --version responses. The test checks failure diagnostics, logged commands, and preserved VM, workspace, host-file, and SSH state.
Successful retry and protocol documentation
e2e/tests/up/provider_microsandbox_compatibility.go, sites/docs-devsy-sh/content/docs/developing-providers/runtime-protocol.mdx
The test retries recreation without an injected fault and checks VM replacement and SSH access. The documentation describes the suite, platform coverage, and the limits of the shim.

Windows Go caching

Layer / File(s) Summary
Runner-specific Go caching
.github/workflows/pr-ci.yml
The build workflow disables Go setup caching on Windows and retains it on other runners.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to 5146b

The compatibility tests and CI changes are mergeable after normal checks. The shim does not establish compatibility with an actual historical MicroSandbox release, as the documentation notes.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 1 files. (2 skipped: 2… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely identifies the runtime tests for preserving workspaces during MicroSandbox compatibility failures. It matches the primary changes in the pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 1 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@netlify

netlify Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for images-devsy-sh canceled.

Name Link
🔨 Latest commit 5146b2e
🔍 Latest deploy log https://app.netlify.com/projects/images-devsy-sh/deploys/6ace8175e574c33731f0a365

@skevetter

Copy link
Copy Markdown
Contributor Author

@greptileai review

@greptile-apps

greptile-apps Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the previous image-import check gap is fixed.

Summary

This PR adds two MicroSandbox compatibility tests that check workspace preservation after injected version failures.

  • MicroSandbox compatibility tests check failed recreation and recovery.
  • Windows CLI builds skip Go cache archiving.

Reviews (2) · Last reviewed commit: "test(runtime): reject image imports duri..." · Reviewed by Greptile

Comment thread e2e/tests/up/provider_microsandbox_compatibility.go
@skevetter

Copy link
Copy Markdown
Contributor Author

@greptileai review

@skevetter

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@skevetter

Copy link
Copy Markdown
Contributor Author

Final review disposition: the docstring-coverage warning concerns private Go test fixtures and assertion helpers, not an undocumented public API. The source already documents the non-obvious fault-injection, cleanup, argument-safety and state-preservation contracts, and the runtime protocol documentation states the historical-runtime limitation. Adding comments that restate each private helper would not improve that contract. No source change is needed for this warning. The completed full review covers all three files at 5146b2e and reports no actionable findings and minimal merge risk.

@skevetter
skevetter marked this pull request as ready for review October 13, 2026 20:29
@skevetter
skevetter merged commit e46b15f into main Oct 13, 2026
106 checks passed
@skevetter
skevetter deleted the codex/microsandbox-compatibility-parity branch October 13, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant