Skip to content

fix(connect): run probe-only recovery when absent authority cannot be created - #9285

Merged
prekshivyas merged 1 commit into
mainfrom
fix/9280-probe-only-pristine-recovery
Aug 17, 2026
Merged

fix(connect): run probe-only recovery when absent authority cannot be created#9285
prekshivyas merged 1 commit into
mainfrom
fix/9280-probe-only-pristine-recovery

Conversation

@Dongni-Yang

@Dongni-Yang Dongni-Yang commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

On Ubuntu 24.04, nemoclaw <name> connect --probe-only exits 1 with "no prior launch-readiness evidence can be accepted, but new launch-readiness authority could not be created" before running any preflight or recovery, whenever the launch-readiness authority and receipt are both securely absent and new authority creation fails (e.g. no per-user OS runtime authority in the current session).

That early exit contradicts the shipped contract in two places:

  • Docs (docs/reference/commands.mdx): "A securely absent runtime authority and receipt let ordinary launch run the complete preflight without optimization if new authority creation fails, but connect --probe-only still exits nonzero because it could not publish launch-readiness evidence" — i.e. the preflight runs and only the publication fails.
  • launchSandbox (src/lib/actions/sandbox/launch.ts): refuses only on decision.recoveryBlocked and runs the complete preflight for this same decision.

Change

  • connectSandbox probe-only guard now exits early only when recoveryBlocked is set (a prior epoch might exist and could not be durably rotated). The securely-absent case (fenceFailed with recoveryBlocked === false) proceeds into the launch-readiness mutation gate.
  • Safety is preserved: the gate independently re-verifies via checkLaunchReadinessMutationAuthority(null) that both stores are still missing before admitting the null-epoch recovery; any other state returns changed/unsafe and recovery does not run.
  • After recovery, publication reports honestly: "Probe failed: complete probe and recovery succeeded, but final launch-readiness evidence could not be verified or published." (exit 1, per the documented Linux contract). Lifecycle callers with requireLaunchReadinessPublication: false now continue after recovery instead of failing before it.
  • Rewrote the test that pinned the early exit (connect-flow.test.ts) to assert recovery runs, publication is attempted, and the misleading message is gone.

Why Refs, not Closes

The issue's Expected Result asks for exit 0. The documented contract deliberately keeps connect --probe-only nonzero on Linux when evidence cannot be published, because a missing per-user runtime authority is a deployment signal infrastructure must repair. This PR fixes the defective part — recovery not running and the misleading refusal — and keeps the documented exit code. If maintainers want exit 0 for the securely-absent case (mirroring #9282's treatment of platform-unsupported evidence), that is a small follow-up on top of this change.

With this fix, the reproduction now prints "Probe complete: … restored dashboard port forward." followed by the honest publication failure, instead of refusing to recover at all.

Refs #9280

Testing

  • npx vitest run src/lib/actions/sandbox/connect-flow.test.ts (35 passed)
  • npx vitest run test/cli/connect-recovery.test.ts (5 passed)
  • sibling probe/readiness suites (93 passed), npm run typecheck:cli, oxlint, npm run format:check all clean

Signed-off-by: Dongni Yang dongniy@nvidia.com

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Improved probe-only sandbox connection recovery when launch-readiness checks encounter fencing issues.
    • Recovery and sandbox setup now proceed in eligible cases, with failures reported as unavailable or unverifiable evidence.
    • Removed misleading authority-creation failure reporting.

… created

`connect --probe-only` exited 1 before any preflight or recovery when the
launch-readiness authority and receipt were both securely absent and new
authority creation failed (for example, no per-user OS runtime authority
in the current session). The refusal message claimed authority creation
was the failure, but the command never ran the recovery it exists for.

The documented contract (docs/reference/commands.mdx) says this state
lets ordinary `launch` run the complete preflight and makes probe-only
exit nonzero only because it could not publish evidence afterwards.
`launchSandbox` already implements that: it refuses only when
`recoveryBlocked` is set. Align probe-only with the same rule: exit
early only for a fenceable-prior-evidence failure (`recoveryBlocked`),
and let the securely-absent case enter the mutation gate, which
independently re-verifies that both stores are still missing before
admitting the null-epoch recovery. Publication then reports honestly
that recovery succeeded but evidence could not be published.

Refs #9280

Signed-off-by: Dongni Yang <dongniy@nvidia.com>
@coderabbitai

coderabbitai Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: eb063045-b76c-4403-ba57-da53c8fdafef

📥 Commits

Reviewing files that changed from the base of the PR and between 8b9a28a and 82257e3.

📒 Files selected for processing (2)
  • src/lib/actions/sandbox/connect-flow.test.ts
  • src/lib/actions/sandbox/connect.ts

Included review availability: Your plan includes up to 12 reviews per rolling hour; 10 remain after this review.


📝 Walkthrough

Walkthrough

Probe-only connections now proceed through recovery after non-blocking launch-readiness fence failures. The test verifies live-sandbox setup, evidence-publication attempts, and final unavailable or unverifiable evidence reporting.

Changes

Probe-only recovery

Layer / File(s) Summary
Recovery fencing and evidence validation
src/lib/actions/sandbox/connect.ts, src/lib/actions/sandbox/connect-flow.test.ts
Probe-only recovery stops only when a non-unsupported fence failure sets recoveryBlocked. Other fence failures continue through preflight, recovery, live-sandbox setup, and evidence publication. The test validates the final evidence error.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 82257

This PR adjusts probe-only recovery so securely absent launch-readiness state can proceed through recovery while preserving the documented publication failure behavior. No actionable merge-blocking risk remains beyond normal checks and review.

Possibly related PRs

Suggested labels: v0.0.110

Suggested reviewers: cv, senthilr-nv, apurvvkumaria

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: enabling probe-only recovery when authority creation fails.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/9280-probe-only-pristine-recovery

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 82257e3 in the fix/9280-probe-only-... branch remains at 96%, unchanged from commit 692d1fa in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 82257e3 in the fix/9280-probe-only-... branch is 83%. The coverage in commit 97761a4 in the main branch is 82%.

Show a code coverage summary of the most impacted files.
File main 97761a4 fix/9280-probe-only-... 82257e3 +/-
src/lib/actions...er-lifecycle.ts 94% 85% -9%
src/lib/actions...aged-profile.ts 88% 84% -4%
src/lib/securit...ntial-filter.ts 88% 86% -2%
src/lib/inferen...-preparation.ts 92% 94% +2%
src/lib/onboard...ce-lifecycle.ts 89% 93% +4%
src/lib/trace.ts 90% 94% +4%
src/lib/state/m...ock-identity.ts 90% 94% +4%
src/lib/messagi...ate-resolver.ts 96% 100% +4%
src/lib/messagi...ate-resolver.ts 88% 94% +6%
src/lib/inferen...m-docker-env.ts 66% 73% +7%

Updated August 17, 2026 06:28 UTC

@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · medium confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Failed

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

3 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • established — launch-readiness evidence at src/lib/actions/sandbox/connect.ts:1300: Keep the established term.
  • established — durably rotated at src/lib/actions/sandbox/connect.ts:1289: Keep the established term.
  • replace — prior epoch at src/lib/actions/sandbox/connect.ts:1288: Replace "prior epoch" with "old runtime epoch" to match the established command documentation.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite for the commit under review.

Recommended E2E: None

Manual-only E2E: onboard-repair, onboard-resume
The manual PR workflow does not run these selectors for the commit under review. Run them from reviewed code on main.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@prekshivyas prekshivyas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed latest PR commit 82257e3990820225717e10724e5c5e674fa1432f. The null-epoch mutation gate revalidates secure absence of both authority and the persistent receipt before recovery. The change preserves blocking behavior for recoveryBlocked, and the focused regression plus existing lease-store tests cover the stale-state race. Required CI passes. I found no blocking correctness or security finding.

@prekshivyas
prekshivyas merged commit fda9eb1 into main Aug 17, 2026
72 of 73 checks passed
@prekshivyas
prekshivyas deleted the fix/9280-probe-only-pristine-recovery branch August 17, 2026 06:51
ericksoa pushed a commit that referenced this pull request Aug 18, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Add the canonical dated changelog entry required before planning the
v0.0.110 release. The entry summarizes user-facing changes merged since
v0.0.109 and links each change to its published documentation route and
source PR.

## Changes

- Add `docs/changelog/2026-08-17.mdx` with the exact `## v0.0.110`
release heading.
- Cover managed local inference, endpoint validation, onboarding and
recovery, explicit experimental Portable OpenClaw, messaging and policy
cleanup, backup and security hardening, and release qualification.
- Preserve the documentation skip list and the current supported-agent
matrix; test-only refactors, dormant activation work, and Pi-only
changes are intentionally excluded.

### Source-to-doc mapping

- #8711 -> `docs/changelog/2026-08-17.mdx`: Add the Muse Glimmer
llama.cpp profile.
- #9099 -> `docs/changelog/2026-08-17.mdx`: Update the Muse Glimmer vLLM
runtime.
- #9319 -> `docs/changelog/2026-08-17.mdx`: Select the provider required
by an explicit serving profile.
- #9311 -> `docs/changelog/2026-08-17.mdx`: Report probe-image pull
failures separately.
- #9345 -> `docs/changelog/2026-08-17.mdx`: Reuse mirrored Windows
Ollama.
- #9284 -> `docs/changelog/2026-08-17.mdx`: Complete the required Ollama
upgrade.
- #9320 -> `docs/changelog/2026-08-17.mdx`: Reject unsafe custom
endpoint URLs before mutation.
- #9119 -> `docs/changelog/2026-08-17.mdx`: Reject unsupported custom
endpoint URL components.
- #9236 -> `docs/changelog/2026-08-17.mdx`: Require native Anthropic
tool-use evidence.
- #9347 -> `docs/changelog/2026-08-17.mdx`: Distinguish Gemini runtime
404 diagnostics.
- #9307 -> `docs/changelog/2026-08-17.mdx`: Preserve the recorded API
family when only the model drifts.
- #9233 -> `docs/changelog/2026-08-17.mdx`: Fail incomplete Hermes route
synchronization.
- #9185 -> `docs/changelog/2026-08-17.mdx`: Serialize Model Router
lifecycle work across gateways.
- #9112 -> `docs/changelog/2026-08-17.mdx`: Stop Model Router after the
last routed sandbox is destroyed.
- #9229 -> `docs/changelog/2026-08-17.mdx`: Verify fresh sandbox
execution readiness.
- #9299 -> `docs/changelog/2026-08-17.mdx`: Verify a separate agent API
host forward before reporting ready.
- #9318 -> `docs/changelog/2026-08-17.mdx`: Honor explicit sandbox
recreation.
- #9325 -> `docs/changelog/2026-08-17.mdx`: Measure readiness reuse
windows from collection completion.
- #9352 -> `docs/changelog/2026-08-17.mdx`: Guide users away from the
deprecated global start command.
- #9370 -> `docs/changelog/2026-08-17.mdx`: Persist managed OpenClaw
agent identity.
- #9366 -> `docs/changelog/2026-08-17.mdx`: Pass messaging dependencies
during reused onboarding.
- #9321 -> `docs/changelog/2026-08-17.mdx`: Detect proxied connect
sessions.
- #9285 -> `docs/changelog/2026-08-17.mdx`: Run probe-only recovery when
absent authority cannot be created.
- #9282 -> `docs/changelog/2026-08-17.mdx`: Complete probe-only recovery
without platform evidence.
- #8920 -> `docs/changelog/2026-08-17.mdx`: Preserve legacy gateway
identity.
- #9198 -> `docs/changelog/2026-08-17.mdx`: Report sandbox config-read
failures.
- #9201 -> `docs/changelog/2026-08-17.mdx`: Remove only the exact Docker
orphan on destroy.
- #9176 -> `docs/changelog/2026-08-17.mdx`: Use rootless Podman for
Portable lifecycle operations.
- #9197 -> `docs/changelog/2026-08-17.mdx`: Preflight Portable CPU
delegation.
- #9289 -> `docs/changelog/2026-08-17.mdx`: Narrow Portable policy
defaults.
- #9270 -> `docs/changelog/2026-08-17.mdx`: Preserve Portable model
intent.
- #9339 -> `docs/changelog/2026-08-17.mdx`: Reconcile timed-out Portable
stop state.
- #9209 -> `docs/changelog/2026-08-17.mdx`: Clean receipt-owned Portable
Podman resources.
- #9186 -> `docs/changelog/2026-08-17.mdx`: Separate Podman activation
readiness.
- #9376 -> `docs/changelog/2026-08-17.mdx`: Settle Portable OpenClaw
pairing before readiness.
- #9296 -> `docs/changelog/2026-08-17.mdx`: Retire messaging channel
presets the host no longer configures.
- #9327 -> `docs/changelog/2026-08-17.mdx`: Drop retired channels from
reused messaging selections.
- #9306 -> `docs/changelog/2026-08-17.mdx`: Remove gateway-enforced
presets without a local record.
- #9248 -> `docs/changelog/2026-08-17.mdx`: Activate Google Chat pairing
approval.
- #9374 -> `docs/changelog/2026-08-17.mdx`: Accept schema-owned
messaging plan fields.
- #9317 -> `docs/changelog/2026-08-17.mdx`: Accept safe hard-linked
package files during backup.
- #9288 -> `docs/changelog/2026-08-17.mdx`: Remove managed CLI shims
with destroyed user data.
- #9239 -> `docs/changelog/2026-08-17.mdx`: Read voice credentials from
fixed descriptors.
- #9269 -> `docs/changelog/2026-08-17.mdx`: Accept bounded native
OpenClaw device modes.
- #9371 -> `docs/changelog/2026-08-17.mdx`: Isolate OpenClaw
startup-guard output.
- #9351 -> `docs/changelog/2026-08-17.mdx`: Restore staging Launchable
validation.
- #9350 -> `docs/changelog/2026-08-17.mdx`: Retry transient
collaborator-permission reads.
- #9353 -> `docs/changelog/2026-08-17.mdx`: Retry transient
exact-artifact downloads.
- #9226 -> `docs/changelog/2026-08-17.mdx`: Add bounded Brev readiness
diagnostics.
- #9237 -> `docs/changelog/2026-08-17.mdx`: Report same-commit E2E
reliability.
- #9232 -> `docs/changelog/2026-08-17.mdx`: Execute native-runtime
qualification.
- #9275 -> `docs/changelog/2026-08-17.mdx`: Define E2E selection and
retry guidance.
- #9234 -> `docs/changelog/2026-08-17.mdx`: Move documentation review
after merge.
- #9365 -> `docs/changelog/2026-08-17.mdx`: Mount documentation reviewer
inputs before startup.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [x] Existing tests cover changed behavior — justification:
`test/changelog-docs.test.ts` validates the dated release-entry
contract.
- [ ] Tests not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable; documentation-only change.
- Station profile/scenario: Not applicable.
- Result: Not applicable.
- Supporting evidence: Not applicable.

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — `npx vitest run
test/changelog-docs.test.ts` (7 passed)
- [x] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: Not applicable to one
prose-only changelog page; `npm run docs` passed the repository's strict
documentation gate.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — passed
with 0 errors and the 2 existing Fern warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
— the SPDX header is present; dated changelog pages intentionally do not
use frontmatter.

---
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
  * Added release notes for v0.0.110.
* Documented experimental managed llama.cpp and Portable OpenClaw
profiles.
* Covered inference validation, onboarding and recovery improvements,
rootless lifecycle handling, messaging and policy updates, backups,
credential handling, filesystem protections, and release qualification
updates.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants