fix(ci): bind E2E gate to exact PR diff#6788
Merged
Merged
Conversation
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage remains at 96%, unchanged from the branch. TypeScript / code-coverage/cliThe overall coverage in the branch remains at 79%, unchanged from the branch. Show a code coverage summary of the most impacted files.
Updated |
Contributor
📝 WalkthroughWalkthroughThe PR gate changes from head-only identity to exact head/base identity, adds trusted workflow SHA validation, supports base-changing PR events, strengthens safe dispatch and exception resolution, and expands controller lifecycle and workflow-boundary tests. ChangesExact-diff PR E2E gate
Estimated code review effort: 5 (Critical) | ~120 minutes Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
PR Review Advisor — InformationalAdvisor assessment: Informational / low confidence E2E guidanceAdvisory only: coverage and selector recommendations are non-authoritative. E2E / PR Gate independently computes and dispatches trusted jobs without consuming this output. Recommended coverage:
This is an automated, non-authoritative review. Findings are inputs to maintainer adjudication. Warnings and optional suggestions do not require a response or follow-up. A human maintainer makes the final merge decision. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
test/pr-review-advisor-workflow-boundary.test.ts (1)
255-263: 🔒 Security & Privacy | 🔵 Trivial | 🏗️ Heavy liftTest the edited-event behavior semantically.
These assertions only check that each condition contains two text fragments. A future operator/grouping mistake could still pass while allowing metadata-only
editedevents to run or publish. Add behavioral fixtures for base-changing and metadata-only edits, and assert the resulting eligibility/concurrency decisions rather than matching source substrings.As per path instructions, tests should prefer observable behavioral outcomes over source-text or implementation-shape assertions.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/pr-review-advisor-workflow-boundary.test.ts` around lines 255 - 263, Replace the substring-based assertions in the edited-event test with behavioral fixtures covering base-changing and metadata-only pull request edits. Exercise the workflow eligibility and concurrency decisions for each fixture, asserting that base changes proceed while metadata-only edits do not run or publish; retain only source checks that are necessary for observable behavior.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@test/pr-review-advisor-workflow-boundary.test.ts`:
- Around line 255-263: Replace the substring-based assertions in the
edited-event test with behavioral fixtures covering base-changing and
metadata-only pull request edits. Exercise the workflow eligibility and
concurrency decisions for each fixture, asserting that base changes proceed
while metadata-only edits do not run or publish; retain only source checks that
are necessary for observable behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 237e816d-1773-4059-800e-47a2852ae447
📒 Files selected for processing (15)
.agents/skills/nemoclaw-maintainer-day/MERGE-GATE.md.github/workflows/e2e.yaml.github/workflows/pr-e2e-gate.yaml.github/workflows/pr-review-advisor.yaml.github/workflows/pr.yamltest/e2e/README.mdtest/e2e/support/e2e-operations-workflow-boundary.test.tstest/pr-e2e-gate-exceptions.test.tstest/pr-e2e-gate-lifecycle.test.tstest/pr-e2e-gate-workflow.test.tstest/pr-e2e-gate.test.tstest/pr-review-advisor-workflow-boundary.test.tstest/pr-workflow-contract.test.tstools/e2e/operations-workflow-boundary.mtstools/e2e/pr-e2e-gate.mts
wscurran
added
area: ci
21 tasks
cv
added a commit
that referenced
this pull request
Jul 13, 2026
<!-- markdownlint-disable MD041 --> ## Summary <!-- 1-3 plain sentences: what changes and why. Describe before-and-after behavior when it applies. Use existing repository terms; do not invent a label for this PR. --> This follow-up to #6745 and #6788 makes the combined PR Review Advisor disclose both model lanes without giving Nemotron authority over GPT's assessment, fixes contradictory confidence and E2E guidance, and ensures stale CI events close only the seeded in-progress E2E check. Missing or malformed Nemotron artifacts now degrade to an unavailable second opinion while a valid GPT review still publishes. ## Changes <!-- List concrete changes. If this adds an abstraction, configuration, fallback, migration, or compatibility path, name its current requirement and consumer, explain why a direct change is insufficient, and identify the test that protects it. --> - Download the exact same-run Nemotron artifact in the write-capable publisher, validate its raw status, final result, summary, size, path, and head SHA, and pass it to the comment renderer only through a trusted validation output. GPT remains fail-closed and authoritative; the nonblocking Nemotron lane reports only allowlisted status, confidence, severity counts, and opaque structural agreement or disagreement. - Render the actual normalized confidence for informational reviews, reconcile trusted E2E coverage and selector tiers through the existing target normalizer, and avoid “Why no…” text when optional guidance exists. - Delete only `github-actions[bot]` comments whose first line is one of the two retired E2E Advisor markers, after the combined sticky comment publishes. The bot-owner and exact-marker restrictions are covered by the security-boundary tests. - Recover an existing exact-diff check ID before stale workflow-run validation only when the check is still `in_progress`, allowing the existing cleanup step to terminalize abandoned checks without rewriting completed historical results. - Extend workflow-boundary, comment, normalization, GitHub API, and gate regression tests for complete, partial, failed, skipped, malformed, stale, and completed-check cases. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates <!-- Check exactly one tests line and one docs line. Check other lines when applicable. Add every requested justification or approval reference. --> - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: This changes internal maintainer automation and GitHub workflow behavior only; the required documentation-writer pass found no user-facing CLI, configuration, API, policy, or product documentation change. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent final reviews found no remaining findings after checking `pull_request_target` privilege separation, same-run artifact provenance, primary fail-closed behavior, nonblocking secondary validation, prose isolation, exact bot-owned legacy comment deletion, E2E selector trust, and stale/completed check races. The nine-category security checklist also found no secret, injection, authorization, dependency, logging, cryptography, configuration, test, or holistic posture regression. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification <!-- Check each applicable item only when supported by the requested evidence. Run targeted tests once per relevant change set and rerun after later edits or hook autofixes that can affect that behavior. Do not rerun hook-covered checks. --> - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — final advisor suites passed 120/120 tests, the compacted advisor regression passed 44/44, and all four PR E2E gate suites passed 69/69; CLI type-check passed in the normal pre-push hook. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- <!-- DCO sign-off is required in this PR description, and every commit must appear as Verified in GitHub. Run: git config user.name && git config user.email --> Signed-off-by: Carlos Villela <cvillela@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * PR review comments now support an optional second-opinion analysis, including model-lane status and comparison details when compatible. * Comments show clearer confidence-aware model-lane information, with improved E2E “no coverage” messaging. * Legacy advisor sticky comments are cleaned up automatically during updates. * **Bug Fixes** * E2E recommendations/coverage handling is corrected when only optional coverage is selected. * E2E coverage/target reconciliation is more accurately aligned with available trusted capabilities. * PR gate handling is improved for stale, duplicate, and superseded checks—avoiding unnecessary check reopens and recording recovered check IDs. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This follow-up to #6745 binds
E2E / PR Gateto the exact pull request number, head SHA, and base SHA, so base retargets and stale workflow events cannot reuse or overwrite another diff's result. Unrelatedmainadvances no longer force a full CI rerun when GitHub proves the trusted E2E control plane is unchanged; ambiguous or changed trust boundaries still fail closed.Changes
.github/workflows/pr.yamlpath before the privileged coordinator accepts it.mainonly when GitHub proves it is a stable descendant with fewer than 300 fully enumerated changed files and noe2e-control-planematch. This compatibility path is required by the trusted workflow-run controller because exact tip equality made unrelated merges invalidate completed PR CI;test/pr-e2e-gate.test.tscovers safe advances, control-plane paths and renames, divergence, truncation, and second advances.fdfindfix from fix(ci): verify packaged fdfind identity #6785 without duplicating it in this diff.Type of Change
Quality Gates
Verification
Verifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailablenpx vitest run --project integration test/pr-e2e-gate.test.ts test/pr-e2e-gate-lifecycle.test.ts test/pr-e2e-gate-exceptions.test.ts test/pr-e2e-gate-workflow.test.ts test/pr-review-advisor-workflow-boundary.test.ts test/pr-workflow-contract.test.ts(104 passed);npx vitest run --project e2e-support test/e2e/support/e2e-operations-workflow-boundary.test.ts(23 passed)npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result:npm run docsbuilds without warnings (doc changes only)Signed-off-by: Carlos Villela cvillela@nvidia.com
Summary by CodeRabbit
New Features
Bug Fixes
Documentation