Skip to content

fix(advisor): preserve complete PR diffs - #8149

Merged
cv merged 5 commits into
mainfrom
codex/fix-pr-review-advisor-full-diff
Aug 4, 2026
Merged

fix(advisor): preserve complete PR diffs#8149
cv merged 5 commits into
mainfrom
codex/fix-pr-review-advisor-full-diff

Conversation

@jyaunches

@jyaunches jyaunches commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The PR Review Advisor previously shortened diffs at 160,000 characters before the first review turn, so later turns could reuse incomplete evidence. It now captures the complete Git diff and fails if Git cannot provide it. Existing tests checked context transport only with small diffs; the regression test now exceeds the former boundary.

Changes

  • Remove the PR diff character limit and child-process capture limit.
  • Fail the advisor analysis when Git cannot return a complete diff.
  • Write failure artifacts directly when trusted pre-session inputs are unavailable.
  • Preserve existing partial failure artifacts and write any missing fallback artifacts when analysis exits early.
  • Label and document the context-tool result and artifact as the complete PR diff.
  • Add regression tests for content after 160,000 characters, Git diff fallback, Git diff failure, and early-analysis failure artifacts.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: No Fern page covers this internal maintainer workflow; the colocated tool README records the complete-diff artifact contract.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification:
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: docs-updated
  • Evidence: tools/pr-review-advisor/README.md describes complete diff delivery and pre-session failure artifacts; no Fern page covers this internal maintainer workflow.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — all 124 tests passed in the eight PR Review Advisor integration test files; the final fixture hardening passed all 34 workflow-boundary tests.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Additional validation: npm run typecheck:cli, npm run test-size:check, and npm run test:changed passed. npm run docs passed with zero errors and two Fern warnings; no Fern source files changed.


Signed-off-by: Julie Yaunches jyaunches@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Pull request reviews now include the complete Git diff, including changes beyond the previous size limit.
    • Diff retrieval now reports errors when comparisons cannot be generated.
    • Analysis failures and skipped analyses now produce clear, status-specific results, recommendations, and failure artifacts.
  • Documentation

    • Clarified that diffs are complete and tool-result payloads are deterministic.
  • Tests

    • Added coverage for large diffs, alternate comparisons, unresolved references, and analysis failure reporting.

@jyaunches jyaunches self-assigned this Aug 3, 2026
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a22179d9-54c0-4b17-9b10-e4e781893c97

📥 Commits

Reviewing files that changed from the base of the PR and between 0ba5ed4 and 72b61de.

📒 Files selected for processing (1)
  • test/pr-review-advisor-workflow-boundary.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/pr-review-advisor-workflow-boundary.test.ts

📝 Walkthrough

Walkthrough

getDiff now returns complete Git diffs without truncation and throws when diff commands fail. The advisor labels complete diff context. Startup and analyzer failures now generate failure artifacts before errors propagate. Tests cover diff resolution, artifact creation, and artifact preservation.

Changes

Advisor reliability updates

Layer / File(s) Summary
Complete diff retrieval
tools/advisors/git.mts, test/pr-review-advisor-diff.test.ts
getDiff removes the character limit and truncation behavior. It falls back to a two-dot diff for unrelated histories and throws when both commands fail. Tests cover large diffs, fallback resolution, unresolved references, and shared temporary Git setup.
Complete diff integration and pre-session failures
tools/pr-review-advisor/analyze.mts, tools/pr-review-advisor/README.md, test/pr-review-advisor-diff.test.ts
The advisor labels the Git result as complete, documents complete diff availability, and writes failure metadata when startup context collection fails.
Failed analysis artifacts
tools/pr-review-advisor/run-analysis.mts, test/pr-review-advisor-workflow-boundary.test.ts
Bootstrap results distinguish failed analysis from skipped analysis. Nonzero analyzer exits create missing failure artifacts before propagating RunAnalysisError. Tests verify exit propagation, failure metadata, and preservation of partial artifacts.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant getDiff
  participant Git
  participant Advisor
  getDiff->>Git: Request three-dot diff
  Git-->>getDiff: Return diff or command failure
  getDiff->>Git: Request two-dot diff when needed
  Git-->>getDiff: Return complete diff or failure
  getDiff-->>Advisor: Return complete diff or throw
Loading
sequenceDiagram
  participant runPrReviewAdvisorAnalysis
  participant runAnalysis
  participant ResultFiles
  runPrReviewAdvisorAnalysis->>runAnalysis: Start analysis
  runAnalysis-->>runAnalysis: Detect analysis failure
  runAnalysis->>ResultFiles: Write failure artifacts
  runAnalysis-->>runPrReviewAdvisorAnalysis: Throw analysis error
Loading

Possibly related PRs

Suggested labels: bug-fix, area: cli

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary change: preserving complete PR diffs in the advisor.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-pr-review-advisor-full-diff

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 72b61de in the codex/fix-pr-review-... branch remains at 96%, unchanged from commit c63e7eb in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 72b61de in the codex/fix-pr-review-... branch remains at 81%, unchanged from commit c63e7eb in the main branch.

Show a code coverage summary of the most impacted files.
File main c63e7eb codex/fix-pr-review-... 72b61de +/-
src/lib/inferen...ompatibility.ts 95% 94% -1%
src/lib/messagi...nnels/policy.ts 100% 100% 0%
src/lib/sandbox...rce-identity.ts 88% 88% 0%
src/lib/state/config-io.ts 93% 93% 0%
src/lib/platform.ts 84% 89% +5%

Updated August 03, 2026 18:55 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tools/pr-review-advisor/analyze.mts`:
- Line 363: Update tools/pr-review-advisor/analyze.mts around getDiff and
writeFailure so errors during pre-session Git setup still produce the required
failure artifacts; ensure writeFailure is available before getDiff can throw and
route both-command failures through that path. In
test/pr-review-advisor-diff.test.ts, add coverage for failure of both diff
commands and for a successful three-dot-to-two-dot fallback.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 18a67c50-ee33-48a8-8bdc-903e77ac7d15

📥 Commits

Reviewing files that changed from the base of the PR and between c63e7eb and 5474caa.

📒 Files selected for processing (4)
  • test/pr-review-advisor-diff.test.ts
  • tools/advisors/git.mts
  • tools/pr-review-advisor/README.md
  • tools/pr-review-advisor/analyze.mts

Comment thread tools/pr-review-advisor/analyze.mts Outdated
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: Review the warnings below.
Findings: 0 blockers · 1 warning · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 1 warning · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Failed after a partial review · low confidence · 0 blockers · 0 warnings · 0 suggestions

Second-opinion terminology and E2E selections are advisory. They do not change the primary assessment or E2E / PR Gate.

1 semantic terminology decision

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • justified — complete PR diff at tools/pr-review-advisor/README.md:166: Keep this term. The modifier identifies the artifact completeness contract.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: None

1 warning · 0 suggestions

Warnings

Warnings do not block.

PRA-1 Warning — Bound complete-diff capture before it exhausts the analysis runner

  • Location: tools/advisors/git.mts:23
  • Category: security
  • Problem: getDiff passes Number.POSITIVE_INFINITY as execFileSync's maxBuffer for PR-controlled Git diff output. A sufficiently large diff can exhaust process memory before analyze.mts can emit its low-confidence failure artifacts.
  • Impact: An untrusted PR can deny service to the trusted advisor job and prevent the intended failure evidence from being written.
  • Recommendation: Stream the diff into a bounded artifact or enforce a documented maximum that fails with a failure artifact before memory exhaustion.
  • Verification: Inspect getDiff's execFileSync maxBuffer and exercise it with diff output larger than the selected bound.
  • Test coverage: Add a test that produces output above the supported diff-artifact limit and verifies a controlled failure result is written without retaining the full output in memory.
  • Evidence: tools/advisors/git.mts:23 passes Number.POSITIVE_INFINITY to gitOutput for complete PR diff output. test/pr-review-advisor-diff.test.ts:15-71 proves only a 170,000-character diff is retained and does not cover oversized-output containment.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tools/pr-review-advisor/run-analysis.mts`:
- Around line 280-284: Update the nonzero-exit fallback in
tools/pr-review-advisor/run-analysis.mts:280-284 around writeBootstrapResult so
it preserves existing partial or summary artifacts, writes only missing fallback
artifacts, and still throws the original RunAnalysisError. In
test/pr-review-advisor-workflow-boundary.test.ts:896-925, pre-create a partial
artifact and verify fallback completion preserves it while producing the
remaining artifacts.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 625a403b-f46c-45de-922f-3b4d68f59d07

📥 Commits

Reviewing files that changed from the base of the PR and between 5474caa and 9cd87c4.

📒 Files selected for processing (3)
  • test/pr-review-advisor-diff.test.ts
  • test/pr-review-advisor-workflow-boundary.test.ts
  • tools/pr-review-advisor/run-analysis.mts

Comment thread tools/pr-review-advisor/run-analysis.mts
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Comment thread test/pr-review-advisor-workflow-boundary.test.ts Fixed
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tools/pr-review-advisor/analyze.mts (1)

399-438: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Pre-session failure path now writes artifacts; add coverage for the double-resolution-failure fallback.

This correctly resolves the earlier critical issue: getChangedFiles, getHeadSha, getDiff, and collectDeterministicContext failures now write pre-session failure artifacts before the original error propagates.

One branch still lacks direct test coverage. At lines 418-424, when getHeadSha(headRef) also fails during recovery, headSha falls back to "unavailable". No test exercises this branch (the current test in test/pr-review-advisor-diff.test.ts only makes getChangedFiles fail while HEAD still resolves). Add a case with both --base and --head pointing at unresolved refs to confirm the failure artifacts still get written with headSha: "unavailable".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/pr-review-advisor/analyze.mts` around lines 399 - 438, Add a test in
the PR review advisor failure-path coverage that invokes the CLI with both base
and head refs unresolved, causing the initial context collection and the
recovery getHeadSha call to fail. Assert that failure artifacts are still
written and their metadata records headSha as "unavailable", while preserving
propagation of the original failure.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tools/pr-review-advisor/analyze.mts`:
- Around line 399-438: Add a test in the PR review advisor failure-path coverage
that invokes the CLI with both base and head refs unresolved, causing the
initial context collection and the recovery getHeadSha call to fail. Assert that
failure artifacts are still written and their metadata records headSha as
"unavailable", while preserving propagation of the original failure.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d8b445aa-3d13-4f9d-b53a-3e0da0d961d1

📥 Commits

Reviewing files that changed from the base of the PR and between e16ad99 and 0ba5ed4.

📒 Files selected for processing (2)
  • test/pr-review-advisor-diff.test.ts
  • tools/pr-review-advisor/analyze.mts

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
@jyaunches
jyaunches requested a review from cv August 3, 2026 18:43
@jyaunches jyaunches added the v0.0.102 Release target label Aug 3, 2026

@apurvvkumaria apurvvkumaria left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 72b61de. The change preserves complete diffs and failure artifacts and I found no blocking defect. Non-blocking fast follow: bound or stream PR-controlled diff capture so an unusually large diff cannot exhaust the trusted advisor process before it writes its failure artifact. Functional CI passes; aggregate and E2E gates remain unresolved.

@cv
cv merged commit ad26164 into main Aug 4, 2026
73 of 80 checks passed
@cv
cv deleted the codex/fix-pr-review-advisor-full-diff branch August 4, 2026 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v0.0.102 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants