ci: check custom-code budgets against main's policy - #925
Conversation
apcha-oai
added
the
custom-code-burndown
Castiron custom code✅ No new custom-code files detected. 57 mixed files remain; 2 existing customizations changed. Compared
55 existing customizations unchanged
15 more in the full report. A changed generated baseline means this report cannot reliably identify which handwritten lines changed. Inspect the custom-code diffDownload the exact patch produced by this run (requires repository access): gh run download 32506621504 --repo openai/openai-java \
--name castiron-custom-code-32506621504-1 --dir /tmp/castiron-custom-code-32506621504-1
git apply --stat /tmp/castiron-custom-code-32506621504-1/custom-code.patch
cat /tmp/castiron-custom-code-32506621504-1/custom-code.patchOr reproduce it from an SDK checkout containing the vendored reporter: git fetch --no-tags origin a5228db57265b92e8c7597fcd4c78cd968483c50 9e578abab2724cc32cfc8b38c6e6619766f25c03
python3 scripts/castiron/custom_code_report.py report \
--base a5228db57265b92e8c7597fcd4c78cd968483c50 \
--head 9e578abab2724cc32cfc8b38c6e6619766f25c03 --fetch --require-head-hash --public \
--out /tmp/castiron-custom-code-9e578abab272
cat /tmp/castiron-custom-code-9e578abab272/custom-code.patchThis is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
jbeckwith-oai
left a comment
jbeckwith-oai
left a comment
There was a problem hiding this comment.
Approved at the reviewer's request.
6ce1590 to
9e578ab
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9e578abab2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| def check_isolation(repo: Path, base: str, head: str) -> int | None: | ||
| # Validate even if the policy was removed by an earlier commit in this PR. | ||
| proposed = read_budget(repo, head) |
There was a problem hiding this comment.
When a merge-queue constituent was branched before .castiron-ratchet.json was introduced, its raw PR head does not contain the file even though the synthetic merge candidate inherits the current policy from main. Calling read_budget(repo, head) before checking changed_paths therefore fails the isolation status for an otherwise mergeable PR that never touched the policy; determine whether the policy path changed first, and only read the proposed value when it did. This also conflicts with the repository invariant that the effective budget comes from main.
AGENTS.md reference: AGENTS.md:L16-L18
Useful? React with 👍 / 👎.
2 participants
Summary
Catch accidental growth in the SDK's remaining generated-file customization before it becomes a large patch to maintain. This ports the same layout and tooling approved in Python #3714 and Python #3715, including Python's trusted report computation.
The existing workflow pair measures additions plus deletions against verified generated output, using the checker and budget from main. A PR cannot increase its own allowance; budget changes must be isolated and explicitly justified, and increases require a human approving review. Agents cannot approve or bypass increases.
Technical details live in
scripts/castiron/CUSTOM_CODE.md, with references from AGENTS and CONTRIBUTING. No extra workflow files, SDK API changes, checkpoint edits, exclusion changes, or CODEOWNERS changes. Ruleset enforcement is intentionally deferred.Do not merge until the compatible WebSocket event regeneration reaches public main. The 2,000-line budget is sized for the fixed generated baseline, not today’s large event customization. A local merge preview of the fixed generation plus this tooling measures +1,524 / −208 = 1,732 lines, leaving 268 lines below the ceiling. The budget-only foundation can merge independently. Verify this tooling against the released fix before merging.
Stack
The budget-only foundation has merged. This tooling commit is now restacked onto
mainata5228db57265; the tooling patch is unchanged and the PR contains no budget-file change. CI is rerunning against main. The trusted statuses become active after the tooling lands, but will not be required until a separately authorized ruleset rollout.