fix(sweep): unstarve markStale via search API; snapshot listings before mutating#75938
Open
Hem1700 wants to merge 4 commits into
Open
fix(sweep): unstarve markStale via search API; snapshot listings before mutating#75938Hem1700 wants to merge 4 commits into
Hem1700 wants to merge 4 commits into
Conversation
…t pagination Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…re mutating markStale walked the oldest-updated-first /issues listing with a fixed 10-page budget, but that window is now fully occupied by permanent non-candidates (open PRs, >=10-upvote issues, already-stale, assigned), so every run visited 1000 items and labeled zero. Query the search API for exactly the labelable set instead, with a per-run write cap. closeExpired offset-paginated a listing it was shrinking with every close (skipping one issue per close at each page boundary) and its stale listing outgrew the 10-page cap; snapshot the listing first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
markStalecurrently labels nothing. It walks/repos/{owner}/{repo}/issues?state=open&sort=updated&direction=ascwith a fixed budget of 10×100, but that listing interleaves the issues it can label with items it can only skip — and the skips are permanent residents of the oldest-updated-first window, so they accumulate at the front until no candidate is reachable.Replaying the script's exact ten listing calls against this repo (2026-07-08), the full 1,000-item window contained:
STALE_UPVOTE_THRESHOLD)stale/autocloseThe newest item in the window was updated 2026-05-07 — seven weeks short of the 14-day cutoff. Meanwhile
is:issue is:open updated:<2026-06-25 -label:stale -label:autoclose no:assigneematches 3,002 issues the sweep should be nudging but can never reach. SincemarkStalefeedscloseExpired(stale → 14 d → close), the lifecycle pipeline is stalled for new issues, and the window only grows more clogged as more PRs and popular issues age into it.Two smaller defects in the same pattern:
issue-lifecycle-comment.ymlposts in response) bumpsupdated_at, reordering the ascending-sorted listing mid-walk; incloseExpired, closing an issue removes it from thelabels=-filtered listing. Either way, each mutation on page N shifts the remaining pages left, so thepage=N+1fetch skips one issue per mutation. Same bug class Fix lock-closed-issues workflow: use search API instead of offset pagination #69470 fixed inlock-closed-issues.yml.closeExpired's 10-page cap is too small:label:stalealone currently lists ~1,490 open issues (15 pages), so the oldest ~500 are never even examined.Fix
markStaleasks the search API for exactly the labelable set (is:issue is:open updated:<cutoff -label:stale -label:autoclose no:assignee, oldest-updated first) — the same approach as Fix lock-closed-issues workflow: use search API instead of offset pagination #69470. PRs, upvoted, assigned, and already-labeled issues no longer occupy the window. A client-side predicate re-checks every condition (the search index can lag) plus the two search can't express: the exact cutoff time (search'supdated:is date-granular) and the 👍-specific threshold (search'sreactions:counts all reaction kinds).lock-closed-issues.yml, so the backlog drains across scheduled runs without tripping secondary rate limits. Search fetches are paced (2 s) and retried with backoff on 403/429 — search's secondary limit is burst-sensitive.closeExpiredsnapshots the full listing before closing anything (up to 30 pages), so closes can't shift pages underneath the walk, and the current stale backlog fits.scripts/sweep-lib.tswith unit tests (bun test scripts/sweep-lib.test.ts, no network), following theissue-lifecycle.tsshared-module pattern.The per-issue close logic (events →
labeledAt, human-comment safety net) is deliberately untouched: #75541 is improving events handling there, and this PR stays out of its way — the two compose.Verification
bun test scripts/sweep-lib.test.ts— 18 pass, no network.markStale's exact walk (same endpoint, same filters, same page budget): deterministically 0 labels per run.--dry-runof the new code against this repo immediately surfaces real candidates and stops at the per-run cap:The
closeExpiredphase of the same dry-run walks the snapshot listing and surfaces real work as well (#57930: would close (stale, 14d old),#23012: skipping (human activity after stale label), …). The run also happened to exercise the retry path end-to-end: the token had a hot secondary rate limit, and the search fetches backed off and completed instead of killing the run.With the cap at 250 and the sweep scheduled twice daily, the ~3,000-issue backlog drains in about six days, then the sweep returns to a steady trickle.