Skip to content

Explore cross-repo batched GraphQL queries #96

Description

@cpcloud

Context

PR #95 introduces per-repo GraphQL queries that replace N*8 REST detail calls with 1 GraphQL query per repo. With ETag 304 handling, most cycles cost 0 calls; changed repos cost 2 (1 ETag check + 1 GraphQL).

Idea

Batch multiple repos into a single GraphQL query using aliases:

{
  repo0: repository(owner: "org", name: "repo1") {
    pullRequests(first: 25, states: OPEN) { ... }
  }
  repo1: repository(owner: "org", name: "repo2") {
    pullRequests(first: 25, states: OPEN) { ... }
  }
}

This would turn N GraphQL queries (one per changed repo) into 1 batched query.

Constraints

  • Node limit: GitHub enforces a 500,000 node limit per query. A repo with 25 PRs and nested connections (comments, reviews, commits, CI) can consume ~5,000-20,000 nodes depending on activity.
  • Pre-computation: We could estimate node count per repo ahead of time based on known open PR count and pack repos into batches that fit under the limit. The DB already tracks open PR counts from the index phase, so this data is available without extra API calls.
  • shurcooL/githubv4 limitation: The typed struct-tag approach requires compile-time field definitions. Dynamic aliases would need raw query string construction via shurcooL/graphql or plain HTTP, losing type safety.
  • Error isolation: A single failing repo (permissions, deleted) would fail the entire batch. Need per-repo error extraction from the GraphQL response.

Approach

  1. After ETag checks identify which repos changed, group them into batches sized by estimated node count
  2. Build raw GraphQL query strings with aliased repository(...) blocks
  3. Parse response into the same RepoBulkResult structs used by the per-repo path
  4. Fall back to per-repo queries for any batch that hits complexity errors

Priority

Low — per-repo queries already achieve ~200x reduction over REST. This saves an additional N-1 calls per cycle (where N is number of changed repos). Most valuable for deployments tracking 50+ repos.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions