fix: support GCP metadata authentication in workerd

[steipete]

Summary

GCP metadata authentication passed redirect: "error" to fetch. Node accepts that option, but workerd rejects it before sending a request; the existing transient-error handler then retried an unsupported option for almost a minute.

Use workerd-supported manual redirect handling and reject every 300–399 response with the existing non-retried trust error before inspecting its marker or body. The endpoint, Google request/response markers, token cache, timeout, service-account path, and legitimate transient retry policy stay unchanged. Redirect destinations and token-looking bodies are never used.

Verification

The regression fails on unchanged main. Full Worker suite: 2,516 passed, 3 skipped. TypeScript, lint, formatting, Worker dry-run build, docs build, and diff checks passed. Tests cover 300/399 boundaries, common redirects, a body-unread 304 without a marker, repeated rejection without caching, and existing healthy/concurrent/transient/deadline behavior. Managed Codex review and independent semantic review found no accepted findings.

Real GCPClient authentication ran in native workerd 2026-08-26, without a fetcher override, preloaded token cache, or patched clock, against strictly synthetic metadata/Compute responses:

unchanged baseline: 500 after 59.035s, zero outbound requests
candidate healthy: 200 after 11ms, one metadata + one Compute request
candidate 302/307/308: trust error in 4ms, one metadata request each
candidate missing marker: trust error in 2ms, one metadata request
candidate transient 503: recovery after 1.007s, two metadata + one Compute request

Both runtimes were disposed. This proves the actual runtime transport and refusal behavior with synthetic upstreams; it is not live GCP IAM, attached-service-account, or cloud-lifecycle proof. No dependency, configuration, credentials, production deployment, or cloud resource changes are required. Changelog and provider documentation are updated.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed September 4, 2026, 9:09 AM ET / 13:09 UTC.

ClawSweeper review

What this changes

The PR changes GCP metadata token fetching to use workerd-supported manual redirects and explicitly rejects every redirect response before reading or using it.

Merge readiness

✅ Ready for maintainer review

Keep open: current main still uses the unsupported workerd redirect mode, while this focused fix preserves the metadata-token trust boundary and includes sufficient native-runtime proof.

Priority: P2
Reviewed head: 641b1173e586e43a30cfb5811636e6529ba8b937

Review scores

Measure	Result	What it means
Overall readiness	🦞 diamond lobster (5/6)	A focused compatibility repair with direct regression coverage and convincing native-workerd behavior evidence.
Proof confidence	🦞 diamond lobster (5/6)	Sufficient (terminal): GCPClient.metadataTokenResponse is reached through the real client’s metadata-auth flow in native workerd 2026-08-26; the supplied terminal trace shows the current-main failure, candidate healthy metadata-plus-Compute request, immediate 302/307/308 rejection, marker rejection, and transient recovery using synthetic upstreams.
Patch quality	🦞 diamond lobster (5/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (terminal): GCPClient.metadataTokenResponse is reached through the real client’s metadata-auth flow in native workerd 2026-08-26; the supplied terminal trace shows the current-main failure, candidate healthy metadata-plus-Compute request, immediate 302/307/308 rejection, marker rejection, and transient recovery using synthetic upstreams.
Evidence reviewed	7 items	Current-main defect: Current main still passes Fetch redirect mode "error" to the metadata-token request, so the requested work is not already implemented. Introduced implementation: The introduced hunk switches to manual redirects and throws the non-retried trust error for all 3xx responses before checking the marker or consuming the body. Regression coverage: Focused tests cover representative 300–399 statuses, repeated rejection without caching, and a 304 response rejected before marker or body access.
Findings	None	None.
Security	None	None.

How this fits together

Crabbox’s Worker-side GCP client obtains a token from either a service-account key or the instance metadata service, then uses it for Compute API calls. Metadata responses must remain Google-marked and must never redirect token processing to another destination.

flowchart LR
  A[GCP credential settings] --> B[GCP client]
  B --> C[Metadata token request]
  C --> D{Redirect response?}
  D -->|yes| E[Reject untrusted response]
  D -->|no| F[Validate Google marker]
  F --> G[Cache token]
  G --> H[Compute API request]

Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Patch scope	4 files; production +7/-1, tests +43/-1, docs/changelog +3/-1	The implementation is narrowly scoped and pairs the runtime change with regression coverage and operator documentation.
Redirect rejection coverage	7 status codes plus 1 no-body-read guard	The tests exercise the full intended 3xx range and verify the rejection happens before token content is consumed.

Technical review

Best possible solution:

Land the narrow manual-redirect implementation and retain the explicit pre-body, non-retried rejection of all metadata 3xx responses.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main’s metadata request uses redirect: "error", and the supplied native-workerd trace records that exact baseline failing before any outbound metadata request.

Is this the best way to solve the issue?

Yes. Manual redirect handling with an immediate fail-closed 3xx check restores workerd compatibility without accepting a redirected metadata response or changing token-cache and retry policy.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 9f1b19a89ffe.

Labels

Label changes:

• add P2: This fixes a bounded failure for GCP metadata-auth deployments running the Worker coordinator under workerd.
• add proof: sufficient: Contributor real behavior proof is sufficient. GCPClient.metadataTokenResponse is reached through the real client’s metadata-auth flow in native workerd 2026-08-26; the supplied terminal trace shows the current-main failure, candidate healthy metadata-plus-Compute request, immediate 302/307/308 rejection, marker rejection, and transient recovery using synthetic upstreams.
• add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): GCPClient.metadataTokenResponse is reached through the real client’s metadata-auth flow in native workerd 2026-08-26; the supplied terminal trace shows the current-main failure, candidate healthy metadata-plus-Compute request, immediate 302/307/308 rejection, marker rejection, and transient recovery using synthetic upstreams.

Label justifications:

• P2: This fixes a bounded failure for GCP metadata-auth deployments running the Worker coordinator under workerd.
• rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): GCPClient.metadataTokenResponse is reached through the real client’s metadata-auth flow in native workerd 2026-08-26; the supplied terminal trace shows the current-main failure, candidate healthy metadata-plus-Compute request, immediate 302/307/308 rejection, marker rejection, and transient recovery using synthetic upstreams.
• proof: sufficient: Contributor real behavior proof is sufficient. GCPClient.metadataTokenResponse is reached through the real client’s metadata-auth flow in native workerd 2026-08-26; the supplied terminal trace shows the current-main failure, candidate healthy metadata-plus-Compute request, immediate 302/307/308 rejection, marker rejection, and transient recovery using synthetic upstreams.

Evidence

What I checked:

• Current-main defect: Current main still passes Fetch redirect mode "error" to the metadata-token request, so the requested work is not already implemented. (worker/src/gcp.ts:1230, 9f1b19a89ffe)
• Introduced implementation: The introduced hunk switches to manual redirects and throws the non-retried trust error for all 3xx responses before checking the marker or consuming the body. (worker/src/gcp.ts:1230, e69e70634694)
• Regression coverage: Focused tests cover representative 300–399 statuses, repeated rejection without caching, and a 304 response rejected before marker or body access. (worker/test/gcp.test.ts:1072, 641b1173e586)
• Cache behavior: The token cache only stores a successfully resolved loader value, so the thrown redirect trust error cannot populate the cache. (worker/src/expiring-token-cache.ts:19, 9f1b19a89ffe)
• Runtime proof: The PR body records native workerd 2026-08-26 runs of the real GCP client against synthetic metadata and Compute upstreams: current-main baseline made zero outbound requests and failed after 59 seconds, while the candidate completed the healthy flow and rejected redirects immediately.
• Feature history: History identifies the merged GCP metadata-auth feature as commit 0bd8a95, with subsequent GCP token-refresh work by the same contributor. (worker/src/gcp.ts, 0bd8a95605e9)

Likely related people:

• steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[steipete]

Merged as 8165fa3; the landed tree matches the verified combined-main preview.

The metadata loader previously requested Fetch redirect mode error, which native workerd rejects before making any outbound request. Node-based stubs accepted that option, so unit tests missed the production-runtime incompatibility. Manual redirect handling is supported by workerd; every 3xx response now immediately fails the existing trust boundary before marker checks or token-body consumption. Token caching and normal transient retry behavior remain unchanged.

Native workerd A/B proof exercised the real GCPClient authentication path with synthetic metadata/Compute upstreams and no fetch override or preloaded cache: baseline failed after 59 seconds with zero requests; the candidate authenticated, rejected 302/307/308 without following them, rejected missing markers, and recovered from a transient 503. All runtimes were disposed. Full Worker tests and static/build/docs checks passed; all exact-head substantive checks passed without reruns. The final main readback contains the documented fix and changelog entry. This proof does not claim a production GCP account deployment.