Skip to content

fix(ai): omit the default service tier, reprice cache writes from message_delta, repoint the zai default - #2032

Open
snimu wants to merge 4 commits into
mainfrom
fix/provider-wire-defects
Open

fix(ai): omit the default service tier, reprice cache writes from message_delta, repoint the zai default#2032
snimu wants to merge 4 commits into
mainfrom
fix/provider-wire-defects

Conversation

@snimu

@snimu snimu commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Three provider wire/pricing defects:

  • GitHub Copilot never receives the service_tier field. Copilot rejects the field itself (400 service_tier is not supported) for every value. The omission is provider-scoped: for everyone else the explicit value — including "default" — stays on the wire, because for OpenAI an absent field means "auto" (the project-configured tier), not "default"; omitting it would silently upgrade users on Priority-configured projects and could under-report cost (response.service_tier is only guaranteed when the request field is present). Fixes the defect reported in discussion [Bug] Error: Provider rejected the request (invalid_request_error, 400): service_tier is not supported #1413.
  • Anthropic cache writes are repriced from message_delta. The cache-write rate was computed once (at message_start, or before the stream), and message_delta billed its updated cache-write token count at that stale rate. When a delta carries a cache_creation breakdown (5m/1h mix), the rate is now recomputed exactly as message_start does; without a breakdown the prior rate is kept. Fixes the defect reported in discussion [Bug] Anthropic reports stale cache-write cost when message_delta updates cache creation #1769.
  • The zai default model exists again. defaultModelPerProvider.zai pointed at glm-5.1, which was removed from the catalog — every default zai selection silently degraded to a fallback template model. The default is now glm-5.3, and a new catalog-existence test makes any future default/catalog drift a CI failure instead of a silent downgrade. Fixes the defect reported in discussion [Bug] Stale default model for zai provider (glm-5.1 removed from catalog) silently falls back to outdated glm-4.7 #1679.

Validation

  • four pins, each verified fail-unfixed: default-tier omission on the OpenAI Responses body and on the Codex body (one per wire builder), message_delta breakdown repricing (exact-cost assertion), and per-provider default catalog existence
  • suites: anthropic-sse-parsing (7), openai-responses-copilot-provider (24), openai-codex-stream (18), model-resolver (25) — all green, sanitized env
  • root npm run check passes via the pre-commit hook

LOC

Total src: +19/−3 (net +16); tests: +78/−9 (net +69).
Src +20/−3 across four files (all point fixes; the only added mechanism is the 8-line delta reprice mirroring the existing message_start path). Tests +143/−2, changelog 2 fragments.

Linear: RES-1266 https://linear.app/primeintellect/issue/RES-1266


Note

Medium Risk
Changes affect live API request bodies (Copilot) and reported Anthropic cache-write costs when delta usage differs from message_start; zai users get a different default model.

Overview
Fixes three provider wiring and billing issues: GitHub Copilot no longer gets a service_tier field on OpenAI Responses requests (Copilot 400s on the field for any value), while other providers still send an explicit tier—including "default", since omitting it would mean "auto" on OpenAI, not default tier/cost behavior.

Anthropic streaming now recomputes cache-write cost when message_delta includes a cache_creation TTL breakdown, so final token counts are priced with the delta’s mix instead of a stale rate from message_start.

Coding-agent changes the zai default from removed glm-5.1 to glm-5.3, and adds a test that every defaultModelPerProvider entry exists in the catalog so silent fallback templates do not return.

Reviewed by Cursor Bugbot for commit f82c7fa. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Reprice Anthropic cache writes from message_delta, omit service_tier for Copilot, repoint zai default to glm-5.3

  • In streamAnthropic, cache-write cost is recalculated from the message_delta cache-creation breakdown (read via a compatibility cast) using the configured cache TTL, instead of retaining the message_start rate for the whole stream.
  • In buildParams, the service_tier field is omitted from the request JSON for the github-copilot provider but still serialized for other OpenAI Responses providers.
  • The zai provider default model in defaultModelPerProvider changes from glm-5.1 to glm-5.3; a new test asserts every provider default exists in its catalog.
  • Behavioral Change: Anthropic streams that report a different cache-creation breakdown in message_delta now produce different cache-write costs than before; Copilot requests no longer send service_tier; zai clients without an explicit model now resolve to glm-5.3.

Macroscope summarized f82c7fa.

…tes from message_delta, and repoint the zai default model

Sending service_tier: "default" explicitly breaks strict endpoints (Copilot rejects it) while meaning nothing to OpenAI, so both Responses wire builders omit it. Anthropic message_delta events that carry a cache_creation breakdown reprice the cache-write rate the same way message_start does, instead of billing new tokens at the stale initial rate. The zai default model glm-5.1 no longer exists in the catalog and silently degraded to a fallback template; the default is now glm-5.3 and a catalog-existence test turns future default drift into a CI failure.
xeophon pushed a commit that referenced this pull request Sep 4, 2026
…rces (#2039)

Explicit, reviewed regeneration via `npm run generate-models`: 1238 -> 1262
models (41 added, 17 removed). No provider dropped wholesale or lost >30% of
its rows.

GitHub Copilot removed claude-opus-4.5, claude-opus-4.6, claude-sonnet-4,
claude-sonnet-4.5, gemini-3.1-pro-preview, gpt-4.1, gpt-5.2, and gpt-5.2-codex;
its stale claude-sonnet-4.5 entry is what broke CI type checks against the
regenerated catalog. Test references to the removed Copilot models move to
claude-sonnet-4.6 and gpt-5.3-codex; the interleaved-thinking beta test uses
claude-haiku-4.5, the only remaining non-adaptive Claude in the Copilot
catalog. All other providers' referenced ids are unchanged.

Every defaultModelPerProvider entry exists in the refreshed catalog except the
pre-existing zai glm-5.1 gap already fixed in #2032.

Linear: RES-1269
… default value

For OpenAI, an absent service_tier means auto (the project-configured tier), not default: omitting an explicitly requested default silently upgraded users on Priority-configured projects and could under-report cost, since response.service_tier is only guaranteed when the request field is present. Copilot rejects the FIELD itself (400 'service_tier is not supported') for every value, per the original report. The omission is now provider-scoped: Copilot never sends the field, everyone else always sends the explicit value, including default.
The Codex default-tier pin becomes a row of the existing service-tier table; the delta-reprice pin reuses createCacheUsageEvents via an optional delta breakdown.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant