feat: refresh Prime Inference models from live API

[sethkarten]

Part of RES-1239. Supersedes #1633.

What changed

• Refreshes Prime Inference models directly from the public https://api.pinference.ai/api/v1/models endpoint.
• Uses an authenticated request to discover team-authorized internal/*, dev/*, and deployment routes.
• Applies live additions, removals, display names, pricing, context limits, modalities, and reasoning support without a Prime Agent release.
• Keeps models for all other providers in the release-bundled catalog.
• Loads a validated disk cache or the bundled Prime Inference list immediately, then refreshes the public list in the background.
• Preserves local models.json definitions, provider/model overrides, extension providers, and bundled client compatibility metadata.
• Removes the proposed R2 publication workflow, credentials, aggregate schema, and catalog URL configuration.

Safety and fallback behavior

• Both public and authenticated requests are pinned to the Prime Inference HTTPS origin.
• Live data cannot inject headers or change the provider, API type, or request URL.
• Unknown models require complete specs; entries without them are skipped rather than assigned guessed limits or capabilities.
• Existing models with missing specs retain bundled capabilities and limits while accepting live names and prices.
• New models use conservative client compatibility defaults and do not send unconfirmed reasoning controls.
• A response must retain at least 50% of the bundled Prime Inference model IDs. Severe truncation, invalid responses, timeouts, and oversized responses fall back to the last valid cache or bundled models.
• Public responses exclude internal/*, dev/*, and deployment IDs containing :. Authenticated-only routes use a separate mode-0600 cache keyed by a credential/team fingerprint; they never enter the public cache.
• PI_OFFLINE=1 skips network refreshes.

Current backend follow-up

The endpoint currently returns 120 public routes. It provides complete specs for 113. Five new Qwen routes and the currently authorized private routes cannot be added dynamically yet because specs is null. To remove the remaining client fallbacks and route-name heuristics, the endpoint should provide:

• complete specs for every public and private route;
• an explicit public/private visibility field;
• supported reasoning controls or a thinking-level map;
• an optional featured field if realtime promotion is desired.

Validation

• npm run check
• AI package: 343 passed, 720 skipped
• Focused Prime Inference and registry tests: 123 passed
• Live generation: 1,290 models across 32 providers; 120 public Prime Inference routes
• Runtime live smoke test: 115 usable public Prime Inference models, five additions, zero unintended removals, zero private-route leaks, and zero transport/header changes

---

Note

Medium Risk
Runtime model pickers and context limits now follow live API data with merge/fallback rules; mistakes could hide models, leak private routes, or mis-state limits, though guards and tests target those cases.

Overview
Prime Inference model lists now update from the live /models endpoint instead of relying only on release bundles and ad hoc generator logic.

@earendil-works/pi-ai adds a shared parsePrimeInferenceModelCatalog parser (pricing, cache costs, specs, private-route detection) and moves generated catalog output through renderModelsFile so remote strings are JSON-escaped. The generate-models script hits the public endpoint without Prime CLI credentials, applies truncation/coverage guards, and still merges OpenRouter gaps where needed.

Prime Agent loads a validated prime-inference-models-cache.json (or bundled models) immediately, background-refreshes the public catalog, and merges live entries via buildPrimeInferenceModels / mergePrimeInferenceModels while keeping bundled compat and thinking maps. Authorized internal/*, dev/*, and deployment routes are built from authenticated catalog responses and persisted as full model records, not ID sets. PI_OFFLINE=1 skips network refresh; docs describe the new behavior.

^{Reviewed by Cursor Bugbot for commit 24d3413. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Refresh Prime Inference models from live API with validated cache fallback

• Adds a shared catalog parser and builder for Prime Inference models, validating metadata like pricing, cache costs, context/output limits, and image support
• The generator now fetches from the public /models endpoint without Prime CLI credentials, with a severe-truncation guard preserving the existing snapshot if live results are too sparse
• The registry merges authorized private model objects with full metadata, excludes public IDs from private results, and reloads models after catalog or authorization changes
• Public refreshes are backgrounded with cache fallback, request deduplication, atomic 0600 cache writes, and PI_OFFLINE=1 support
• Risk: loadModels in model-registry.ts replaces bundled Prime Inference entries with live entries when supplied; existing tests for fixed maxTokens and context assertions are relaxed to allow clamped live metadata

Changes since #1928 opened

• Changed ModelRegistry private Prime authorization flow to cache and restore full model metadata instead of model IDs, enabling immediate availability of authorized private deployments at startup without network access [24d3413]
• Tightened validation in parsePrimeInferenceModelCatalog utility to require both input and output modalities to be arrays of strings, treating modalities as undefined when validation fails and only attaching full specs when all required fields are valid [24d3413]
• Changed minimum coverage criteria in buildPrimeInferenceModels utility to measure coverage of bundled models rather than total model count [24d3413]
• Added test coverage for cached authorized deployment metadata restoration without network access, retention of priced entries without complete specs, and coverage measurement against bundled models [24d3413]

^{Macroscope summarized e7e810c.}

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[sethkarten]

@snimu The PR has been fully redesigned around the live Prime Inference /api/v1/models endpoint. The R2 catalog workflow is removed, public refreshes are anonymous and nonblocking, team-private routes stay authorization-gated, and incomplete unknown routes are skipped. All 21 checks are green and all bot threads are resolved. Could you take the final approval review?

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit e7e810c. Configure here.}

[macroscopeapp]

🟡 Medium core/model-registry.ts:884

Dynamically authorized private deployments are rebuilt without the models.json provider or per-model overrides, so settings such as providers["prime-inference"].modelOverrides["vendor/model:deployment"] and baseUrl are silently ignored. reloadModelsAfterCatalogChange() calls loadModels(), which applies overrides in loadBuiltInModels() before appending authorizedPrivatePrimeInferenceModels; apply the same override pipeline to authorized models before adding them to this.models.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @packages/coding-agent/src/core/model-registry.ts around line 884:

Dynamically authorized private deployments are rebuilt without the `models.json` provider or per-model overrides, so settings such as `providers["prime-inference"].modelOverrides["vendor/model:deployment"]` and `baseUrl` are silently ignored. `reloadModelsAfterCatalogChange()` calls `loadModels()`, which applies overrides in `loadBuiltInModels()` before appending `authorizedPrivatePrimeInferenceModels`; apply the same override pipeline to authorized models before adding them to `this.models`.

Evidence trail:
Commit 24d3413. `packages/coding-agent/src/core/model-registry.ts:511-514,537-576,578-600,654-669,884-885,1366-1425`. Git commands: `git show REVIEWED_COMMIT -- packages/coding-agent/src/core/model-registry.ts`; `git grep -n "authorizedPrivatePrimeInferenceModels\|loadBuiltInModels\|modelOverrides" REVIEWED_COMMIT -- packages/coding-agent/src/core/model-registry.ts`

[sethkarten]

Included in #2068: #2068