Skip to content

docs(model): canonical model-configuration reference + fix stale defaults #742

Description

@scottschreckengaust

Child 1 of 6 — parent #740. No deploy. Docs + one new test.

Self-contained: docs-only plus a drift test. Mergeable in any order relative to its siblings; blocks nothing but should land first so no sibling lands a new default while the docs still say Sonnet 4.6.

Scope

1. Add one canonical section (suggest docs/guides/DEVELOPER_GUIDE.md, linked from USER_GUIDE.md and agent/README.md) containing:

  • The five-layer table (IAM allowlist / platform default / auxiliary model / per-repo blueprint / per-task+local) with file:line pointers — see Docs: document model-ID configuration layers, governing env vars, and global Bedrock endpoints #740 for the researched table.
  • The env-var table: ANTHROPIC_MODEL, ANTHROPIC_DEFAULT_HAIKU_MODEL, CLAUDE_CODE_USE_BEDROCK, with who sets each and which prefix form it takes.
  • The precedence chain, narrowest wins:
    per-task payload model_id > blueprint agent.modelId > stack env ANTHROPIC_MODEL > agent/src/config.py fallback, all gated by the IAM allowlist, which is itself gated by account-level Bedrock model access.
  • The prefix rule, with a worked failure. bedrockModels context takes bare foundation-model IDs (anthropic.claude-…); every other layer takes the prefixed inference-profile ID (us./global./…). Both grant sites derive the profile ARN by prefixing, so a prefixed entry in bedrockModels would yield us.us.anthropic.…. Verified: a bare ID cannot be invoked on-demand at all — aws bedrock-runtime invoke-model --model-id anthropic.claude-opus-5 returns ValidationException: Invocation of model ID anthropic.claude-opus-5 with on-demand throughput isn't supported. Retry your request with the ID or ARN of an inference profile that contains this model.

2. Correct the stale defaults. Code default is us.anthropic.claude-opus-4-8 (agent/src/config.py:563); these say Sonnet 4.6:

  • agent/README.md:122 and :149
  • docs/guides/DEVELOPER_GUIDE.md:250
  • docs/design/REPO_ONBOARDING.md:123 — says "Claude Sonnet 4" and attributes the source to "CDK stack props", which is wrong: it's a Python literal with no CDK/env knob today.

3. Fix the haiku prefix contradiction. agent/README.md:126 documents a bare default (anthropic.claude-haiku-…), contradicting the us.-prefix requirement asserted at agent/src/models.py:162 and the value actually deployed at cdk/src/stacks/agent.ts:393. The deployed value is authoritative.

4. Delete a dead reference. cdk/test/stacks/agent.test.ts:289 explains a fixed revise-model grant via DEFAULT_REVISE_MODEL_ID = sonnet, but that identifier does not exist anywhere in the repo (grep -rn DEFAULT_REVISE_MODEL_ID hits only this comment). Either restore the constant if the behavior it describes is real, or correct the comment to match what the test actually asserts. Do not silently delete the comment without establishing which.

4b. Add a "Cost and model selection" section — the customer-facing deliverable.

This is the reason the configuration layers matter: the model choice is a cost decision, and it must be adjustable without a code change. Cover:

Per-token rate vs. token volume. Measured on the pinned toolchain, same one-turn prompt, same system prompt:

Model Input tokens Reported cost_usd Implied input rate
us.anthropic.claude-opus-4-8 32,145 $0.160850 $5.00/MTok
us.anthropic.claude-opus-5 37,584 $0.188020 $5.00/MTok

Token ratio 1.169, cost ratio 1.169 — identical. The per-token rate is unchanged; the delta is token volume on an identical prompt. Document it this way. "Opus 5 costs ~17% more per task" invites the wrong remedy (switch models); "same rate, more tokens" points at the right levers (prompt size, caching, max_turns).

Answer "Where do I set max_budget_usd?" explicitly. Today the answer is scattered and partly wrong. Verified surfaces:

Surface How Status
Per task, CLI bgagent submit --max-budget <dollars> (cli/src/commands/submit.ts:69), range 0.01–100 ✅ works
Per task, REST max_budget_usd in POST /v1/tasks body (USER_GUIDE.md:370) ✅ works
Per repo, Blueprint agent.maxBudgetUsd does not exist — see below
Platform default ❌ none by design; unlimited when unset (USER_GUIDE.md:231)

Documented gap — NOW OWNED BY #748, do not edit those lines here. The per-repo Blueprint budget default promised at USER_GUIDE.md:226/:231 is unimplemented (cdk/src/constructs/blueprint.ts has no maxBudgetUsd prop; it implements maxTurns at line 90). #748 is the filed follow-up and is the sole owner of USER_GUIDE.md:226/:231 and REPO_ONBOARDING.md:125. This issue must only link #748 from its cost section — editing those lines here would collide with #748 in the same file. Scope split approved by @scottschreckengaust 2026-08-12. Original analysis retained for context:

USER_GUIDE.md:226 and :231 both promise a per-repo Blueprint budget default ("Overrides the per-repo Blueprint default"), but cdk/src/constructs/blueprint.ts has no maxBudgetUsd prop — it implements maxTurns (line 90) and not the budget. RepoConfig.max_budget_usd exists (repo-config.ts:39) and cli/src/repo-onboard.ts:116 preserves an existing value, but nothing can write one. So the Blueprint row is currently aspirational. Either document it as unavailable (recommended for this issue — docs must match code) or file a follow-up to add the prop; do not leave the docs claiming a knob that isn't there.

Unlimited-by-default is the deliberate posture — pair it with the escape hatch. Since no platform budget default applies, the documented mitigation is switching to a lighter-token model rather than relying on a cost ceiling:

  • Per repo: blueprint agent.modelId (e.g. us.anthropic.claude-sonnet-4-6) — no code change, no redeploy of the agent
  • Per task: model_id in the task payload
  • Platform-wide: the bedrockModels context + the default-model call sites documented above
  • Note the model must be in the IAM grant list or the task fails at turn 0 with AccessDenied — the grant is the gate, so a lighter model is only reachable if granted

Trust boundary on the number. cost_usd is the Claude Agent SDK's client-side estimate from a price table bundled at SDK build time, not authoritative billing — see docs/guides/COST_ATTRIBUTION.md:6, which warns it drifts when "the SDK version does not recognize a model." Link it, and state that authoritative cost comes from Cost Explorer / CUR 2.0.

4c. Add "verify the SDK prices the new model" as a documented model-bump step. Because the budget guardrail is computed from the SDK's bundled price table, an unrecognized model silently degrades max_budget_usd enforcement. Verified for Opus 5 on the pinned SDK (0.2.110 / CLI 2.1.191): both Opus 4.8 and Opus 5 imply exactly $5.00/MTok, so the table recognizes it. Make this a cheap standing check for future bumps — run agent/scripts/diagnostics/test_sdk_smoke.py with ANTHROPIC_MODEL set, divide reported cost by input tokens, confirm the implied rate matches published Bedrock pricing. A wildly-off or $0.00 result means the price table doesn't know the model and budgets cannot be trusted.

5. Add a doc-drift test — the durable fix. cdk/test/constructs/bedrock-models.test.ts:83 already proves the cross-language regex-grep pattern works (it greps agent/src/config.py for the ANTHROPIC_MODEL fallback). Extend that idea: assert the documented default in docs/guides/DEVELOPER_GUIDE.md and agent/README.md matches the config.py fallback, so the next model bump fails CI instead of silently rotting the docs. This is why items 2-3 recurred in the first place — nothing guarded them.

Acceptance criteria

  • Canonical section added with layer table, env-var table, precedence chain, and the bare-vs-prefixed rule incl. the verified ValidationException example
  • All four stale Sonnet-4.6 references corrected
  • REPO_ONBOARDING.md:123 source attribution corrected (Python literal, not CDK stack props)
  • agent/README.md:126 haiku default corrected to the us.-prefixed profile ID
  • DEFAULT_REVISE_MODEL_ID comment resolved (constant restored, or comment corrected to match the assertion)
  • Doc-drift test added and failing when the documented default is edited out of sync
  • "Cost and model selection" section added: per-token-rate-vs-token-volume table, the "Where do I set max_budget_usd?" surface table (CLI / REST / Blueprint / platform), unlimited-by-default posture + the lighter-model escape hatch, and the cost_usd-is-an-estimate boundary linking COST_ATTRIBUTION.md
  • Blueprint max_budget_usd gap linked, not fixed here — the cost section references docs(cost): "Where do I set max_budget_usd?" has no complete answer — Blueprint knob is documented but unimplemented #748 as the owner of USER_GUIDE.md:226/:231. This issue must NOT edit those two lines (scope split, avoids a same-file collision with docs(cost): "Where do I set max_budget_usd?" has no complete answer — Blueprint knob is documented but unimplemented #748). REPO_ONBOARDING.md:123 (model_id) is in scope here; :125 (max_budget_usd) is not.
  • Model-bump checklist includes "verify the SDK price table recognizes the new model" with the implied-rate check
  • mise //docs:sync run — docs/src/content/docs/ mirrors regenerated, never hand-edited
  • mise run build green

Notes

Docs only, plus a test. Nothing here changes deployed behavior; the drift test is additive.

Metadata

Metadata

Labels

P1medium priorityagent-runtimePython agent container: pipeline, runner, hooks, prompts, tools, DockerfileapprovedWhen an issue has been approved and readydocumentationImprovements or additions to documentationinfra-cdkCDK stacks/constructs, bootstrap, deploy topology, tags, IAM wiring, teardown

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions