AttributeError: input_cost_per_token when using an unregistered model (e.g. Fireworks DeepSeek V4 Flash)

[kalch]

Bug report: AttributeError: input_cost_per_token when using an unregistered model (e.g. Fireworks DeepSeek V4 Flash)

Repo: AnswerDotAI/fastllm
Type: Bug
Severity: Medium — crashes at the end of every turn for any model missing cost metadata, after the response has already been delivered.

Summary

Calling acomplete / AsyncChat with a valid vendor-prefixed model that is not present in either model_info_registry (fastllm/types.py) or the bundled litellm price DB (fastllm/model_prices.json) succeeds over the wire, streams the full response, and then crashes while tracking usage with:

AttributeError: input_cost_per_token

The metadata lookup returns an empty AttrDict (via dict2obj({})), and the API-specific cost() functions access pricing fields by attribute, e.g. openai_chat.py:

cost  = in_txt  * m.input_cost_per_token  + out_txt * m.output_cost_per_token

AttrDict raises AttributeError on missing attributes (.get() works fine, attribute access does not).

Concrete example

Model: accounts/fireworks/models/deepseek-v4-flash-0731 via fireworks_ai/ prefix — a valid, working model on the Fireworks API (the unversioned deepseek-v4-flash ID returns 404). It is absent from both metadata sources, so the crash fires on completion.

Verified with fastllm 0.0.41 + shell_sage 1.1.1; same code exists on main (checked 2026-08-17).

Minimal reproduction (no API key required)

from fastllm.types import get_model_info, Usage
from fastllm.openai_chat import cost

meta = get_model_info('accounts/fireworks/models/deepseek-v4-flash-0731', 'fireworks_ai')
assert not dict(meta)                     # metadata lookup misses

cost(Usage(prompt_tokens=10, completion_tokens=5,
           raw={'prompt_tokens': 10, 'completion_tokens': 5}), meta)
# AttributeError: input_cost_per_token

End-to-end path (requires an API key): AsyncChat._call → self._track(res) → UsageStats.from_response(res) → cost=r.cost → Completion.cost property (types.py) → api.cost(self.usage, meta).

Root cause

• get_model_info(mn, vendor_name) (types.py) falls back to get_model_meta, which returns dict2obj({}) when the model is in neither model_info_registry nor litellm's model_prices.json.
• cost() in all four API modules then does attribute access on that empty object:
  • openai_chat.py:202 — m.input_cost_per_token etc.
  • openai_responses.py:251 — via tier_rate(m, 'input_cost_per_token', tier)
  • anthropic.py:336 — m.input_cost_per_token
  • gemini.py:274 — via tier_rate(m, ...) (this one is partially guarded since tier_rate uses meta.get(...), but approx_pricing in types.py does p['input_cost_per_token'] → KeyError on empty dict)
• The codebase is already defensive elsewhere against missing metadata — e.g. _prep_call uses model_info.get('max_output_tokens', 32_000) — but cost() is not.

Impact

• Any model that is valid but unregistered crashes at the end of each turn, after the response has fully streamed — confusing for users (looks like the answer caused an error), and exit code becomes non-zero.
• This includes common hosted models whose IDs are versioned (e.g. -0731) and thus never match the price DB.
• max_output_tokens also silently degrades to the 32k fallback for these models.

Suggested fix

Make cost computation defensive when metadata is missing. Either:

1. In types.py, Completion.cost property: if the resolved metadata dict is empty, return 0.0 (or skip cost entirely):

meta = get_model_info(self.model, self.vendor_name)
if not dict(meta): return 0.0

2. And/or switch the per-API cost() implementations to .get(..., 0)/tier_rate() guards so a partial metadata dict can't crash either.

Workaround (until fixed)

Register the model via fastllm's public API at startup:

from fastllm.types import register_model_info, modern_llm
register_model_info(
    'accounts/fireworks/models/deepseek-v4-flash-0731', vendor_name='fireworks_ai', **modern_llm,
    max_input_tokens=1048576, max_output_tokens=384000, max_tokens=384000,
    input_cost_per_token=1.4e-07, output_cost_per_token=2.8e-07,
    cache_read_input_token_cost=2.8e-08,
)

(pricing values taken from the bundled litellm entry for fireworks_ai/accounts/fireworks/models/deepseek-v4-flash)