fix(llm): scope prompt cache keys to Moonshot APIs - #2535
Conversation
Third-party Kimi-compatible endpoints can reject the Moonshot-specific prompt_cache_key parameter. Only attach session cache keys to known Moonshot API hosts and cover both custom and official endpoint behavior.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR fixes a compatibility regression for third-party Kimi-compatible endpoints by only sending Moonshot’s prompt_cache_key when the configured base_url points at official Kimi/Moonshot hosts, avoiding 400 “unsupported parameter” errors on providers like Nvidia NIM.
Changes:
- Add allowlisted host detection for whether
prompt_cache_keyshould be sent forkimiproviders. - Update
create_llmto includeprompt_cache_keyonly for official Kimi/Moonshot endpoints. - Add tests covering both omission for third-party endpoints and retention for official endpoints.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
src/kimi_cli/llm.py |
Introduces host allowlist logic and gates prompt_cache_key injection accordingly. |
tests/core/test_create_llm.py |
Adds regression tests ensuring third-party endpoints don’t receive prompt_cache_key while official endpoints still do. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def _supports_prompt_cache_key(base_url: str) -> bool: | ||
| return urlparse(base_url).hostname in MOONSHOT_API_HOSTS |
There was a problem hiding this comment.
Good catch - api.moonshot.ai/v1 parses entirely as a path, so hostname is None and caching was silently dropped for an official host.
Fixed in 25c37b9: _supports_prompt_cache_key now prefixes // when the value has no //, so both forms resolve to the same hostname. Kept it to that rather than assuming https://, since the check only cares about the host.
Extended test_create_llm_keeps_prompt_cache_key_for_moonshot_endpoints with api.moonshot.ai/v1 and api.kimi.com/coding/v1. Both fail on the previous commit (KeyError: prompt_cache_key) and pass now; tests/core/test_create_llm.py is 33 passed.
Community PRs (ported with kimi_cli->cran_code adaptation): - MoonshotAI#2507 ACP AskUserQuestion signals QuestionNotSupported (no phantom dismissal) - MoonshotAI#2520 fork/undo context truncation aligns to wire turns by text matching - MoonshotAI#2572 double-encoded tool args; MoonshotAI#2530 shell pipe hang - MoonshotAI#2592 media results degrade gracefully when model lacks image_in - MoonshotAI#2539 MCP tool-name normalization with collision guard; MoonshotAI#2535 cache-key scoping kimi-code Tier-1 (compaction quality): - compact.md rewritten as first-person handoff note - summary prefix teaches 'notes, not proof — verify before relying' - post-compaction token estimate adds system-prompt + tool-schema overhead
Related Issue
Resolve #2534
Description
Third-party Kimi-compatible endpoints no longer receive Moonshot's
prompt_cache_keyparameter, while official Kimi and Moonshot APIs retain session caching.Checklist
make gen-changelogto update the changelog.make gen-docsto update the user documentation.