Support for prompt caching

[simonw]

https://openrouter.ai/docs/features/prompt-caching

Could be a tiny bit tricky to implement. I expect whatever mechanism I use for llm-anthropic will mostly work here too:

• Implement support for prompt caching on top of new Fragments core concept llm-anthropic#28

[irthomasthomas]

I've added prompt caching for anthropic models, and I am about to work on gemini. Do you want me to make a PR now, or wait until all models are done?

irthomasthomas#18 (comment)

[irthomasthomas]

This is done.
#37

[irthomasthomas]

In case it's of interest, here is the main thread where my llm based agent built the feature irthomasthomas#18

The first attempt is also there in irthomasthomas#16 - this failed as I set it running in the wrong branch. I also switched models a few times to try and save money as the conversation grew. Gemini flash got confused, so I got it to write a post-game analysis and then included that in a fresh request, and that one went quite smoothly.