What happened?

The documentation states that you can provide a custom LITELLM_MODEL_COST_MAP_URL. However, setting the env variable does not have any effect since the value is hard-coded here and then used when calling get_model_cost_map.

A bit of context:
I started investigating this since I wanted to add a local version of this that we fetch on built time of our docker image. This way, the file would be more up to date than the local version.
The goal was to reduce the startup time which is currently around 20s mostly because of fetching the file (related to #2677 and #7605)

Relevant log output

Are you a ML Ops Team?

Yes

What LiteLLM version are you on ?

v1.77.5

Twitter / LinkedIn details

No response