How to use with Aider?

[splitbrain]

From what I understand, this should be compatible with Ollama, but I can't get it to work with Aider.

I'm starting the model like this:

ramalama serve --port 11434 deepseek-r1:8b

And aider with:

OLLAMA_API_BASE=http://127.0.0.1:11434/  aider --model ollama_chat/deepseek-r1:8b

But all I get are 404s:

litellm.APIConnectionError: Ollama_chatException - {"error":{"code":404,"message":"File Not Found","type":"not_found_error"}}

The ramalama console shows:

srv  log_server_r: request: POST //api/show 172.17.0.1 404
srv  log_server_r: request: POST //api/chat 172.17.0.1 404

Can anyone give me a hint on how to get it working?

[rhatdan]

@engelmi @bmahabirbu is this something we should handle via the relay server?

RamaLama is not necessarily the exact same as ollama. we are providing a compatible API to OpenAI, but not necessarily the full Ollama API. By default when you talk to an API running within the container, this API is provided by llama.cpp.

[engelmi]

Yes, Ollama essentially spins up a server which provides a REST API used by its CLI. RamaLama CLI (run/serve) spins up a container and starts the LLM (providing the OpenAI compatible API).

@rhatdan Yes, I think this the daemon could be extended for this.

@splitbrain There is an effort to split RamaLama into a CLI ("frontend") and a server providing a REST API ("backend"):
https://github.com/containers/ramalama/tree/main/ramalama/daemon
In fact, this daemon already provides a subset of the endpoints compatible with Ollama, i.e. listing available models (/api/tags) and listing running models (api/ps). Back then we already thought of RamaLama being able to provide the backend that the Ollama CLI can communicate with. And I think implementing the full set of Ollama's REST API endpoints could be done quite easily. I am not sure about maintaining the compatibility, though. But since they would have the same issues when breaking the API, we'd at least notice it and be able to do the same migration steps.

@splitbrain I assume with OLLAMA_API_BASE=xyz and the path prefix ollama_chat/ aider is using Ollama's API definition?

[splitbrain]

Thanks for the hint that the ollama API is different from the OpenAI compatible API. Using the following works:

OPENAI_API_BASE=http://localhost:11434  aider --model openai/deepseek-r1:8b