Automatic unloading of models after inactivity

[Seven-94]

Hi,

Thank you for the new features, especially the model router in the web UI. It’s been really useful to switch models on the fly.

I was wondering: is there an existing way to automatically unload model after a period of inactivity? I’ve noticed that model stay loaded in VRAM until manually unloaded or when switching model. Since I use llama.cpp on my main machine, this sometimes causes issues when switching to other applications that use VRAM and i forget to manually unload it. Maybe I missed a setting or a command that enables this?

I acutally use the preset.ini method --models-max 1.

If it’s not already possible, would it be feasible to add an option to unload models automatically after, say, 5 minutes of inactivity? This could help free up VRAM when the app is not in use.

Thanks for your work

[Seven-94]

I'm responding to myself: the parameter --sleep-idle-seconds allows unloading the model from VRAM after a specified number of seconds, as indicated in the PR: #18228.
The model is reloaded again when a new task arrives.
Note that in the UI, the model still appears as "active" (a green dot next to the model name), but the VRAM is indeed freed.

For example, with a preset file preset.ini:

[MOE-qwen3-30b]
m = /home/IA/modeles/Qwen3-30B-A3B-Instruct-2507-UD-Q4_K_XL.gguf
top-p = 0.80
temp = 0.7
top-k = 20
min-p = 0.0
presence-penalty = 1.0
ctx-size = 52428
ngl = 99
jinja = true
fa = on
sleep-idle-seconds = 30