Hello, I’m trying to use ToolNeuron more seriously for local RAG/document search on Android, but I’m running into a problem with embedding models.
The built-in Nomic embedding model works, but when I try to download or import other embedding GGUF models, ToolNeuron seems to treat them like regular chat models instead of embedding models.
Models I was trying to use:
- Qwen3-Embedding-0.6B-GGUF
- BGE-M3 GGUF
- EmbeddingGemma 300M GGUF
These are embedding models, not chat models, but after downloading/importing them they seem to appear as normal GGUF/chat models. Then the RAG/document feature still acts like no embedding model is loaded.
I’m not sure if I’m missing a setting, but I don’t see a clear way to tell ToolNeuron that “This local GGUF file is an embedding model, not a chat model.”
It would be really helpful if ToolNeuron had one of these options:
- a separate “Import local embedding model” button
- a way to change an imported model from chat/GGUF to embedding
- better automatic detection for Hugging Face embedding repos
- an embedding model picker that lets users manually select a local GGUF file
I also noticed that the built-in Hugging Face search inside ToolNeuron does not seem to work the same way as the actual Hugging Face website search. On Hugging Face itself, there are more filter options and it is easier to narrow models by task, format, tags, etc. In ToolNeuron, the search/filter system seems more limited, so embedding models may not be getting detected correctly.
Is there a reason the in-app Hugging Face search uses different filters than Hugging Face’s actual search page? Could this be related to embedding models being imported as chat models?
I really like the app and I’m trying to use it fully offline for document search, PDFs, notes, and local memory. Being able to use newer embedding models like Qwen3-Embedding, BGE-M3, or EmbeddingGemma would make the RAG feature much more useful.
Thanks.
Hello, I’m trying to use ToolNeuron more seriously for local RAG/document search on Android, but I’m running into a problem with embedding models.
The built-in Nomic embedding model works, but when I try to download or import other embedding GGUF models, ToolNeuron seems to treat them like regular chat models instead of embedding models.
Models I was trying to use:
These are embedding models, not chat models, but after downloading/importing them they seem to appear as normal GGUF/chat models. Then the RAG/document feature still acts like no embedding model is loaded.
I’m not sure if I’m missing a setting, but I don’t see a clear way to tell ToolNeuron that “This local GGUF file is an embedding model, not a chat model.”
It would be really helpful if ToolNeuron had one of these options:
I also noticed that the built-in Hugging Face search inside ToolNeuron does not seem to work the same way as the actual Hugging Face website search. On Hugging Face itself, there are more filter options and it is easier to narrow models by task, format, tags, etc. In ToolNeuron, the search/filter system seems more limited, so embedding models may not be getting detected correctly.
Is there a reason the in-app Hugging Face search uses different filters than Hugging Face’s actual search page? Could this be related to embedding models being imported as chat models?
I really like the app and I’m trying to use it fully offline for document search, PDFs, notes, and local memory. Being able to use newer embedding models like Qwen3-Embedding, BGE-M3, or EmbeddingGemma would make the RAG feature much more useful.
Thanks.