Repository navigation
Conversation
A chat completion sent to an embedding model whose config sets use_tokenizer_template runs the thinking probe, and persistProbedReasoning writes the probed reasoning slots back into the shared model config. EmbedderFingerprint hashed the whole config, so that single request moved the fingerprint and every knn router built on the embedder failed with ErrLiveEmbeddingMismatch (HTTP 503) until LocalAI was restarted. Reasoning settings never change the embedding space, so leave them out of the hash. Docs-Not-Needed: internal fingerprint computation, no user-facing config, flag or API change Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
Collaborator
|
@mudler Code review sign-off at Reviewed against current master |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
No upstream issue exists for this; the bug is described here.
EmbedderFingerprinthashes the whole model config of the embedding model behind a knn router. A chat completion sent to that embedding model (withuse_tokenizer_templateset) runs the thinking probe, andpersistProbedReasoningwrites the probed reasoning slots (DisableReasoning,DisableReasoningTagPrefill) back into the shared config. That single request moved the fingerprint, and from then on every knn router built on the embedder answered withErrLiveEmbeddingMismatch(HTTP 503) until LocalAI was restarted.Reasoning settings never change the embedding space, so this change leaves
ReasoningConfigout of the hashed copy of the config. Everything else that does affect the embeddings (model file, backend, options, …) still moves the fingerprint as before.Notes for Reviewers
core/application/router_factories_test.go("stays stable when a chat request persists probed reasoning slots") — it fails without the change and passes with it; the existingrouter_factoriesspecs still pass (12/12,ginkgo --focus router_factories ./core/application/).go vet ./core/application/andgofmtare clean. I did not rungolangci-lintor the fullmake testlocally.autorouters returned 503 right after someone chatted with the embedding model; a restart cleared it.Docs-Not-Neededin the commit).Signed commits
🤖 Generated with Claude Code