Skip to content

Update the llama.cpp submodule to b10405 and port two API changes - #1097

Open
bogdanr wants to merge 1 commit into
utilityai:mainfrom
bogdanr:pr/bump-llama-cpp-b10405
Open

Update the llama.cpp submodule to b10405 and port two API changes#1097
bogdanr wants to merge 1 commit into
utilityai:mainfrom
bogdanr:pr/bump-llama-cpp-b10405

Conversation

@bogdanr

@bogdanr bogdanr commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

The submodule moves from b10200 to b10405. Two functions in the sampler API change in this range. The Rust wrappers now agree with them.

llama_sampler_init_penalties has a new first parameter, n_vocab. Upstream moves n_vocab from llama_sampler_data into the penalty sampler (ggml-org/llama.cpp#26520). LlamaSampler::penalties has the same new first parameter. This is a breaking change, and each caller must add n_vocab as the first argument. The mirostat and logit_bias wrappers already take n_vocab as the first parameter, so the penalties wrapper is now consistent with them.

llama_sampler_init_dry no longer has the n_ctx_train parameter. Upstream removes the full-context window from the samplers that keep a history (ggml-org/llama.cpp#26524). LlamaSampler::dry keeps its public signature. The wrapper no longer reads n_ctx_train from the model.

The same upstream change also changes the meaning of penalty_last_n. A negative value selected the context length. Upstream now sets a negative value to 0, and 0 disables the penalty. The documentation of LlamaSampler::penalties shows the new behaviour.

The reason for this update is the output of the DeepSeek-V4 models. The current submodule gives incorrect text for these models. A build from b10405 gives correct text for the same model, the same prompt and the same hardware.

cargo build --workspace, cargo clippy and cargo test --features sampler are clean with the new submodule.

One note on CI: cargo fmt --check already fails on main in examples/llguidance.rs, llama-cpp-2/src/llguidance_sampler.rs and llama-cpp-2/src/model.rs. This branch does not add to that list, and it does not correct it, because the files are not related to this change. Tell me if you prefer a separate pull request for the format.

The submodule moves from b10200 to b10405. Two functions in the sampler API
change in this range. The Rust wrappers now agree with them.

llama_sampler_init_penalties has a new first parameter, n_vocab. Upstream
moves n_vocab from llama_sampler_data into the penalty sampler (#26520).
LlamaSampler::penalties has the same new first parameter. This is a breaking
change, and each caller must add n_vocab as the first argument. The mirostat
and logit_bias wrappers already take n_vocab as the first parameter, so the
penalties wrapper is now consistent with them.

llama_sampler_init_dry no longer has the n_ctx_train parameter. Upstream
removes the full-context window from the samplers that keep a history
(#26524). LlamaSampler::dry keeps its public signature. The wrapper no longer
reads n_ctx_train from the model.

The same upstream change also changes the meaning of penalty_last_n. A
negative value selected the context length. Upstream now sets a negative value
to 0, and 0 disables the penalty. The documentation of LlamaSampler::penalties
shows the new behaviour.

The reason for this update is the output of the DeepSeek-V4 models. The
current submodule gives incorrect text for these models. A build from b10405
gives correct text for the same model, the same prompt and the same hardware.

cargo build --workspace, cargo clippy and cargo test --features sampler are
clean with the new submodule.

Signed-off-by: Bogdan Radulescu <bogdan@nimblex.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant