Skip to content

chore(gallery): add Huihui Qwen3.8 Q5/Q6 builds - #12592

Open
localai-org-maint-bot wants to merge 1 commit into
masterfrom
cron/model-gallery-20261009-run4
Open

localai-org-maint-bot wants to merge 1 commit into
masterfrom
cron/model-gallery-20261009-run4

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

Description

Add Q5_K and Q6_K GGUF builds of Huihui Qwen3.8 27B to the existing variant group. These provide intermediate quantizations between the existing Q4_K and Q8_0 builds. Both preserve the llama.cpp MTP settings, sampling defaults, and shared BF16 vision projector. Document explicit installation commands.

Notes for Reviewers

  • Model source: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF/tree/b146b417080fd3e0e842e9539e69921707360aae
  • All new download URLs are revision-pinned. SHA256 values and byte sizes for both weights and the projector match Hugging Face LFS metadata and independent HTTP HEAD x-linked-etag / x-linked-size responses.
  • Parsed the full gallery and verified 2,085 unique names, valid non-nested variant references, matching weight/projector paths, unchanged Q8 runtime settings in the new builds, and 2,082 unrelated entries unchanged. git diff --check passes.
  • go test ./core/gallery -count=1 cannot start in this environment: generated pkg/grpc/proto is missing, and protoc is unavailable. No inference test or full weight download was performed.
  • Based on master at 1997c6cb832711db333de69c50b11913268a2bf7.

Signed commits

  • Human DCO sign-off remains required; repository policy prohibits an AI from supplying it. The commit includes Assisted-by: Codex:gpt-6.
  • Documentation updated (docs/content/) for user-facing changes

Offer intermediate quantizations alongside the existing Q4 and Q8 builds.
Keep the shared BF16 projector and llama.cpp MTP configuration. Pin new
files to a Hugging Face revision and document explicit variant selection.

Assisted-by: Codex:gpt-6

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant