Skip to content

chore(gallery): add Humanlike Chat Q5/Q6 variants - #12591

Open
localai-org-maint-bot wants to merge 1 commit into
masterfrom
cron/model-gallery-20261009-run3
Open

localai-org-maint-bot wants to merge 1 commit into
masterfrom
cron/model-gallery-20261009-run3

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

Description

Add Q5_K_M and Q6_K llama.cpp variants for Qwen3.8-27B Humanlike Chat and document how to select them. They fill the gap between the existing Q4_K_M and Q8_0 builds.

Both variants pin revision ba6d29fb2505241d7dae88df4fda9038999c3ca9, matching the existing step-576 adapter merge at 0.7 strength. The current upstream card describes a newer release; using the existing revision keeps the variant group consistent. Runtime settings match the existing Q8 entry, including text-only use and disabled thinking.

Notes for Reviewers

  • Verified both SHA256 values and byte sizes against independent HF LFS metadata and x-linked-etag / x-linked-size headers at the pinned revision.
  • Parsed all 2,085 gallery entries: unique names, valid variant targets, no nested variants or multiply claimed targets.
  • Confirmed all 2,082 unrelated entries remain unchanged; the parent only gains two variant references.
  • git diff --check passes.
  • go test ./core/gallery -run Test -count=1 cannot start: generated pkg/grpc/proto files are absent. No inference or full model downloads were run.
  • Publisher source: https://huggingface.co/LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-GGUF/blob/ba6d29fb2505241d7dae88df4fda9038999c3ca9/README.md

Signed commits

  • Yes, I signed my commits. Human DCO sign-off is required; repository policy prohibits AI sign-off.
  • Documentation updated (docs/content/) for user-facing changes, or not applicable

Offer intermediate quantizations between the existing Q4 and Q8 builds.
Pin both to the same publisher revision to preserve the step-576 merge.
Document explicit variant selection.

Assisted-by: Codex:gpt-6

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant