mxfp4

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率，量化感知训练的精度

WHIRL (Windows HIP Inference for RDNA LLMs): native Windows C++/HIP LLM inference engine for the AMD Ryzen AI Max+ 395 (Radeon 8060S) and Radeon AI PRO R9700. Unzip and run, only the AMD driver.

The Kimi K3 and DeepSeek-V4-Flash-0731 architectures, reimplemented from scratch in pure Rust (zero dependencies) and verified 1:1 against the official code.

XTU = X Transformers Unity — hybrid (CPU-expert + GPU) inference acceleration for vLLM: mainline plugin + CPU MoE engine (AVX512-VNNI/BF16 MXFP4/FP8), long-prefill GPU streaming, SM80 (A100) port.

vLLM for AMD RDNA4 (gfx1201): Radeon AI PRO R9700 & RX 9070 XT — native MXFP4 linear kernel + spec-decode verify fix, carried as rebased branches (fork-carry model)

Tiny engine, immense models — run large MoE LLMs (gpt-oss, Mixtral, Qwen3-MoE) on ordinary machines by streaming experts from disk. OpenAI-compatible server with tool calling + hybrid cloud relay; CPU, Apple Silicon (MLX) & CUDA.

面向 8×RTX 4090（SM89）的 DeepSeek-V4-Flash-0731 MXFP4 SGLang 实验分支

Run Kimi-K3 (2.8T MoE) from SSD with a tiny readable C++17 engine—native MXFP4, io_uring, and O_DIRECT.

Run the official Kimi K3 MoE checkpoint on one 128 GB AMD Strix Halo box. ROCm-resident static weights, MXFP4 experts streamed from NVMe via io_uring. C engine, chat client, OpenAI-compatible server. Very experimental.

DeepSeek-V4.1-Flash-1017 at native precision on one RTX PRO 6000 + 128 GB RAM: 31.6 tok/s decode, 1M context. ds41d engine with VRAM/RAM/NVMe expert tiers, GPUDirect Storage and DSpark speculative decoding

Local inference engine for DeepSeek-V4-Flash (284B MoE, MXFP4 experts streamed from disk) on consumer hardware — C11, zero deps, bit-exactness gated. macOS / Linux / Windows.

Fine-tune and serve large Mixture-of-Experts models on one GPU, with 4-bit experts in GPU memory, RAM or SSD.

Qwen3-8B quantization study across vLLM, TensorRT-LLM, AutoRound, INT8, and MXFP4

4-bit GPU kernels that compute directly on packed Mixture-of-Experts weights, for training and inference.

Out-of-core LoRA fine-tuning and expert-routing measurement for Kimi K3 (2.78T MoE) on a 7.6 GB laptop: 93 layers streamed from a USB disk, forward checked against an independent C implementation, gradients checked by finite differences

MXFP4 for RDNA4 (gfx12): W4A8 FP8 expert path, LDS FP8 GEMM, grow config (dense on GPU, VRAM to the expert cache)

A dependency-light, transformers-free toolkit for data-free weight-only quantization (FP8 blockwise, MXFP8, MXFP4), with file-level multi-node support and HF / compressed-tensors output.

Run OpenAI's gpt-oss-20b on AMD Radeon (RDNA2) via a from-scratch Vulkan compute backend. No CUDA, no ROCm, no Triton.

Accelerate local LLM inference on Windows RDNA GPUs with HIP and Vulkan, outperforming llama.cpp ROCm and Vulkan builds.