Skip to content
@kvcache-ai

kvcache.ai

KVCache.AI is an open source orgnization between MADSys and top industry collaborators, focusing on efficient Agent/LLM serving.

Pinned Loading

  1. Mooncake Mooncake Public

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    C++ 6.7k 1.3k

  2. ktransformers ktransformers Public

    A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

    Python 19.6k 1.6k

  3. AgentENV AgentENV Public

    AgentENV (AENV) is a distributed platform for running agent environments at scale.

    Rust 3.6k 326

Repositories

Showing 10 of 22 repositories
  • Mooncake Public

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    kvcache-ai/Mooncake's past year of commit activity
    C++ 6,718 Apache-2.0 1,289 320 (1 issue needs help) 365 Updated Oct 5, 2026
  • firecracker Public
    kvcache-ai/firecracker's past year of commit activity
    Rust 1 Apache-2.0 8 0 3 Updated Oct 5, 2026
  • AgentENV Public

    AgentENV (AENV) is a distributed platform for running agent environments at scale.

    kvcache-ai/AgentENV's past year of commit activity
    Rust 3,591 MIT 326 32 31 Updated Oct 3, 2026
  • ktransformers Public

    A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

    kvcache-ai/ktransformers's past year of commit activity
    Python 19,566 Apache-2.0 1,590 468 (1 issue needs help) 44 Updated Oct 1, 2026
  • sglang Public Forked from sgl-project/sglang

    SGLang is a fast serving framework for large language models and vision language models.

    kvcache-ai/sglang's past year of commit activity
    Python 14 Apache-2.0 9,369 0 24 Updated Sep 30, 2026
  • transformers Public Forked from huggingface/transformers

    🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

    kvcache-ai/transformers's past year of commit activity
    Python 2 Apache-2.0 35,553 0 1 Updated Sep 30, 2026
  • kvcache-blog Public
    kvcache-ai/kvcache-blog's past year of commit activity
    JavaScript 28 MIT 23 0 4 Updated Sep 24, 2026
  • accelerate Public Forked from huggingface/accelerate

    🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

    kvcache-ai/accelerate's past year of commit activity
    Python 3 Apache-2.0 1,568 0 1 Updated Sep 10, 2026
  • vllm Public Forked from vllm-project/vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    kvcache-ai/vllm's past year of commit activity
    Python 19 Apache-2.0 23,256 0 1 Updated Sep 7, 2026
  • AFD-Ledger Public

    AFD-Ledger is an analytical toolkit for provisioning and evaluating attention–FFN disaggregated LLM inference deployments.

    kvcache-ai/AFD-Ledger's past year of commit activity
    Python 5 MIT 1 0 0 Updated Aug 7, 2026
View all repositories

Top languages

Loading…

Most used topics

Loading…