sglang repositories

Trending repositories tagged sglang, ranked by trending score. Updated daily.

  1. 1
    sgl-project/sglang-omni

    High Performance Serving Runtime For Interactive Super Intelligence System

    GitHub · Python · 1,879 stars · 780 forks · +43 stars in 24 hours · Trending score 3.29

  2. 2
    datawhalechina/zero-to-sglang

    Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

    GitHub · Python · 2,306 stars · 189 forks · +37 stars in 24 hours · Trending score 3.20

  3. 3
    Zyphra/ZONOS3-Mini

    compact zero-shot voice-cloning TTS with state-of-the-art performance, leading voice control, real-time on-device.

    GitHub · Python · 260 stars · 29 forks · +19 stars in 24 hours · Trending score 2.90

  4. 4
    kvcache-ai/Mooncake

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    GitHub · C++ · 7,094 stars · 1,511 forks · +21 stars in 24 hours · Trending score 2.82

  5. 5
    ai-dynamo/dynamo

    A Datacenter Scale Distributed Inference Serving Framework

    GitHub · Rust · 8,543 stars · 1,887 forks · +18 stars in 24 hours · Trending score 2.76

  6. 6
    gpustack/gpustack

    A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

    GitHub · Python · 6,094 stars · 748 forks · +18 stars in 24 hours · Trending score 2.57

  7. 7
    loxilb-io/loxilb-inference-gateway

    Inference-aware L4/L7 load balancer for LLM serving fleets, forked from loxilb

    GitHub · Go · 73 stars · 14 forks · +15 stars in 24 hours · Trending score 2.54

  8. 8
    SemiAnalysisAI/InferenceX

    Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

    GitHub · Python · 2,074 stars · 386 forks · +13 stars in 24 hours · Trending score 2.43

  9. 9
    ovg-project/kvcached

    Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

    GitHub · Python · 2,051 stars · 243 forks · +15 stars in 24 hours · Trending score 2.12

  10. 10
    smg-project/smg

    Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

    GitHub · Rust · 735 stars · 230 forks · +4 stars in 24 hours · Trending score 1.76

  11. 11
    meshapplied/goethe

    Serve your models fast on your own infra in 3 steps.

    GitHub · Rust · 82 stars · 8 forks · +3 stars in 24 hours · Trending score 1.66

  12. 12
    hasso5703/alzeo

    Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.

    GitHub · Python · 271 stars · 24 forks · +6 stars in 24 hours · Trending score 1.60

  13. 13
    openfabric-systems/simllm

    Network-faithful simulation of LLM serving and training deployments

    GitHub · Python · 52 stars · 7 forks · +6 stars in 24 hours · Trending score 1.43

  14. 14
    AMD-AGI/Hyperloom

    An agentic system that auto-optimizes LLM workloads on AMD GPUs.

    GitHub · Python · 282 stars · 90 forks · +2 stars in 24 hours · Trending score 1.43

  15. 15
    MemTensor/MemT-1.0

    Official repository for MemT-1.0

    GitHub · Python · 545 stars · 44 forks · +2 stars in 24 hours · Trending score 1.30

  16. 16
    ome-projects/ome

    Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

    GitHub · Go · 529 stars · 107 forks · +4 stars in 24 hours · Trending score 1.27

  17. 17
    zejia-lin/BulletServe

    Boosting GPU utilization for LLM serving via dynamic spatial-temporal prefill & decode orchestration

    GitHub · Python · 57 stars · 12 forks · +4 stars in 24 hours · Trending score 1.25

  18. 18
    davanstrien/saturate

    Batch inference for datasets: rows in, any OpenAI-compatible endpoint, resumable parquet out. Congestion-aware concurrency — no tuning.

    GitHub · Python · 56 stars · 2 forks · +2 stars in 24 hours · Trending score 1.21

  19. 19
    sgl-project/SpecForge

    Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

    GitHub · Python · 1,291 stars · 400 forks · +1 stars in 24 hours · Trending score 1.20

  20. 20
    modelsphere/modelsphere

    ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.

    GitHub · Shell · 195 stars · 11 forks · +1 stars in 24 hours · Trending score 1.12

  21. 21
    ModelCloud/GPTQModel

    LLM model quantization (compression) toolkit with HW acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU, Tenstorrent, Huawei Ascent NPU, and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

    GitHub · Python · 1,294 stars · 222 forks · +1 stars in 24 hours · Trending score 1.02

  22. 22
    inclusionAI/ASpec

    A high-throughput training framework for Speculative decoding models.

    GitHub · Python · 58 stars · 8 forks · +1 stars in 24 hours · Trending score 0.92

  23. 23
    simranjeet97/AI_Engineer_Roadmap_2027

    A 10-week, staff-level curriculum and engineering roadmap focused on LLM inference systems, serving physics, custom kernels, disaggregated architectures, and production optimization.

    GitHub · Jupyter Notebook · 40 stars · 5 forks · 0 stars in 24 hours · Trending score 0.91

  24. 24
    sybil-solutions/local-studio

    Control panel for VLLM, Sglang, llama.cpp, exllamav3

    GitHub · TypeScript · 1,850 stars · 172 forks · +1 stars in 24 hours · Trending score 0.85