sglang repositories
Trending repositories tagged sglang, ranked by trending score. Updated daily.
-
1
sgl-project/sglang-omni
High Performance Serving Runtime For Interactive Super Intelligence System
GitHub · Python · 1,879 stars · 780 forks · +43 stars in 24 hours · Trending score 3.29
-
2
datawhalechina/zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
GitHub · Python · 2,306 stars · 189 forks · +37 stars in 24 hours · Trending score 3.20
-
3
Zyphra/ZONOS3-Mini
compact zero-shot voice-cloning TTS with state-of-the-art performance, leading voice control, real-time on-device.
GitHub · Python · 260 stars · 29 forks · +19 stars in 24 hours · Trending score 2.90
-
4
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
GitHub · C++ · 7,094 stars · 1,511 forks · +21 stars in 24 hours · Trending score 2.82
-
5
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
GitHub · Rust · 8,543 stars · 1,887 forks · +18 stars in 24 hours · Trending score 2.76
-
6
gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
GitHub · Python · 6,094 stars · 748 forks · +18 stars in 24 hours · Trending score 2.57
-
7
loxilb-io/loxilb-inference-gateway
Inference-aware L4/L7 load balancer for LLM serving fleets, forked from loxilb
GitHub · Go · 73 stars · 14 forks · +15 stars in 24 hours · Trending score 2.54
-
8
SemiAnalysisAI/InferenceX
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
GitHub · Python · 2,074 stars · 386 forks · +13 stars in 24 hours · Trending score 2.43
-
9
ovg-project/kvcached
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
GitHub · Python · 2,051 stars · 243 forks · +15 stars in 24 hours · Trending score 2.12
-
10
smg-project/smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
GitHub · Rust · 735 stars · 230 forks · +4 stars in 24 hours · Trending score 1.76
-
11
meshapplied/goethe
Serve your models fast on your own infra in 3 steps.
GitHub · Rust · 82 stars · 8 forks · +3 stars in 24 hours · Trending score 1.66
-
12
hasso5703/alzeo
Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.
GitHub · Python · 271 stars · 24 forks · +6 stars in 24 hours · Trending score 1.60
-
13
openfabric-systems/simllm
Network-faithful simulation of LLM serving and training deployments
GitHub · Python · 52 stars · 7 forks · +6 stars in 24 hours · Trending score 1.43
-
14
AMD-AGI/Hyperloom
An agentic system that auto-optimizes LLM workloads on AMD GPUs.
GitHub · Python · 282 stars · 90 forks · +2 stars in 24 hours · Trending score 1.43
-
15
MemTensor/MemT-1.0
Official repository for MemT-1.0
GitHub · Python · 545 stars · 44 forks · +2 stars in 24 hours · Trending score 1.30
-
16
ome-projects/ome
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
GitHub · Go · 529 stars · 107 forks · +4 stars in 24 hours · Trending score 1.27
-
17
zejia-lin/BulletServe
Boosting GPU utilization for LLM serving via dynamic spatial-temporal prefill & decode orchestration
GitHub · Python · 57 stars · 12 forks · +4 stars in 24 hours · Trending score 1.25
-
18
davanstrien/saturate
Batch inference for datasets: rows in, any OpenAI-compatible endpoint, resumable parquet out. Congestion-aware concurrency — no tuning.
GitHub · Python · 56 stars · 2 forks · +2 stars in 24 hours · Trending score 1.21
-
19
sgl-project/SpecForge
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
GitHub · Python · 1,291 stars · 400 forks · +1 stars in 24 hours · Trending score 1.20
-
20
modelsphere/modelsphere
ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.
GitHub · Shell · 195 stars · 11 forks · +1 stars in 24 hours · Trending score 1.12
-
21
ModelCloud/GPTQModel
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU, Tenstorrent, Huawei Ascent NPU, and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
GitHub · Python · 1,294 stars · 222 forks · +1 stars in 24 hours · Trending score 1.02
-
22
inclusionAI/ASpec
A high-throughput training framework for Speculative decoding models.
GitHub · Python · 58 stars · 8 forks · +1 stars in 24 hours · Trending score 0.92
-
23
simranjeet97/AI_Engineer_Roadmap_2027
A 10-week, staff-level curriculum and engineering roadmap focused on LLM inference systems, serving physics, custom kernels, disaggregated architectures, and production optimization.
GitHub · Jupyter Notebook · 40 stars · 5 forks · 0 stars in 24 hours · Trending score 0.91
-
24
sybil-solutions/local-studio
Control panel for VLLM, Sglang, llama.cpp, exllamav3
GitHub · TypeScript · 1,850 stars · 172 forks · +1 stars in 24 hours · Trending score 0.85