GitHub · Python
xorbitsai/inference
Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
- Stars
- 9,667
- Forks
- 901
- Stars gained in 24 hours
- +7
- Forks gained in 24 hours
- 0
Topics: artificial-intelligence, diffusers, gemma, glm, glm-5-3, inference, kimi, kimi-k3, llama-cpp, llamacpp
Latest data point: Nov 20, 2026 · 9,667 stars · 901 forks