serving repositories
Trending repositories tagged serving, ranked by trending score. Updated daily.
-
1
skyzh/tiny-llm
learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen
GitHub · Python · 5,802 stars · 538 forks · +55 stars in 24 hours · Trending score 3.24
-
2
Lightning-AI/LitServe
A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.
GitHub · Python · 4,016 stars · 334 forks · +37 stars in 24 hours · Trending score 2.74
-
3
ray-project/ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
GitHub · Python · 44,352 stars · 8,296 forks · +12 stars in 24 hours · Trending score 2.49
-
4
openvinotoolkit/model_server
A scalable inference server for models optimized with OpenVINO™
GitHub · C++ · 962 stars · 297 forks · +2 stars in 24 hours · Trending score 1.14
-
5
volcano-sh/volcano
A Cloud Native Batch System (Project under CNCF)
GitHub · Go · 6,098 stars · 1,640 forks · 0 stars in 24 hours · Trending score 0.97
-
6
pixeltable/pixeltable
The backend agents build with - Multimodal database, orchestration, and serving in one file
GitHub · Python · 1,664 stars · 245 forks · +1 stars in 24 hours · Trending score 0.90
-
7
dingodb/dingo
A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.
GitHub · Java · 1,706 stars · 263 forks · +3 stars in 24 hours · Trending score 0.90
-
8
deepjavalibrary/djl-serving
A universal scalable machine learning model deployment solution
GitHub · Java · 258 stars · 100 forks · 0 stars in 24 hours · Trending score 0.22
-
9
polyaxon/haupt
Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon
GitHub · Python · 451 stars · 205 forks · −1 stars in 24 hours · Trending score 0.18
-
10
tensorflow/serving
A flexible, high-performance serving system for machine learning models
GitHub · C++ · 6,362 stars · 2,201 forks · 0 stars in 24 hours · Trending score 0.08
-
11
clearml/clearml-serving
ClearML - Model-Serving Orchestration and Repository Solution
GitHub · Python · 170 stars · 54 forks · 0 stars in 24 hours · Trending score 0.07
-
12
AI-Hypercomputer/gpu-recipes
Recipes for reproducing training and serving benchmarks for large machine learning models using GPUs on Google Cloud.
GitHub · Python · 148 stars · 94 forks · 0 stars in 24 hours · Trending score 0.02
-
13
sgyaqing/verdict
A high-performance serving engine for System One decision models.
GitHub · Rust · 13 stars · 0 forks · 0 stars in 24 hours · Trending score 0.02
-
14
PaddlePaddle/FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
GitHub · Python · 3,722 stars · 756 forks · 0 stars in 24 hours · Trending score 0.00