efficient-inference repositories
Trending repositories tagged efficient-inference, ranked by trending score. Updated daily.
-
1
cvlab-kaist/GRACE
Official implementation of "GRACE: Generation-Aware Latent Compression for Efficient Video Generation" (arXiv 2610.10524)
GitHub · Python · 34 stars · 3 forks · +9 stars in 24 hours · Trending score 1.62
-
2
LynnReal-AI/LynnReal
Hand-controlled video world models with full-sequence and three-step causal generation.
GitHub · Python · 80 stars · 2 forks · +19 stars in 24 hours · Trending score 1.45
-
3
AI-Efficiency/Awesome-Efficient-Diffusion
A curated list of papers and code on efficient diffusion models for image, video, world modeling, and language generation. Covering acceleration, quantization, compression, caching, and distillation.
GitHub · 212 stars · 11 forks · +3 stars in 24 hours · Trending score 1.16
-
4
AI-Efficiency/Awesome-Model-Quantization
A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.
GitHub · 2,470 stars · 246 forks · 0 stars in 24 hours · Trending score 0.86
-
5
LAMDA-Model-Reuse/ORBIT
ORBIT: A Unified Toolbox for LLM Routing. Unified LLM/model routing, cost-aware inference, multimodal benchmarks, and reproducible evaluation. Accepted by Frontiers of Computer Science (2026).
GitHub · Python · 19 stars · 1 forks · +2 stars in 24 hours · Trending score 0.72
-
6
ranjithrajv/awesome-cpu-first-ai
Curated, evidence-backed list of runtimes, formats & tools for running AI inference on CPU — start with CPU, justify the GPU.
GitHub · Python · 5 stars · 4 forks · 0 stars in 24 hours · Trending score 0.07
-
7
Picovoice/picollm
On-device LLM Inference Powered by X-Bit Quantization
GitHub · Python · 319 stars · 26 forks · 0 stars in 24 hours · Trending score 0.00
-
8
vdev-ctrl/Adaptive-Learnable-Hierarchical-Routing-
This repo is a alternative solution for standard self-attention. It uses tree routing with learnable functions for cheaper self-attention while not compromising accuracy.
GitHub · Python · 43 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00
-
9
sejong-rcv/ST-Pruner
Official implementation of "Don't Ask the LLM: Visual Token Pruning with Spectral Structure and Semantic Relevance" (WACV 2027)
GitHub · Python · 23 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
10
KooroshRH/AReA
Official implementation of "AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers"
GitHub · Python · 153 stars · 5 forks · 0 stars in 24 hours · Trending score 0.00
-
11
yxwang1215/Glance-ETD
Glance, then Select: Evidence-Tempered Visual Token Selection for Extremely Compressed Multimodal LLMs
GitHub · Python · 9 stars · 0 forks · Trending score 0.09