int4 repositories
Trending repositories tagged int4, ranked by trending score. Updated daily.
-
1
tuan3w/qwen-warp
Fast Qwen3.8-27B for one RTX 3090. 200k context, 100+ tok/s. Custom CUDA decode engine, OpenAI-compatible API.
GitHub · Python · 50 stars · 6 forks · +1 stars in 24 hours · Trending score 0.83
-
2
intel/auto-round
PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度
GitHub · Python · 1,706 stars · 205 forks · −2 stars in 24 hours · Trending score 0.64
-
3
intel/neural-compressor
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
GitHub · Python · 2,730 stars · 333 forks · 0 stars in 24 hours · Trending score 0.03
-
4
Dicklesworthstone/franken_ocr
Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.
GitHub · Rust · 347 stars · 41 forks · 0 stars in 24 hours · Trending score 0.02
-
5
lokutor-ai/oido
Oído: open-vocabulary speech recognition on a $5 ESP32-S3, faster than real time. No cloud, no NPU. English + Spanish.
GitHub · C · 141 stars · 13 forks · 0 stars in 24 hours · Trending score 0.00
-
6
DominikBucko/qwen38-flash-next-3090
Run Qwen3.8-Flash-Next on ONE RTX 3090 (24 GB) + 64 GB RAM: 128K context, up to 2,100 tok/s prefill, 43–51 tok/s decode. vLLM runtime with hot MoE experts on the GPU and cold experts computed on the CPU, INT8 KV cache, Docker, OpenAI-compatible API.
GitHub · Python · 16 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00