exl3 repositories
Trending repositories tagged exl3, ranked by trending score. Updated daily.
-
1
TandemLLM/TandemEngine
TandemEngine: local inference that drafts and verifies its own tokens, behind an OpenAI-compatible API.
GitHub · Python · 82 stars · 17 forks · +8 stars in 24 hours · Trending score 1.72
-
2
samuelcardillo/glm-5.3-flash-2x-rtx-pro-6000-blackwell
Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.
GitHub · Python · 65 stars · 6 forks · +7 stars in 24 hours · Trending score 1.36
-
3
Yamz-Labs/kyojin
Kyojin: open-source engine that runs 100 GB-class open models (GLM-5.3, Qwen3.8, DeepSeek V4.1) on one desktop machine. AMD Strix Halo, NVIDIA RTX and Apple Mac. OpenAI API, built on ExLlamaV3.
GitHub · Python · 245 stars · 39 forks · +3 stars in 24 hours · Trending score 1.32
-
4
jaedync/crucible-strix-halo
Run Qwen3.8-Flash-Next on one AMD Strix Halo (Ryzen AI Max+ 395, gfx1151). EXL3 4.05 and 3.05 bpw packs, own ROCm/HIP kernels, OpenAI-compatible server, 4 sessions of 262K context, setup guide included.
GitHub · Python · 35 stars · 7 forks · 0 stars in 24 hours · Trending score 0.79
-
5
vcruz305/vllm-exl3
Serve EXL3 (ExLlamaV3 trellis) quantized models on vLLM fork runtimes — any architecture, mixed per-layer bitrates, composable with source-format non-routed weights
GitHub · Python · 27 stars · 14 forks · +1 stars in 24 hours · Trending score 0.61
-
6
bertholomus/beam-1.0-2x-dgx-spark
Reflection Beam-1.0-501B-A23B (Apache-2.0) on two DGX Spark / GB10: EXL3 3-bit, TP2 with DFlash2 on TensorFold (868K context) or vLLM (655K)
GitHub · Shell · 9 stars · 0 forks · +1 stars in 24 hours · Trending score 0.46
-
7
Enntity/sparkglm
Fast, capable GLM-5.3-Flash FP4-Local on two DGX Sparks: opinionated Atlas serving and reproducible inference research.
GitHub · Python · 76 stars · 3 forks · 0 stars in 24 hours · Trending score 0.36
-
8
pepuscz/strix-halo-llm
DeepSeek-V4.1-Flash (1M context) and GLM-5.3-Flash on a BOSGAME M5 (Ryzen AI Max+ 395, Radeon 8060S, 128 GiB) with two RX 7900 XTX over OCuLink: pinned Ansible recipes and measured results. Also the M5-only Qwen and DeepSeek setups.
GitHub · Python · 31 stars · 2 forks · 0 stars in 24 hours · Trending score 0.29
-
9
troycheng/vllm-mach
High-performance runtime extensions for vLLM.
GitHub · Python · 5 stars · 0 forks · 0 stars in 24 hours · Trending score 0.09
-
10
0xZKnw/mlxl3
Native EXL3 inference engine for macOS and Apple Silicon. Rust, MLX and Metal runtime, with a SwiftUI app and CLI.
GitHub · Rust · 8 stars · 2 forks · 0 stars in 24 hours · Trending score 0.08
-
11
Cedriceuh/freetoken-rdna3
Fast local LLM inference on AMD Radeon RX 7900 XTX / XT (RDNA3, ROCm): big Mixture-of-Experts models like Qwen3.8-Flash-Next on one or two consumer GPUs. Up to 4 parallel agents with 262k context each, or one 1M-token context, OpenAI/Anthropic API, NVFP4 and EXL3 checkpoints. A tuned build of FreeToken.
GitHub · Python · 12 stars · 0 forks · 0 stars in 24 hours · Trending score 0.06
-
12
WestWaters/pollard-weights
A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flash. Ships the residency calculator, measurement harnesses, and the build recipes behind it. Predictions validated against public benchmarks.
GitHub · Python · 33 stars · 7 forks · 0 stars in 24 hours · Trending score 0.00
-
13
vcruz305/GLM-5.3-Flash-EXL3-K2-DGX-Spark-recipe
vLLM recipe: GLM-5.3-Flash EXL3 K2 on one DGX Spark GB10. Native MTP k=2. Measured tok/s.
GitHub · Python · 26 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00
-
14
r0b0tlab/qwen38-exl3-dflash2
Qwen3.8-27B EXL3 (4.00 bpw) + DFlash2 speculative decoding for ExLlamaV3, validated at 262k context on a 24 GB RTX 3090
GitHub · Python · 25 stars · 3 forks · 0 stars in 24 hours · Trending score 0.00
-
15
0xSero/DeepSeek-V4.1-Flash-Two-Sparks
DeepSeek-V4.1-Flash on two DGX Sparks: EXL3 routed experts, 262k context, 2M-token KV, vision, tools, MTP, bundled with Pi
GitHub · Shell · 96 stars · 8 forks · 0 stars in 24 hours · Trending score 0.00
-
16
Aevonix/GLM-5.3-Flash-EXL3-4x-RTX-PRO-6000-TensorFold
GLM-5.3-Flash EXL3 on 4x RTX PRO 6000 with TensorFold: one-command recipe and engine patches. By Aevonix Research in collaboration with Mia's AI Lab.
GitHub · Python · 11 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00
-
17
Yuepixel/tensorfold-qwen38-exl3-lookup
Qwen3.8-Flash-Next EXL3 3.05bpw on ONE DGX Spark (GB10): ~2.4x prefill (700->1777 t/s), prompt-lookup drafter, native vision/video, 262k ctx. Unofficial TensorFold v0.6.5 fork, 18 patches, byte-exact receipts.
GitHub · Python · 9 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00
-
18
sudoingX/dgx-spark-deepseek-v41
DeepSeek-V4.1-Flash on two NVIDIA DGX Sparks: EXL3 weights from the official checkpoint, upstream vLLM with our patches and decode kernels
GitHub · Python · 42 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00
-
19
lucataco/DeepSeek-V4.1-Flash-2x-DGX-Spark
DeepSeek-V4.1-Flash-1017 on two NVIDIA DGX Sparks (GB10) with TensorFold TP2: scripts, chat-template fix, speed and quality results
GitHub · Python · 8 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00
-
20
Elevatormusic/DeepSeek-V4.1-Flash-1017-EXL3-TensorFold-2x-DGX-Spark
DeepSeek-V4.1-Flash-1017 EXL3 2.9bpw on 2x NVIDIA DGX Spark (GB10) with TensorFold: 2.1-2.3x the vLLM kit's decode speed.
GitHub · Python · 78 stars · 11 forks · 0 stars in 24 hours · Trending score 0.00
-
21
parth-thakre/exl3-rdna4-recipe
Qwen3.8-27B (EXL3) on an AMD RX 9070 XT: ~150 tok/s for code, 96k context, one-line install (container, any Linux distro)
GitHub · Shell · 10 stars · 1 forks · 0 stars in 24 hours · Trending score 0.00
-
22
vcruz305/MiniMax-M3.1-Flash-EXL3-Spark-recipe
MiniMax-M3.1-Flash EXL3 (SAGE 2.15 bpw) on ONE NVIDIA DGX Spark: pinned vLLM + vllm-exl3 recipe, q4 KV 32K, measured
GitHub · Shell · 6 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00