gguf repositories

Trending repositories tagged gguf, ranked by trending score. Updated daily.

  1. 1
    sgaofen/humanizer-local-model

    Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts

    GitHub · Python · 1,001 stars · 163 forks · +625 stars in 24 hours · Trending score 4.98

  2. 2
    qybaihe/bla

    🛡️ BLA (bla-agents / Blind LLM Agents) — Open-source Blue Defense MCP server & CLI protecting JS/TS, Electron ASAR, WASM, Python, Native Binaries, SQLite, GGUF/Safetensors & Android DEX against AI reverse-engineering agents (REA).

    GitHub · TypeScript · 6,260 stars · 778 forks · +154 stars in 24 hours · Trending score 4.42

  3. 3
    mw00/project-maya

    Run GLM-5.3 (744B), DeepSeek-V4.1-Flash (748B), MiniMax-M3.1-Flash (428B), GLM-5.3-Flash (321B) and Hy4-Flash (220B) on your own GPUs - from 8 GB of VRAM, NVIDIA and AMD, one dashboard, OpenAI- and Anthropic-compatible API.

    GitHub · C++ · 3,094 stars · 430 forks · +69 stars in 24 hours · Trending score 3.63

  4. 4
    simplexai-labs/PolyStrata

    Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

    GitHub · C++ · 1,280 stars · 166 forks · +60 stars in 24 hours · Trending score 3.63

  5. 5
    synchopate/llama-modchip

    Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).

    GitHub · Python · 1,387 stars · 134 forks · +38 stars in 24 hours · Trending score 3.55

  6. 6
    CapriChooms/CapriCore

    Capri — an AI desktop companion for Windows with local/API models, Hugging Face, optional voice, memory and permission-based desktop tools.

    GitHub · JavaScript · 259 stars · 29 forks · +74 stars in 24 hours · Trending score 3.53

  7. 7
    wici-ai/inferssd

    Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

    GitHub · C++ · 1,262 stars · 113 forks · +32 stars in 24 hours · Trending score 3.48

  8. 8
    AlexsJones/llmfit

    Hundreds of models & providers. One command to find what runs on your hardware.

    GitHub · Rust · 39,127 stars · 2,536 forks · +64 stars in 24 hours · Trending score 3.41

  9. 9
    MakazhanAlpamys/Soup

    Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

    GitHub · Python · 10,666 stars · 1,777 forks · +43 stars in 24 hours · Trending score 3.38

  10. 10
    JakeATX/llamAmpere

    llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx. 120+ TPS, 3300+ prefill for QFN 3.8.

    GitHub · C++ · 403 stars · 50 forks · +87 stars in 24 hours · Trending score 3.35

  11. 11
    Veloresearch/kinetic

    The open engine for running AI on your own computer. Fast, exact, open source.

    GitHub · Rust · 202 stars · 20 forks · +62 stars in 24 hours · Trending score 3.17

  12. 12
    Helldez/BigMoeOnEdge

    Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

    GitHub · C++ · 1,033 stars · 107 forks · +23 stars in 24 hours · Trending score 3.07

  13. 13
    huawei-csl/MACaQ

    SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.

    GitHub · C++ · 380 stars · 22 forks · +22 stars in 24 hours · Trending score 2.98

  14. 14
    Bai-Cute/big-blue-fish-desktop-pet

    蓝色大肥鱼 Windows AI 桌宠,内置 4B 本地大模型,以鲸鱼娘人设结合天气、科技新闻和当前应用,通过文字气泡温柔陪伴。支持一键安装与离电暂停推理。Big Blue Fish desktop pet with local LLM.

    GitHub · C# · 374 stars · 28 forks · +51 stars in 24 hours · Trending score 2.97

  15. 15
    RustingStudio/RustingStrata

    Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.

    GitHub · Rust · 291 stars · 36 forks · +31 stars in 24 hours · Trending score 2.90

  16. 16
    Video-Rebirth/ComfyUI-HyperFlow-Flash

    ComfyUI custom nodes and workflow templates for HyperFlow H3 Flash GGUF checkpoints

    GitHub · Python · 109 stars · 21 forks · +21 stars in 24 hours · Trending score 2.61

  17. 17
    AtomicBot-ai/Atomic-Chat

    Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

    GitHub · TypeScript · 2,034 stars · 234 forks · +21 stars in 24 hours · Trending score 2.50

  18. 18
    prabanta-dev/janas

    37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

    GitHub · C · 735 stars · 73 forks · +11 stars in 24 hours · Trending score 2.41

  19. 19
    off-grid-ai/OGAM

    The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.

    GitHub · TypeScript · 3,398 stars · 339 forks · +7 stars in 24 hours · Trending score 2.24

  20. 20
    Indras-Mirror/Xolotl

    Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s on agent tasks, ~41 in fast mode, 512K context. Plus GLM-5.3-Flash, Qwen3.8 and LFM3-Omni. MoE experts tiered across VRAM, pinned RAM and NVMe, with Xolotl Direct NVMe->VRAM. OpenAI + Anthropic API.

    GitHub · C++ · 545 stars · 67 forks · +12 stars in 24 hours · Trending score 2.18

  21. 21
    scarletkc/colab-playground

    A desktop app for the Google Colab CLI: accounts, GPU sessions with cost guards, port forwarding and plugins.

    GitHub · Rust · 297 stars · 23 forks · +87 stars in 24 hours · Trending score 2.15

  22. 22
    affifuddin-hazam/ghost-infer

    Fast local LLM inference for the GPU you already have: one Rust binary, GGUF models, an OpenAI-compatible server and a chat page with a ghost.

    GitHub · HTML · 246 stars · 17 forks · +3 stars in 24 hours · Trending score 2.15

  23. 23
    StartLuxLabs/StartLux-Decision

    Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

    GitHub · Python · 1,776 stars · 169 forks · +9 stars in 24 hours · Trending score 2.04

  24. 24
    nobodywho-ooo/nobodywho

    NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

    GitHub · Rust · 1,691 stars · 102 forks · +16 stars in 24 hours · Trending score 2.03