llama-cpp repositories

Trending repositories tagged llama-cpp, ranked by trending score. Updated daily.

  1. 1
    FrigadeHQ/yogurt

    Private AI meeting notes that stay on your computer. A free, open-source alternative to Granola for Mac, Windows and Linux.

    GitHub · TypeScript · 2,675 stars · 205 forks · +610 stars in 24 hours · Trending score 5.22

  2. 2
    sgaofen/humanizer-local-model

    Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts

    GitHub · Python · 1,001 stars · 163 forks · +625 stars in 24 hours · Trending score 4.98

  3. 3
    hyperionaidev/smolmaxxing

    maxxing the ability to run different LLMs on the smallest devices possible

    GitHub · C · 1,035 stars · 54 forks · +65 stars in 24 hours · Trending score 3.88

  4. 4
    thetahealth/mirobody

    Mirror your body in data. One standard, every source, one complete picture of you: a self-hosted health record built from lab reports, wearables and genetics, with every number traced to its source and a built-in MCP server. Run it with Docker and one LLM key, or keep every model local on llama.cpp.

    GitHub · Python · 1,626 stars · 273 forks · +125 stars in 24 hours · Trending score 3.63

  5. 5
    synchopate/llama-modchip

    Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).

    GitHub · Python · 1,387 stars · 134 forks · +38 stars in 24 hours · Trending score 3.55

  6. 6
    wici-ai/inferssd

    Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

    GitHub · C++ · 1,262 stars · 113 forks · +32 stars in 24 hours · Trending score 3.48

  7. 7
    JakeATX/llamAmpere

    llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx. 120+ TPS, 3300+ prefill for QFN 3.8.

    GitHub · C++ · 403 stars · 50 forks · +87 stars in 24 hours · Trending score 3.35

  8. 8
    altic-dev/FluidVoice

    Fastest and only macOS Dictation app with on-device STT and custom trained AI enhancement model. Windows, iOS testflight! Android, Linux waitlist open! A local Wispr Flow alternative. Follow https://x.com/fluidvoiceapp for more

    GitHub · Swift · 13,172 stars · 997 forks · +41 stars in 24 hours · Trending score 3.22

  9. 9
    Veloresearch/kinetic

    The open engine for running AI on your own computer. Fast, exact, open source.

    GitHub · Rust · 202 stars · 20 forks · +62 stars in 24 hours · Trending score 3.17

  10. 10
    Helldez/BigMoeOnEdge

    Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

    GitHub · C++ · 1,033 stars · 107 forks · +23 stars in 24 hours · Trending score 3.07

  11. 11
    huawei-csl/MACaQ

    SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.

    GitHub · C++ · 380 stars · 22 forks · +22 stars in 24 hours · Trending score 2.98

  12. 12
    Bai-Cute/big-blue-fish-desktop-pet

    蓝色大肥鱼 Windows AI 桌宠,内置 4B 本地大模型,以鲸鱼娘人设结合天气、科技新闻和当前应用,通过文字气泡温柔陪伴。支持一键安装与离电暂停推理。Big Blue Fish desktop pet with local LLM.

    GitHub · C# · 374 stars · 28 forks · +51 stars in 24 hours · Trending score 2.97

  13. 13
    RustingStudio/RustingStrata

    Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.

    GitHub · Rust · 291 stars · 36 forks · +31 stars in 24 hours · Trending score 2.90

  14. 14
    ARahim3/DigUp

    Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.

    GitHub · Swift · 6,481 stars · 518 forks · +24 stars in 24 hours · Trending score 2.75

  15. 15
    Soulmate-Halo/heterogeneous-gpu-pd-lab

    使用稠密算力卡对稀疏大显存主机进行加速,主机包括 AI Max+ 395、M3 512G、DGX Spark 等。

    GitHub · C++ · 1,284 stars · 203 forks · +19 stars in 24 hours · Trending score 2.67

  16. 16
    DevDock-AI/Speedy

    Qwen3.8-Flash-Next (125B MoE) up to 1M context on one or two RTX cards (tuned on 3080 10gb). A measured, tuned and benchmarked fork of Strata.

    GitHub · C++ · 125 stars · 8 forks · +26 stars in 24 hours · Trending score 2.59

  17. 17
    nace-ai/kapitsa

    Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai

    GitHub · C++ · 833 stars · 115 forks · +14 stars in 24 hours · Trending score 2.56

  18. 18
    cobanov/PS5LM

    llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

    GitHub · C++ · 857 stars · 77 forks · +12 stars in 24 hours · Trending score 2.36

  19. 19
    nicedreamzapp/claude-code-local

    Hit your Claude Code usage limit? Keep going free and offline on a local model: Qwen 3.8, Gemma 4, and Qwen 4 the day it ships. Runs on Apple Silicon Macs (MLX), Linux and Windows (llama.cpp) with no API key. Your code stays home.

    GitHub · Python · 3,524 stars · 650 forks · +12 stars in 24 hours · Trending score 2.34

  20. 20
    jammaru/sakutto

    A browser extension for easy on-device translation with a local LLM (PLaMo). Private, free, no server needed.

    GitHub · TypeScript · 274 stars · 32 forks · +11 stars in 24 hours · Trending score 2.30

  21. 21
    Osmantic/ODS

    ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.

    GitHub · Python · 7,857 stars · 1,120 forks · +12 stars in 24 hours · Trending score 2.29

  22. 22
    off-grid-ai/OGAM

    The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.

    GitHub · TypeScript · 3,398 stars · 339 forks · +7 stars in 24 hours · Trending score 2.24

  23. 23
    scarletkc/colab-playground

    A desktop app for the Google Colab CLI: accounts, GPU sessions with cost guards, port forwarding and plugins.

    GitHub · Rust · 297 stars · 23 forks · +87 stars in 24 hours · Trending score 2.15

  24. 24
    StartLuxLabs/StartLux-Decision

    Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

    GitHub · Python · 1,776 stars · 169 forks · +9 stars in 24 hours · Trending score 2.04