sparse-attention repositories

Trending repositories tagged sparse-attention, ranked by trending score. Updated daily.

  1. 1
    veda-sparse/Miowtion

    Sparse acceleration and LoRA fine-tuning of MiniMax-H3 on resource-constrained machines.

    GitHub · Python · 62 stars · 6 forks · +24 stars in 24 hours · Trending score 2.33

  2. 2
    Interactive-Active-AI/NextTransformer

    :sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers/LLM, with compute-matched benchmarking and a platform for autonomous architecture research.

    GitHub · Python · 489 stars · 41 forks · +6 stars in 24 hours · Trending score 2.15

  3. 3
    Tencent-Hunyuan/Prism

    We propose Prism, a dynamic sparse attention framework for natively training joint video-audio generation models at 2K.

    GitHub · Python · 1,518 stars · 12 forks · +1 stars in 24 hours · Trending score 0.45

  4. 4
    skylight-org/skylight-research

    Advancing the frontier of efficient AI

    GitHub · Python · 70 stars · 16 forks · 0 stars in 24 hours · Trending score 0.29

  5. 5
    HKUSTDial/flash-sparse-attention

    Trainable fast and memory-efficient sparse attention

    GitHub · Python · 750 stars · 54 forks · −9 stars in 24 hours · Trending score 0.19

  6. 6
    Clare-Nie/MemOnce

    Official implementation of "MemOnce: A Scalable Memory Architecture for Large Language Models".

    GitHub · Python · 10 stars · 1 forks · 0 stars in 24 hours · Trending score 0.08

  7. 7
    SOLRICKS/comfyui-solricks

    Solricks is an Istanbul-based AI production and technology company specializing in cinematic content, custom AI models, VFX, software and enterprise AI solutions.

    GitHub · Python · 14 stars · 13 forks · 0 stars in 24 hours · Trending score 0.05

  8. 8
    Tencent-Hunyuan/FlashTopK

    Fast, exact fused top-k selection for learned sparse attention training and inference, from HyScale (Tencent Hy & University of Edinburgh).

    GitHub · Python · 16 stars · 2 forks · 0 stars in 24 hours · Trending score 0.02

  9. 9
    zechengtang/Spark-H3

    Adaptive Block Sparse Attention for MiniMax-H3

    GitHub · Python · 161 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00

  10. 10
    Charlotte-CharMLab/Fibottention

    Official Repository of "Fibottention: Inceptive Visual Representation Learning with Diverse Attention Across Heads"

    GitHub · Python · 20 stars · 2 forks · 0 stars in 24 hours · Trending score 0.00

  11. 11
    Reederey87/glm53-flash-exl3-2x-dgx-spark

    GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec decode

    GitHub · Python · 65 stars · 5 forks · 0 stars in 24 hours · Trending score 0.00

  12. 12
    FemtoRhythm/pyramis-l3d

    Pyramis-L3D: hierarchical sparse attention research prototype (code only; weights on Hugging Face)

    GitHub · Python · 26 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  13. 13
    veda-sparse/Veda-on-ComfyUI

    Veda Sparse Attention for ComfyUI

    GitHub · Python · 204 stars · 18 forks · 0 stars in 24 hours · Trending score 0.00

  14. 14
    vdev-ctrl/Adaptive-Learnable-Hierarchical-Routing-

    This repo is a alternative solution for standard self-attention. It uses tree routing with learnable functions for cheaper self-attention while not compromising accuracy.

    GitHub · Python · 43 stars · 4 forks · 0 stars in 24 hours · Trending score 0.00