bigdata repositories

Trending repositories tagged bigdata, ranked by trending score. Updated daily.

  1. 1
    rustfs/rustfs

    RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.

    GitHub · Rust · 36,323 stars · 1,687 forks · +81 stars in 24 hours · Trending score 3.70

  2. 2
    DataExpert-io/data-engineer-handbook

    This is a repo with links to everything you'd ever want to learn about data engineering

    GitHub · Jupyter Notebook · 45,225 stars · 9,438 forks · +14 stars in 24 hours · Trending score 2.55

  3. 3
    juicedata/juicefs

    JuiceFS is a distributed POSIX file system built on top of Redis and S3.

    GitHub · Go · 14,763 stars · 1,364 forks · +14 stars in 24 hours · Trending score 2.40

  4. 4
    volcano-sh/volcano

    A Cloud Native Batch System (Project under CNCF)

    GitHub · Go · 6,098 stars · 1,640 forks · +4 stars in 24 hours · Trending score 1.80

  5. 5
    databendlabs/databend

    Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.

    GitHub · Rust · 9,501 stars · 902 forks · +4 stars in 24 hours · Trending score 1.26

  6. 6
    apache/shardingsphere

    Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.

    GitHub · Java · 20,832 stars · 6,911 forks · +2 stars in 24 hours · Trending score 1.18

  7. 7
    reductstore/reductstore

    High-performance, time-indexed object storage for robotics and industrial IoT

    GitHub · Rust · 381 stars · 52 forks · +3 stars in 24 hours · Trending score 1.15

  8. 8
    nebulastream/nebulastream

    Data management for the sensor-edge-cloud continuum

    GitHub · C++ · 105 stars · 55 forks · +2 stars in 24 hours · Trending score 0.97

  9. 9
    marmotdata/marmot

    The open source knowledgehouse for your company. Marmot knows what your data means, governs who may use it and runs every query your people and agents make, checked and recorded. Catalog, lineage, governed SQL and an MCP server in one Go binary.

    GitHub · Go · 638 stars · 33 forks · +2 stars in 24 hours · Trending score 0.91

  10. 10
    NationalSecurityAgency/datawave

    DataWave is an ingest/query framework that leverages Apache Accumulo to provide fast, secure data access.

    GitHub · Java · 778 stars · 303 forks · +1 stars in 24 hours · Trending score 0.90

  11. 11
    apache/hudi

    Upserts, Deletes And Incremental Processing on Big Data.

    GitHub · Java · 6,386 stars · 2,545 forks · 0 stars in 24 hours · Trending score 0.79

  12. 12
    oxnr/awesome-bigdata

    A curated list of awesome big data frameworks, ressources and other awesomeness.

    GitHub · 14,738 stars · 2,585 forks · 0 stars in 24 hours · Trending score 0.63

  13. 13
    datafusion-contrib/StreamFusion

    World's First OSS Flink Accelerator built on Apache DataFusion

    GitHub · Java · 166 stars · 14 forks · 0 stars in 24 hours · Trending score 0.58

  14. 14
    transferia/transferia

    Open Source Cloud Native Ingestion engine

    GitHub · Go · 199 stars · 29 forks · +1 stars in 24 hours · Trending score 0.54

  15. 15
    unum-cloud/UStore

    Embedded transactional database for blobs, documents & graphs, with kernel-bypass NVMe through io_uring & SPDK, GPU compaction, and one C ABI over its own, RocksDB & LevelDB engines 🗄️

    GitHub · C++ · 648 stars · 35 forks · +1 stars in 24 hours · Trending score 0.54

  16. 16
    taosdata/TDengine

    High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios

    GitHub · C · 25,194 stars · 5,011 forks · 0 stars in 24 hours · Trending score 0.46

  17. 17
    apache/airavata

    A general purpose Distributed Systems Framework

    GitHub · Go · 159 stars · 155 forks · 0 stars in 24 hours · Trending score 0.38

  18. 18
    scikit-hep/uproot

    ROOT I/O in pure Python and NumPy.

    GitHub · Python · 274 stars · 119 forks · 0 stars in 24 hours · Trending score 0.32

  19. 19
    apache/avro

    Apache Avro is a data serialization system.

    GitHub · Java · 3,313 stars · 1,793 forks · 0 stars in 24 hours · Trending score 0.30

  20. 20
    apache/celeborn

    Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.

    GitHub · Java · 1,080 stars · 463 forks · −1 stars in 24 hours · Trending score 0.25

  21. 21
    Netflix/genie

    Distributed Big Data Orchestration Service

    GitHub · Java · 1,767 stars · 378 forks · 0 stars in 24 hours · Trending score 0.18

  22. 22
    achtman-lab/GrapeTree

    GrapeTree is a fully interactive, tree visualization program, which supports facile manipulations of both tree layout and metadata. Click the first link to launch: https://achtman-lab.github.io/GrapeTree/MSTree_holder.html

    GitHub · JavaScript · 92 stars · 31 forks · 0 stars in 24 hours · Trending score 0.14

  23. 23
    arungtm775-wq/sliit-fire-sensor-app

    SLIIT Fire Sensor App: Real-Time Fire Alarm Platform for Web and Desktop

    GitHub · TypeScript · 153 stars · 0 forks · 0 stars in 24 hours · Trending score 0.10

  24. 24
    NewLifeX/NewLife.XCode

    20 年演进的 .NET 高性能数据中间件,聚焦极致性能、海量数据、自动建模/迁移、多级缓存、自动分表分库,支持 MySQL/SQLite/SqlServer/Oracle/Postgresql/达梦 等

    GitHub · C# · 104 stars · 41 forks · 0 stars in 24 hours · Trending score 0.09