bigdata repositories
Trending repositories tagged bigdata, ranked by trending score. Updated daily.
-
1
rustfs/rustfs
RustFS is an open-source, S3-compatible high-performance object storage system supporting migration and coexistence with other S3-compatible platforms such as MinIO and Ceph.
GitHub · Rust · 36,323 stars · 1,687 forks · +81 stars in 24 hours · Trending score 3.70
-
2
DataExpert-io/data-engineer-handbook
This is a repo with links to everything you'd ever want to learn about data engineering
GitHub · Jupyter Notebook · 45,225 stars · 9,438 forks · +14 stars in 24 hours · Trending score 2.55
-
3
juicedata/juicefs
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
GitHub · Go · 14,763 stars · 1,364 forks · +14 stars in 24 hours · Trending score 2.40
-
4
volcano-sh/volcano
A Cloud Native Batch System (Project under CNCF)
GitHub · Go · 6,098 stars · 1,640 forks · +4 stars in 24 hours · Trending score 1.80
-
5
databendlabs/databend
Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.
GitHub · Rust · 9,501 stars · 902 forks · +4 stars in 24 hours · Trending score 1.26
-
6
apache/shardingsphere
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
GitHub · Java · 20,832 stars · 6,911 forks · +2 stars in 24 hours · Trending score 1.18
-
7
reductstore/reductstore
High-performance, time-indexed object storage for robotics and industrial IoT
GitHub · Rust · 381 stars · 52 forks · +3 stars in 24 hours · Trending score 1.15
-
8
nebulastream/nebulastream
Data management for the sensor-edge-cloud continuum
GitHub · C++ · 105 stars · 55 forks · +2 stars in 24 hours · Trending score 0.97
-
9
marmotdata/marmot
The open source knowledgehouse for your company. Marmot knows what your data means, governs who may use it and runs every query your people and agents make, checked and recorded. Catalog, lineage, governed SQL and an MCP server in one Go binary.
GitHub · Go · 638 stars · 33 forks · +2 stars in 24 hours · Trending score 0.91
-
10
NationalSecurityAgency/datawave
DataWave is an ingest/query framework that leverages Apache Accumulo to provide fast, secure data access.
GitHub · Java · 778 stars · 303 forks · +1 stars in 24 hours · Trending score 0.90
-
11
apache/hudi
Upserts, Deletes And Incremental Processing on Big Data.
GitHub · Java · 6,386 stars · 2,545 forks · 0 stars in 24 hours · Trending score 0.79
-
12
oxnr/awesome-bigdata
A curated list of awesome big data frameworks, ressources and other awesomeness.
GitHub · 14,738 stars · 2,585 forks · 0 stars in 24 hours · Trending score 0.63
-
13
datafusion-contrib/StreamFusion
World's First OSS Flink Accelerator built on Apache DataFusion
GitHub · Java · 166 stars · 14 forks · 0 stars in 24 hours · Trending score 0.58
-
14
transferia/transferia
Open Source Cloud Native Ingestion engine
GitHub · Go · 199 stars · 29 forks · +1 stars in 24 hours · Trending score 0.54
-
15
unum-cloud/UStore
Embedded transactional database for blobs, documents & graphs, with kernel-bypass NVMe through io_uring & SPDK, GPU compaction, and one C ABI over its own, RocksDB & LevelDB engines 🗄️
GitHub · C++ · 648 stars · 35 forks · +1 stars in 24 hours · Trending score 0.54
-
16
taosdata/TDengine
High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios
GitHub · C · 25,194 stars · 5,011 forks · 0 stars in 24 hours · Trending score 0.46
-
17
apache/airavata
A general purpose Distributed Systems Framework
GitHub · Go · 159 stars · 155 forks · 0 stars in 24 hours · Trending score 0.38
-
18
scikit-hep/uproot
ROOT I/O in pure Python and NumPy.
GitHub · Python · 274 stars · 119 forks · 0 stars in 24 hours · Trending score 0.32
-
19
apache/avro
Apache Avro is a data serialization system.
GitHub · Java · 3,313 stars · 1,793 forks · 0 stars in 24 hours · Trending score 0.30
-
20
apache/celeborn
Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
GitHub · Java · 1,080 stars · 463 forks · −1 stars in 24 hours · Trending score 0.25
-
21
Netflix/genie
Distributed Big Data Orchestration Service
GitHub · Java · 1,767 stars · 378 forks · 0 stars in 24 hours · Trending score 0.18
-
22
achtman-lab/GrapeTree
GrapeTree is a fully interactive, tree visualization program, which supports facile manipulations of both tree layout and metadata. Click the first link to launch: https://achtman-lab.github.io/GrapeTree/MSTree_holder.html
GitHub · JavaScript · 92 stars · 31 forks · 0 stars in 24 hours · Trending score 0.14
-
23
arungtm775-wq/sliit-fire-sensor-app
SLIIT Fire Sensor App: Real-Time Fire Alarm Platform for Web and Desktop
GitHub · TypeScript · 153 stars · 0 forks · 0 stars in 24 hours · Trending score 0.10
-
24
NewLifeX/NewLife.XCode
20 年演进的 .NET 高性能数据中间件,聚焦极致性能、海量数据、自动建模/迁移、多级缓存、自动分表分库,支持 MySQL/SQLite/SqlServer/Oracle/Postgresql/达梦 等
GitHub · C# · 104 stars · 41 forks · 0 stars in 24 hours · Trending score 0.09