big-data repositories

Trending repositories tagged big-data, ranked by trending score. Updated daily.

  1. 1
    ClickHouse/ClickHouse

    ClickHouse® is a real-time analytics database management system

    GitHub · C++ · 51,034 stars · 9,303 forks · +51 stars in 24 hours · Trending score 3.42

  2. 2
    binhnguyennus/awesome-scalability

    The Patterns of Scalable, Reliable, and Performant Large-Scale Systems

    GitHub · 75,901 stars · 7,252 forks · +56 stars in 24 hours · Trending score 3.21

  3. 3
    apache/spark

    Apache Spark - A unified analytics engine for large-scale data processing

    GitHub · Scala · 44,590 stars · 29,593 forks · +30 stars in 24 hours · Trending score 2.84

  4. 4
    apache/datafusion

    Apache DataFusion SQL Query Engine

    GitHub · Rust · 9,620 stars · 2,663 forks · +13 stars in 24 hours · Trending score 2.32

  5. 5
    StarRocks/starrocks

    The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

    GitHub · Java · 12,272 stars · 2,684 forks · +9 stars in 24 hours · Trending score 2.19

  6. 6
    trinodb/trino

    Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

    GitHub · Java · 13,529 stars · 3,882 forks · +8 stars in 24 hours · Trending score 2.11

  7. 7
    metabrainz/listenbrainz-server

    Server for the ListenBrainz project, including the front-end (javascript/react) code that it serves and all of the data processing components that LB uses.

    GitHub · Python · 1,071 stars · 399 forks · +11 stars in 24 hours · Trending score 2.06

  8. 8
    thingsboard/thingsboard

    All-in-one IoT Platform - Device management, data collection, processing and visualization.

    GitHub · Java · 22,694 stars · 6,565 forks · +6 stars in 24 hours · Trending score 1.95

  9. 9
    apache/flink

    Apache Flink

    GitHub · Java · 26,485 stars · 14,198 forks · +5 stars in 24 hours · Trending score 1.89

  10. 10
    Erkinbek/bigtext

    Free editor for huge text and log files: opens 50 GB at once, searches at 2.5 GB/s, never freezes. A free alternative to EmEditor and UltraEdit for Windows, Linux and macOS. Written in Rust.

    GitHub · Rust · 47 stars · 0 forks · +5 stars in 24 hours · Trending score 1.52

  11. 11
    prestodb/presto

    The official home of the Presto distributed SQL query engine for big data

    GitHub · Java · 16,778 stars · 5,556 forks · +5 stars in 24 hours · Trending score 1.51

  12. 12
    apache/amoro

    Apache Amoro(incubating) is a Lakehouse management system built on open data lake formats.

    GitHub · Java · 1,207 stars · 402 forks · +4 stars in 24 hours · Trending score 1.48

  13. 13
    man-group/ArcticDB

    ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.

    GitHub · C++ · 2,652 stars · 236 forks · +2 stars in 24 hours · Trending score 1.47

  14. 14
    arkime/arkime

    Arkime is an open source, large scale, full packet capturing, indexing, and database system.

    GitHub · C · 7,583 stars · 1,180 forks · +3 stars in 24 hours · Trending score 1.39

  15. 15
    delta-io/delta-sharing

    An open protocol for secure data sharing

    GitHub · Scala · 963 stars · 245 forks · +4 stars in 24 hours · Trending score 1.34

  16. 16
    apache/paimon

    Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

    GitHub · Java · 3,452 stars · 1,438 forks · +2 stars in 24 hours · Trending score 1.34

  17. 17
    quickwit-oss/quickwit

    Cloud-native OSS search engine for observability

    GitHub · Rust · 11,804 stars · 628 forks · +3 stars in 24 hours · Trending score 1.32

  18. 18
    caioricciuti/ch-ui

    Use CH-UI to work with your data from Click House self-hosted with a user-friendly interface. CH-UI is a modern and feature-rich user interface for ClickHouse databases. It offers an intuitive platform for querying ClickHouse databases, executing queries, and visualizing metrics about your instance.

    GitHub · Go · 728 stars · 72 forks · +5 stars in 24 hours · Trending score 1.26

  19. 19
    apache/auron

    Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.

    GitHub · Rust · 1,849 stars · 252 forks · +3 stars in 24 hours · Trending score 1.23

  20. 20
    apache/calcite

    Apache Calcite

    GitHub · Java · 5,216 stars · 2,550 forks · +2 stars in 24 hours · Trending score 1.20

  21. 21
    NVIDIA/cudf-spark

    NVIDIA cuDF for Apache Spark plugin - accelerate Apache Spark with GPUs

    GitHub · Scala · 1,020 stars · 435 forks · 0 stars in 24 hours · Trending score 1.20

  22. 22
    vespa-engine/vespa

    The AI search platform

    GitHub · Java · 7,186 stars · 751 forks · +3 stars in 24 hours · Trending score 1.18

  23. 23
    apache/beam

    Apache Beam is a unified programming model for Batch and Streaming data processing.

    GitHub · Java · 8,722 stars · 4,819 forks · +1 stars in 24 hours · Trending score 1.18

  24. 24
    reductstore/reductstore

    High-performance, time-indexed object storage for robotics and industrial IoT

    GitHub · Rust · 381 stars · 52 forks · +3 stars in 24 hours · Trending score 1.15