etl repositories

Trending repositories tagged etl, ranked by trending score. Updated daily.

  1. 1
    datazip-inc/olake-go

    OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.

    GitHub · Go · 2,154 stars · 320 forks · +163 stars in 24 hours · Trending score 4.18

  2. 2
    apache/airflow

    Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

    GitHub · Python · 47,584 stars · 18,364 forks · +44 stars in 24 hours · Trending score 3.32

  3. 3
    vectordotdev/vector

    A high-performance observability data pipeline.

    GitHub · Rust · 22,926 stars · 2,405 forks · +22 stars in 24 hours · Trending score 2.73

  4. 4
    kestra-io/kestra

    Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.

    GitHub · Java · 30,303 stars · 3,546 forks · +18 stars in 24 hours · Trending score 2.71

  5. 5
    dagster-io/dagster

    An orchestration platform for the development, production, and observation of data assets.

    GitHub · Python · 16,444 stars · 2,427 forks · +9 stars in 24 hours · Trending score 2.36

  6. 6
    SQLMesh/sqlmesh

    Scalable and efficient data transformation framework - backwards compatible with dbt.

    GitHub · Python · 3,360 stars · 493 forks · +15 stars in 24 hours · Trending score 2.21

  7. 7
    galaxy-io/filament

    Pluggable data replication with checkpointing, batching, and integrity events

    GitHub · Go · 336 stars · 48 forks · +8 stars in 24 hours · Trending score 2.16

  8. 8
    cocoindex-io/cocoindex

    Incremental engine for long horizon agents 🌟 Star if you like it!

    GitHub · Rust · 11,923 stars · 930 forks · +13 stars in 24 hours · Trending score 1.99

  9. 9
    Open-Source-Legal/OpenContracts

    The open document intelligence platform for builders and hackers - DMS for the agentic world

    GitHub · Python · 1,550 stars · 201 forks · +8 stars in 24 hours · Trending score 1.68

  10. 10
    morph-kgc/morph-kgc

    Powerful RDF knowledge graph generation with RML mappings

    GitHub · Python · 303 stars · 63 forks · +7 stars in 24 hours · Trending score 1.60

  11. 11
    turbot/steampipe

    Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.

    GitHub · Go · 8,020 stars · 348 forks · +5 stars in 24 hours · Trending score 1.54

  12. 12
    airbytehq/airbyte

    Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

    GitHub · Python · 22,411 stars · 5,550 forks · 0 stars in 24 hours · Trending score 1.47

  13. 13
    yuyuejia/datay

    DataY 是一套轻量、高性能的数据平台,一个 JAR、自研流批一体处理引擎,自研分布式调度框架,在 2 核 4G 的机器上 就能跑通 数据集成 → 数据建模 → 数据开发 → 任务调度 → 数据应用 的完整链路。

    GitHub · Java · 50 stars · 23 forks · +3 stars in 24 hours · Trending score 1.41

  14. 14
    apache/hop

    Hop Orchestration Platform

    GitHub · Java · 1,544 stars · 493 forks · +2 stars in 24 hours · Trending score 1.28

  15. 15
    Unknown-333/awesome-data-engineering-skills

    36 data-engineering skills for AI coding agents — dbt, Airflow, Dagster, Prefect, Spark, Snowflake, BigQuery, Databricks, Kafka, Iceberg. Portable SKILL.md Agent Skills (the format Anthropic launched as Claude Skills), with idempotency, backfill, and data-quality know-how. Runs in Claude Code, Cursor, Codex, Copilot, Gemini CLI and more.

    GitHub · Python · 27 stars · 1 forks · +4 stars in 24 hours · Trending score 1.16

  16. 16
    ariacom/Seal-Report

    Database Reporting Tool and Tasks (.Net)

    GitHub · C# · 1,646 stars · 511 forks · +3 stars in 24 hours · Trending score 1.15

  17. 17
    planarnetwork/gb-transit

    DTD feed importing, timetable conversion and GTFS tooling

    GitHub · TypeScript · 36 stars · 16 forks · +3 stars in 24 hours · Trending score 1.10

  18. 18
    ucbepic/docetl

    A system for agentic LLM-powered data processing and ETL

    GitHub · Python · 4,259 stars · 471 forks · 0 stars in 24 hours · Trending score 1.08

  19. 19
    ara3d/bim-open-schema

    Representing BIM Data as Parquet

    GitHub · C# · 57 stars · 10 forks · +4 stars in 24 hours · Trending score 1.05

  20. 20
    nao1215/filesql

    loads CSV, TSV, LTSV, JSON, JSONL, Parquet, XLSX, ACH, and Fedwire files into SQLite; includes prep and frame for cleanup and in-memory transforms

    GitHub · Go · 388 stars · 12 forks · +2 stars in 24 hours · Trending score 1.04

  21. 21
    reductstore/reductstore

    High-performance, time-indexed object storage for robotics and industrial IoT

    GitHub · Rust · 381 stars · 52 forks · +2 stars in 24 hours · Trending score 1.01

  22. 22
    apache/devlake

    Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

    GitHub · Go · 3,204 stars · 842 forks · +1 stars in 24 hours · Trending score 0.97

  23. 23
    vfaner/jync

    Jync 捷同 - 开箱即用的异构数据库实时同步工具。单 JAR 启动、浏览器配置,无需 Kafka/ZooKeeper;支持 MySQL、Oracle、PostgreSQL、达梦、金仓等 34 种数据库,内置结构同步、六类日志 CDC、ETL、告警与企业统一认证,内网离线可用。

    GitHub · Java · 93 stars · 25 forks · +1 stars in 24 hours · Trending score 0.96

  24. 24
    ogbinar/DataEngineeringPilipinas

    Data Engineering Pilipinas is a community for data engineers, data analysts, data scientists, developers, AI / ML engineers, and users of closed and open source data tools and methods / techniques in the Philippines. Data Engineering Pilipinas is a PyData group.

    GitHub · HTML · 239 stars · 64 forks · +2 stars in 24 hours · Trending score 0.87