etl repositories
Trending repositories tagged etl, ranked by trending score. Updated daily.
-
1
datazip-inc/olake-go
OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.
GitHub · Go · 2,154 stars · 320 forks · +163 stars in 24 hours · Trending score 4.18
-
2
apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
GitHub · Python · 47,584 stars · 18,364 forks · +44 stars in 24 hours · Trending score 3.32
-
3
vectordotdev/vector
A high-performance observability data pipeline.
GitHub · Rust · 22,926 stars · 2,405 forks · +22 stars in 24 hours · Trending score 2.73
-
4
kestra-io/kestra
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
GitHub · Java · 30,303 stars · 3,546 forks · +18 stars in 24 hours · Trending score 2.71
-
5
dagster-io/dagster
An orchestration platform for the development, production, and observation of data assets.
GitHub · Python · 16,444 stars · 2,427 forks · +9 stars in 24 hours · Trending score 2.36
-
6
SQLMesh/sqlmesh
Scalable and efficient data transformation framework - backwards compatible with dbt.
GitHub · Python · 3,360 stars · 493 forks · +15 stars in 24 hours · Trending score 2.21
-
7
galaxy-io/filament
Pluggable data replication with checkpointing, batching, and integrity events
GitHub · Go · 336 stars · 48 forks · +8 stars in 24 hours · Trending score 2.16
-
8
cocoindex-io/cocoindex
Incremental engine for long horizon agents 🌟 Star if you like it!
GitHub · Rust · 11,923 stars · 930 forks · +13 stars in 24 hours · Trending score 1.99
-
9
Open-Source-Legal/OpenContracts
The open document intelligence platform for builders and hackers - DMS for the agentic world
GitHub · Python · 1,550 stars · 201 forks · +8 stars in 24 hours · Trending score 1.68
-
10
morph-kgc/morph-kgc
Powerful RDF knowledge graph generation with RML mappings
GitHub · Python · 303 stars · 63 forks · +7 stars in 24 hours · Trending score 1.60
-
11
turbot/steampipe
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
GitHub · Go · 8,020 stars · 348 forks · +5 stars in 24 hours · Trending score 1.54
-
12
airbytehq/airbyte
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
GitHub · Python · 22,411 stars · 5,550 forks · 0 stars in 24 hours · Trending score 1.47
-
13
yuyuejia/datay
DataY 是一套轻量、高性能的数据平台,一个 JAR、自研流批一体处理引擎,自研分布式调度框架,在 2 核 4G 的机器上 就能跑通 数据集成 → 数据建模 → 数据开发 → 任务调度 → 数据应用 的完整链路。
GitHub · Java · 50 stars · 23 forks · +3 stars in 24 hours · Trending score 1.41
-
14
apache/hop
Hop Orchestration Platform
GitHub · Java · 1,544 stars · 493 forks · +2 stars in 24 hours · Trending score 1.28
-
15
Unknown-333/awesome-data-engineering-skills
36 data-engineering skills for AI coding agents — dbt, Airflow, Dagster, Prefect, Spark, Snowflake, BigQuery, Databricks, Kafka, Iceberg. Portable SKILL.md Agent Skills (the format Anthropic launched as Claude Skills), with idempotency, backfill, and data-quality know-how. Runs in Claude Code, Cursor, Codex, Copilot, Gemini CLI and more.
GitHub · Python · 27 stars · 1 forks · +4 stars in 24 hours · Trending score 1.16
-
16
ariacom/Seal-Report
Database Reporting Tool and Tasks (.Net)
GitHub · C# · 1,646 stars · 511 forks · +3 stars in 24 hours · Trending score 1.15
-
17
planarnetwork/gb-transit
DTD feed importing, timetable conversion and GTFS tooling
GitHub · TypeScript · 36 stars · 16 forks · +3 stars in 24 hours · Trending score 1.10
-
18
ucbepic/docetl
A system for agentic LLM-powered data processing and ETL
GitHub · Python · 4,259 stars · 471 forks · 0 stars in 24 hours · Trending score 1.08
-
19
ara3d/bim-open-schema
Representing BIM Data as Parquet
GitHub · C# · 57 stars · 10 forks · +4 stars in 24 hours · Trending score 1.05
-
20
nao1215/filesql
loads CSV, TSV, LTSV, JSON, JSONL, Parquet, XLSX, ACH, and Fedwire files into SQLite; includes prep and frame for cleanup and in-memory transforms
GitHub · Go · 388 stars · 12 forks · +2 stars in 24 hours · Trending score 1.04
-
21
reductstore/reductstore
High-performance, time-indexed object storage for robotics and industrial IoT
GitHub · Rust · 381 stars · 52 forks · +2 stars in 24 hours · Trending score 1.01
-
22
apache/devlake
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
GitHub · Go · 3,204 stars · 842 forks · +1 stars in 24 hours · Trending score 0.97
-
23
vfaner/jync
Jync 捷同 - 开箱即用的异构数据库实时同步工具。单 JAR 启动、浏览器配置,无需 Kafka/ZooKeeper;支持 MySQL、Oracle、PostgreSQL、达梦、金仓等 34 种数据库,内置结构同步、六类日志 CDC、ETL、告警与企业统一认证,内网离线可用。
GitHub · Java · 93 stars · 25 forks · +1 stars in 24 hours · Trending score 0.96
-
24
ogbinar/DataEngineeringPilipinas
Data Engineering Pilipinas is a community for data engineers, data analysts, data scientists, developers, AI / ML engineers, and users of closed and open source data tools and methods / techniques in the Philippines. Data Engineering Pilipinas is a PyData group.
GitHub · HTML · 239 stars · 64 forks · +2 stars in 24 hours · Trending score 0.87