data-extraction repositories
Trending repositories tagged data-extraction, ranked by trending score. Updated daily.
-
1
firecrawl/firecrawl
Knowledge infrastructure to make your agents smarter. Search, scrape and 550+ data providers through one API.
GitHub · TypeScript · 204,946 stars · 10,743 forks · +1,470 stars in 24 hours · Trending score 5.69
-
2
D4Vinci/Scrapling
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
GitHub · Python · 104,598 stars · 10,941 forks · +468 stars in 24 hours · Trending score 5.28
-
3
unclecode/crawl4ai
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
GitHub · Python · 87,596 stars · 9,143 forks · +87 stars in 24 hours · Trending score 3.84
-
4
getmaxun/maxun
Turn any website into a structured API. Extract, automate, search and monitor the web.
GitHub · TypeScript · 18,532 stars · 1,596 forks · +28 stars in 24 hours · Trending score 3.20
-
5
browserbase/stagehand
The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.
GitHub · TypeScript · 26,101 stars · 1,835 forks · +13 stars in 24 hours · Trending score 2.63
-
6
fastcrw/crw
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
GitHub · Rust · 1,281 stars · 102 forks · +14 stars in 24 hours · Trending score 2.30
-
7
soxoj/socid-extractor
⛏️ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites
GitHub · Python · 1,141 stars · 133 forks · +15 stars in 24 hours · Trending score 2.15
-
8
YOLOW92/pagesift
Local-first web & document ingestion engine in Rust — HTTP/browser capture, layout-aware PDF/Office/OCR parsing, persistent sync, verifiable data graphs, and evidence-grounded AI context.
GitHub · Rust · 284 stars · 37 forks · +5 stars in 24 hours · Trending score 1.71
-
9
browser-act/skills
Browser automation CLI built for AI agents. Break through anti-bot walls, hand off to humans across platforms when stuck. Parallel multi-task execution, independent multi-session operation, isolated multi-account browsing.
GitHub · Python · 6,378 stars · 345 forks · +5 stars in 24 hours · Trending score 1.62
-
10
MontFerret/ferret
Declarative data automation language and Go runtime for structured extraction workflows.
GitHub · Go · 6,026 stars · 327 forks · +6 stars in 24 hours · Trending score 1.51
-
11
JonathanLink/PDFLayoutTextStripper
Converts a pdf file into a text file while keeping the layout of the original pdf. Useful to extract the content from a table in a pdf file for instance. This is a subclass of PDFTextStripper class (from the Apache PDFBox library).
GitHub · Java · 1,613 stars · 213 forks · +3 stars in 24 hours · Trending score 0.99
-
12
2captcha/2captcha-mcp
2Captcha MCP server for AI agents with web search, web scraping, browser automation, captcha solving, and scalable web data collection.
GitHub · JavaScript · 41 stars · 3 forks · 0 stars in 24 hours · Trending score 0.78
-
13
brightdata/brightdata-mcp
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.
GitHub · JavaScript · 2,724 stars · 337 forks · 0 stars in 24 hours · Trending score 0.69
-
14
Ayyouboss0011/SherlockMaps
Powerful Google Maps Crawler / Scraper tool with REST API, Docker support & multi-format export
GitHub · JavaScript · 96 stars · 12 forks · 0 stars in 24 hours · Trending score 0.60
-
15
elflodge/elflodge
Hundreds of agents working unattended. One job API. Run sandboxed agents in parallel on your own machine, without a costly fleet of hosted sandboxes. Send a task or a whole batch; get structured results your code can use.
GitHub · TypeScript · 8 stars · 2 forks · +1 stars in 24 hours · Trending score 0.56
-
16
edwardtay/awesome-scrapers
100+ web scraping tools sorted by job: AI extraction, stealth browsers, MCP servers for agents, proxies and CAPTCHA solving. Dead tools listed with what to use instead.
GitHub · 24 stars · 20 forks · 0 stars in 24 hours · Trending score 0.55
-
17
bolvein-firex/scraping-skills
70 agent skills (SKILL.md) for Claude Code and other AI agents that pull public web data through Apify Actors: Google Maps leads, Amazon prices and reviews, Zillow listings, Indeed jobs, TikTok and Reddit trends. Each skill lists the real input and output fields and a runnable example.
GitHub · JavaScript · 12 stars · 1 forks · 0 stars in 24 hours · Trending score 0.53
-
18
cporter202/coreclaw-api-directory
An unofficial, categorized directory of 118 CoreClaw Worker APIs for web scraping, automation, lead generation, e-commerce, social data, and more.
GitHub · Python · 180 stars · 45 forks · 0 stars in 24 hours · Trending score 0.43
-
19
arpitpatel25/ReadAnyDoc
Open-source Reducto alternative: parse PDFs, scans and forms into Markdown, tables and typed data with a citation for every value. Claude Haiku or any model. Python library, MCP server, Reducto-compatible API.
GitHub · Python · 11 stars · 3 forks · +1 stars in 24 hours · Trending score 0.42
-
20
ilien-dev/svipall
Local-first MCP server and CLI in Rust: any page as LLM-ready Markdown, whole-site crawls, keyless search, and local captcha solving. No cloud, no API keys.
GitHub · Rust · 22 stars · 2 forks · 0 stars in 24 hours · Trending score 0.36
-
21
yagyeshVyas/linkedin-scraper
Browser automation tool using Playwright for web scraping (for specially recruiter)
GitHub · Python · 21 stars · 7 forks · 0 stars in 24 hours · Trending score 0.31
-
22
cernodatainc/cernodata
Open-source control layer over PDF parsing that picks the right method, verifies output quality, and falls back automatically when results fall short
GitHub · Python · 24 stars · 6 forks · 0 stars in 24 hours · Trending score 0.30
-
23
clifton/rstructor
Pydantic + Instructor for Rust: extract structured, validated data from LLMs (OpenAI GPT, Anthropic Claude, Gemini, Grok/xAI) into native Rust structs & enums. Derive macros auto-generate JSON Schema, parse responses, and retry on validation errors. Type-safe structured outputs & function calling.
GitHub · Rust · 34 stars · 4 forks · 0 stars in 24 hours · Trending score 0.28
-
24
apify/apify-sdk-python
Apify SDK for Python—The official library for building Apify Actors: serverless cloud programs for web scraping, browser automation, data processing, and AI agents. Manages the Actor lifecycle, storages (datasets, key-value stores, request queues), events, proxies, and pay-per-event monetization. Built on top of the the Apify API Client.
GitHub · Python · 180 stars · 32 forks · 0 stars in 24 hours · Trending score 0.25