webscraping repositories
Trending repositories tagged webscraping, ranked by trending score. Updated daily.
-
1
firecrawl/firecrawl
Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.
GitHub · TypeScript · 206,734 stars · 10,802 forks · +2,005 stars in 24 hours · Trending score 5.95
-
2
D4Vinci/Scrapling
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
GitHub · Python · 104,965 stars · 10,981 forks · +411 stars in 24 hours · Trending score 5.15
-
3
daijro/camoufox
🦊 Anti-detect browser
GitHub · C · 13,563 stars · 1,163 forks · +359 stars in 24 hours · Trending score 4.47
-
4
ProxyShard/ShardBrowser
Free, open-source anti-detect browser launcher for web scraping and multi-accounting. By the ShardX team (formerly ProxyShard). Engine-level fingerprint spoofing in Chromium 158 (WebGL / WebGPU / Client Hints / fonts / TLS), 220+ device profiles bundled, stable QUIC + WebRTC over SOCKS5.
GitHub · TypeScript · 1,754 stars · 252 forks · +39 stars in 24 hours · Trending score 3.52
-
5
CloakHQ/CloakBrowser
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.
GitHub · Python · 33,021 stars · 2,745 forks · +32 stars in 24 hours · Trending score 3.04
-
6
getmaxun/maxun
Turn any website into a structured API. Extract, automate, search and monitor the web.
GitHub · TypeScript · 18,543 stars · 1,599 forks · +14 stars in 24 hours · Trending score 2.92
-
7
ovobrowsersupport/ovobrowser
源码级 Chromium 指纹内核,检测站按普通 Chrome 评分。Source-level Chromium fingerprint kernel that passes bot detection tests. Not JS injection — C++ patches compiled into the binary.
GitHub · 1,288 stars · 72 forks · +22 stars in 24 hours · Trending score 2.58
-
8
huginn/huginn
Create agents that monitor and act on your behalf. Your agents are standing by!
GitHub · Ruby · 50,606 stars · 4,362 forks · +17 stars in 24 hours · Trending score 2.55
-
9
oss-invisible/playwright
invisible playwright: Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
GitHub · Python · 2,615 stars · 278 forks · +9 stars in 24 hours · Trending score 2.02
-
10
itsOwen/CyberScraper-2077
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
GitHub · Python · 3,302 stars · 360 forks · +14 stars in 24 hours · Trending score 1.76
-
11
moistfella/comic-book-downloader
A simple comic book downloader. Created for personal use, shared for anyone else who may need it.
GitHub · Python · 35 stars · 7 forks · +8 stars in 24 hours · Trending score 1.55
-
12
requests-cache/requests-cache
Persistent HTTP cache for python requests
GitHub · Python · 1,511 stars · 175 forks · +5 stars in 24 hours · Trending score 1.49
-
13
anaskhan96/soup
Web Scraper in Go, similar to BeautifulSoup
GitHub · Go · 2,303 stars · 180 forks · +5 stars in 24 hours · Trending score 1.36
-
14
tiliondev/fortress
Stealth Chromium engine that stops scrapers and browser agents from getting blocked, with one line of code change.
GitHub · Python · 810 stars · 55 forks · 0 stars in 24 hours · Trending score 1.20
-
15
adbar/htmldate
Fast and robust date extraction from web pages, with Python or on the command-line
GitHub · Python · 162 stars · 44 forks · +3 stars in 24 hours · Trending score 1.13
-
16
BrenoFariasdaSilva/E-Commerces-WebScraper
E-Commerces WebScraping Python Project. Main goal is to implement Web Scraping on Aliexpress, Amazon, Mercado Livre, Shein and Shopee Platforms.
GitHub · Python · 43 stars · 8 forks · +3 stars in 24 hours · Trending score 1.12
-
17
henriqqw/AnimeCaos
AnimeCaos é um app desktop open source para streaming de animes. Assista seus animes favoritos sem anúncios em um player nativo, faça downloads rápidos para assistir offline e busque conteúdo em múltiplas fontes em uma única aplicação minimalista.
GitHub · Python · 33 stars · 4 forks · +4 stars in 24 hours · Trending score 1.06
-
18
StoneSteel27/AutomatiQ
Simplify Web Reverse-engineering for AI Agents
GitHub · Python · 180 stars · 15 forks · +4 stars in 24 hours · Trending score 1.05
-
19
scrapfly/scrapfly-scrapers
Scalable Python web scraping scripts for +80 popular domains
GitHub · Python · 1,133 stars · 207 forks · +2 stars in 24 hours · Trending score 0.91
-
20
pakistan-hijri/hijri-api
Open Hijri calendar and prayer times data for Pakistan: a verified, auditable archive of sighted month starts with confidence levels, served as static JSON.
GitHub · TypeScript · 15 stars · 2 forks · +2 stars in 24 hours · Trending score 0.82
-
21
Hyper-Solutions/hyper-sdk-go
Go SDK for Bot Protection Bypass - Automate Akamai, Incapsula, Kasada, and DataDome. No browsers required. Solve challenges and generate valid sensors/cookies via API.
GitHub · Go · 71 stars · 4 forks · +2 stars in 24 hours · Trending score 0.80
-
22
wkoverfield/nba2kapi
NBA 2k API
GitHub · TypeScript · 22 stars · 5 forks · 0 stars in 24 hours · Trending score 0.78
-
23
felipeall/transfermarkt-api
API service to get data from Transfermarkt
GitHub · Python · 473 stars · 162 forks · 0 stars in 24 hours · Trending score 0.77
-
24
Vinyzu/cursory
Generate 100% human-realistic Mouse Trajectories with Timings.
GitHub · Python · 195 stars · 21 forks · 0 stars in 24 hours · Trending score 0.74