webscraping repositories

Trending repositories tagged webscraping, ranked by trending score. Updated daily.

  1. 1
    firecrawl/firecrawl

    Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

    GitHub · TypeScript · 206,734 stars · 10,802 forks · +2,005 stars in 24 hours · Trending score 5.95

  2. 2
    D4Vinci/Scrapling

    🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

    GitHub · Python · 104,965 stars · 10,981 forks · +411 stars in 24 hours · Trending score 5.15

  3. 3
    daijro/camoufox

    🦊 Anti-detect browser

    GitHub · C · 13,563 stars · 1,163 forks · +359 stars in 24 hours · Trending score 4.47

  4. 4
    ProxyShard/ShardBrowser

    Free, open-source anti-detect browser launcher for web scraping and multi-accounting. By the ShardX team (formerly ProxyShard). Engine-level fingerprint spoofing in Chromium 158 (WebGL / WebGPU / Client Hints / fonts / TLS), 220+ device profiles bundled, stable QUIC + WebRTC over SOCKS5.

    GitHub · TypeScript · 1,754 stars · 252 forks · +39 stars in 24 hours · Trending score 3.52

  5. 5
    CloakHQ/CloakBrowser

    Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.

    GitHub · Python · 33,021 stars · 2,745 forks · +32 stars in 24 hours · Trending score 3.04

  6. 6
    getmaxun/maxun

    Turn any website into a structured API. Extract, automate, search and monitor the web.

    GitHub · TypeScript · 18,543 stars · 1,599 forks · +14 stars in 24 hours · Trending score 2.92

  7. 7
    ovobrowsersupport/ovobrowser

    源码级 Chromium 指纹内核,检测站按普通 Chrome 评分。Source-level Chromium fingerprint kernel that passes bot detection tests. Not JS injection — C++ patches compiled into the binary.

    GitHub · 1,288 stars · 72 forks · +22 stars in 24 hours · Trending score 2.58

  8. 8
    huginn/huginn

    Create agents that monitor and act on your behalf. Your agents are standing by!

    GitHub · Ruby · 50,606 stars · 4,362 forks · +17 stars in 24 hours · Trending score 2.55

  9. 9
    oss-invisible/playwright

    invisible playwright: Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.

    GitHub · Python · 2,615 stars · 278 forks · +9 stars in 24 hours · Trending score 2.02

  10. 10
    itsOwen/CyberScraper-2077

    A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

    GitHub · Python · 3,302 stars · 360 forks · +14 stars in 24 hours · Trending score 1.76

  11. 11
    moistfella/comic-book-downloader

    A simple comic book downloader. Created for personal use, shared for anyone else who may need it.

    GitHub · Python · 35 stars · 7 forks · +8 stars in 24 hours · Trending score 1.55

  12. 12
    requests-cache/requests-cache

    Persistent HTTP cache for python requests

    GitHub · Python · 1,511 stars · 175 forks · +5 stars in 24 hours · Trending score 1.49

  13. 13
    anaskhan96/soup

    Web Scraper in Go, similar to BeautifulSoup

    GitHub · Go · 2,303 stars · 180 forks · +5 stars in 24 hours · Trending score 1.36

  14. 14
    tiliondev/fortress

    Stealth Chromium engine that stops scrapers and browser agents from getting blocked, with one line of code change.

    GitHub · Python · 810 stars · 55 forks · 0 stars in 24 hours · Trending score 1.20

  15. 15
    adbar/htmldate

    Fast and robust date extraction from web pages, with Python or on the command-line

    GitHub · Python · 162 stars · 44 forks · +3 stars in 24 hours · Trending score 1.13

  16. 16
    BrenoFariasdaSilva/E-Commerces-WebScraper

    E-Commerces WebScraping Python Project. Main goal is to implement Web Scraping on Aliexpress, Amazon, Mercado Livre, Shein and Shopee Platforms.

    GitHub · Python · 43 stars · 8 forks · +3 stars in 24 hours · Trending score 1.12

  17. 17
    henriqqw/AnimeCaos

    AnimeCaos é um app desktop open source para streaming de animes. Assista seus animes favoritos sem anúncios em um player nativo, faça downloads rápidos para assistir offline e busque conteúdo em múltiplas fontes em uma única aplicação minimalista.

    GitHub · Python · 33 stars · 4 forks · +4 stars in 24 hours · Trending score 1.06

  18. 18
    StoneSteel27/AutomatiQ

    Simplify Web Reverse-engineering for AI Agents

    GitHub · Python · 180 stars · 15 forks · +4 stars in 24 hours · Trending score 1.05

  19. 19
    scrapfly/scrapfly-scrapers

    Scalable Python web scraping scripts for +80 popular domains

    GitHub · Python · 1,133 stars · 207 forks · +2 stars in 24 hours · Trending score 0.91

  20. 20
    pakistan-hijri/hijri-api

    Open Hijri calendar and prayer times data for Pakistan: a verified, auditable archive of sighted month starts with confidence levels, served as static JSON.

    GitHub · TypeScript · 15 stars · 2 forks · +2 stars in 24 hours · Trending score 0.82

  21. 21
    Hyper-Solutions/hyper-sdk-go

    Go SDK for Bot Protection Bypass - Automate Akamai, Incapsula, Kasada, and DataDome. No browsers required. Solve challenges and generate valid sensors/cookies via API.

    GitHub · Go · 71 stars · 4 forks · +2 stars in 24 hours · Trending score 0.80

  22. 22
    wkoverfield/nba2kapi

    NBA 2k API

    GitHub · TypeScript · 22 stars · 5 forks · 0 stars in 24 hours · Trending score 0.78

  23. 23
    felipeall/transfermarkt-api

    API service to get data from Transfermarkt

    GitHub · Python · 473 stars · 162 forks · 0 stars in 24 hours · Trending score 0.77

  24. 24
    Vinyzu/cursory

    Generate 100% human-realistic Mouse Trajectories with Timings.

    GitHub · Python · 195 stars · 21 forks · 0 stars in 24 hours · Trending score 0.74