web-scraper repositories

Trending repositories tagged web-scraper, ranked by trending score. Updated daily.

  1. 1
    firecrawl/firecrawl

    Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

    GitHub · TypeScript · 206,734 stars · 10,802 forks · +2,005 stars in 24 hours · Trending score 5.95

  2. 2
    D4Vinci/Scrapling

    🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

    GitHub · Python · 104,965 stars · 10,981 forks · +411 stars in 24 hours · Trending score 5.15

  3. 3
    Panniantong/Agent-Reach

    Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.

    GitHub · Python · 113,937 stars · 10,232 forks · +447 stars in 24 hours · Trending score 5.07

  4. 4
    gosom/google-maps-scraper

    scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place

    GitHub · Go · 8,306 stars · 1,322 forks · +196 stars in 24 hours · Trending score 4.53

  5. 5
    lexmount/moli

    Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust

    GitHub · Rust · 24,239 stars · 696 forks · +116 stars in 24 hours · Trending score 3.97

  6. 6
    AhmadHassan-BTed/JinaClip

    Professional-grade, local-first browser extension that transforms any webpage into high-fidelity, LLM-friendly Markdown entirely within the browser's runtime. Zero-cloud, privacy-centric, and designed specifically for AI workflows.

    GitHub · JavaScript · 4,160 stars · 0 forks · +295 stars in 24 hours · Trending score 3.71

  7. 7
    dondai44423/donsetch

    Web fetch, search, and crawl for AI agents. Built from scratch in Rust. No keys, no accounts. AGPL v3.

    GitHub · Rust · 1,148 stars · 78 forks · +134 stars in 24 hours · Trending score 3.64

  8. 8
    getmaxun/maxun

    Turn any website into a structured API. Extract, automate, search and monitor the web.

    GitHub · TypeScript · 18,543 stars · 1,599 forks · +14 stars in 24 hours · Trending score 2.92

  9. 9
    MrAdex77/google-play-scraper

    Google Play scraper for Node.js with a fully typed TypeScript API. App details, search, top charts, reviews, permissions and data safety. A modern replacement for the unmaintained google-play-scraper.

    GitHub · TypeScript · 107 stars · 10 forks · +11 stars in 24 hours · Trending score 1.80

  10. 10
    itsOwen/CyberScraper-2077

    A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

    GitHub · Python · 3,302 stars · 360 forks · +14 stars in 24 hours · Trending score 1.76

  11. 11
    fastcrw/crw

    Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

    GitHub · Rust · 1,288 stars · 102 forks · +5 stars in 24 hours · Trending score 1.70

  12. 12
    caiopizzol/fipe-data-pipeline

    Coleta e processa dados históricos de preços da Tabela FIPE para PostgreSQL.

    GitHub · TypeScript · 59 stars · 8 forks · +10 stars in 24 hours · Trending score 1.67

  13. 13
    jolie-z/jobhunter-ai

    🚀 AI 全链路求职自动化平台:抓取→清洗→AI评估→简历改写→自动投递,四大招聘平台 + 飞书 ChatOps 指挥 | Industrial-grade AI job-hunting automation: scrape → clean → LLM eval → resume rewrite → auto-apply, with Feishu ChatOps

    GitHub · Python · 29 stars · 9 forks · +7 stars in 24 hours · Trending score 1.57

  14. 14
    anaskhan96/soup

    Web Scraper in Go, similar to BeautifulSoup

    GitHub · Go · 2,303 stars · 180 forks · +5 stars in 24 hours · Trending score 1.36

  15. 15
    SenZmaKi/Senpwai

    A less annoying way to download anime

    GitHub · Dart · 407 stars · 54 forks · +1 stars in 24 hours · Trending score 1.26

  16. 16
    monbooru/monloader

    Online media downloader for monbooru.

    GitHub · Go · 21 stars · 8 forks · +1 stars in 24 hours · Trending score 1.24

  17. 17
    spider-rs/spider

    Foundational low latency web data collecting in Rust

    GitHub · Rust · 2,857 stars · 332 forks · 0 stars in 24 hours · Trending score 1.05

  18. 18
    jaebradley/basketball_reference_web_scraper

    NBA Stats API via Basketball Reference

    GitHub · HTML · 563 stars · 135 forks · +2 stars in 24 hours · Trending score 1.00

  19. 19
    milahu/opensubtitles-scraper

    scrape subtitles from opensubtitles.org

    GitHub · Python · 57 stars · 6 forks · +3 stars in 24 hours · Trending score 0.91

  20. 20
    MEOMcGill/pytok

    A web scraper for TikTok

    GitHub · Python · 178 stars · 24 forks · +2 stars in 24 hours · Trending score 0.81

  21. 21
    0x676e67/wreq

    An ergonomic, privacy-aware Rust HTTP Client

    GitHub · Rust · 1,121 stars · 149 forks · 0 stars in 24 hours · Trending score 0.77

  22. 22
    juancarlospaco/faster-than-requests

    Faster requests on Python 3

    GitHub · Nim · 1,132 stars · 89 forks · +1 stars in 24 hours · Trending score 0.76

  23. 23
    gmastergreatee/Fanfiction-Manager

    Provides extreme flexibility with the help of Rule system to power-users* in downloading, tracking-status & reading novels from ANY site they want.

    GitHub · JavaScript · 55 stars · 3 forks · +2 stars in 24 hours · Trending score 0.72

  24. 24
    gildas-lormeau/single-file-cli

    CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)

    GitHub · JavaScript · 1,697 stars · 147 forks · 0 stars in 24 hours · Trending score 0.52