scraper repositories

Trending repositories tagged scraper, ranked by trending score. Updated daily.

  1. 1
    firecrawl/firecrawl

    Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

    GitHub · TypeScript · 205,406 stars · 10,742 forks · +2,082 stars in 24 hours · Trending score 5.92

  2. 2
    unclecode/crawl4ai

    Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

    GitHub · Python · 87,614 stars · 9,134 forks · +70 stars in 24 hours · Trending score 3.71

  3. 3
    explorium-ai/gtm-skills

    GTM Skills for Claude & Codex

    GitHub · TypeScript · 365 stars · 7 forks · +173 stars in 24 hours · Trending score 3.70

  4. 4
    getmaxun/maxun

    Turn any website into a structured API. Extract, automate, search and monitor the web.

    GitHub · TypeScript · 18,533 stars · 1,597 forks · +17 stars in 24 hours · Trending score 3.00

  5. 5
    apify/crawlee

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

    GitHub · TypeScript · 26,505 stars · 1,774 forks · +19 stars in 24 hours · Trending score 2.91

  6. 6
    guohuiyuan/go-novel-dl

    go-novel-dl 是 Go 语言开发的多源小说下载器,支持 CLI 与 Web 界面,可跨站点并发搜索、一键下载并导出,已适配 ESJZone、SFACG、刺猬猫、若初文学等多个书源。novel-dl交流群:1094455212

    GitHub · Go · 337 stars · 32 forks · +34 stars in 24 hours · Trending score 2.80

  7. 7
    huginn/huginn

    Create agents that monitor and act on your behalf. Your agents are standing by!

    GitHub · Ruby · 50,601 stars · 4,363 forks · +22 stars in 24 hours · Trending score 2.67

  8. 8
    openzim/mwoffliner

    MediaWiki scraper: all your wiki articles in one highly compressed ZIM file

    GitHub · TypeScript · 497 stars · 134 forks · +8 stars in 24 hours · Trending score 1.79

  9. 9
    yiromo/pytrends-modern

    The Modern Google Trends API - Combining the best features from pytrends, trendspyg, and more.

    GitHub · Python · 78 stars · 7 forks · +12 stars in 24 hours · Trending score 1.78

  10. 10
    itsOwen/CyberScraper-2077

    A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

    GitHub · Python · 3,302 stars · 360 forks · +14 stars in 24 hours · Trending score 1.76

  11. 11
    edoardottt/cariddi

    Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

    GitHub · Go · 3,816 stars · 345 forks · +8 stars in 24 hours · Trending score 1.61

  12. 12
    apify/crawlee-python

    Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.

    GitHub · Python · 9,680 stars · 837 forks · +3 stars in 24 hours · Trending score 1.58

  13. 13
    justoneapi-labs/taobao-api

    淘宝API、淘宝接口、淘宝爬虫示例。

    GitHub · Python · 26 stars · 8 forks · +3 stars in 24 hours · Trending score 1.56

  14. 14
    TadMSTR/searxng-mcp

    Private web search MCP: SearXNG metasearch → ML reranking → 4-tier fetch cascade (Firecrawl / Crawl4AI / raw / Wayback) with per-domain learning, Ollama query expansion and synthesis. No cloud APIs.

    GitHub · TypeScript · 30 stars · 2 forks · +6 stars in 24 hours · Trending score 1.38

  15. 15
    maximilianfeix/proxy-scraper

    Free HTTP/SOCKS proxy list, rebuilt every hour, with the result of every check on record. CLI, Python API, rotating proxy server, MCP server.

    GitHub · Python · 98 stars · 24 forks · +3 stars in 24 hours · Trending score 1.38

  16. 16
    oxylabs/how-to-scrape-google-jobs

    Learn how to build your own Google Jobs scraper that simultaneously scrapes Google Jobs for multiple search queries and geo-locations with Python and Oxylabs’ Google Jobs Scraper API. https://oxylabs.io/blog/how-to-scrape-google-jobs

    GitHub · Python · 1,770 stars · 9 forks · 0 stars in 24 hours · Trending score 1.36

  17. 17
    coflyn/Mori

    Client-side media downloader App and CLI built with Capacitor and Node.js.

    GitHub · JavaScript · 472 stars · 67 forks · 0 stars in 24 hours · Trending score 1.33

  18. 18
    NaiboWang/EasySpider

    A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/网页爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。

    GitHub · JavaScript · 44,802 stars · 5,391 forks · +3 stars in 24 hours · Trending score 1.23

  19. 19
    yaroslav2901/OE_OUTAGE_DATA

    Репозиторій містить JSON-дані про відключення електроенергії та згенеровані зображення розкладів для кожної групи та загальне зображення для всіх груп.

    GitHub · 42 stars · 2 forks · +4 stars in 24 hours · Trending score 1.22

  20. 20
    jikan-me/jikan

    Unofficial MyAnimeList PHP+REST API which provides functions other than the official API

    GitHub · PHP · 1,095 stars · 126 forks · +4 stars in 24 hours · Trending score 1.14

  21. 21
    Dhairya3391/kari

    Kari just a tui for getting media from providers and playing it in your media player with nice to have features.

    GitHub · Go · 74 stars · 3 forks · +3 stars in 24 hours · Trending score 1.14

  22. 22
    plhery/universal-parcel-scraper

    Self-hosted parcel tracking: 105 carriers, a TypeScript library, CLI, and HTTP API.

    GitHub · TypeScript · 15 stars · 7 forks · +1 stars in 24 hours · Trending score 1.00

  23. 23
    henrique-coder/perplexity-webui-scraper

    Unofficial Python client, CLI, MCP server, and OpenAI-compatible REST API for Perplexity's WebUI.

    GitHub · Python · 143 stars · 30 forks · +3 stars in 24 hours · Trending score 1.00

  24. 24
    Solr159/JavBoss

    开箱即用的本地 JAV/视频 刮削、管理、播放软件,支持命令行一键安装和 Docker 部署。只需简单添加目录,即可打造你的私人 JAV/视频 媒体库,带给你顶级的浏览体验,懒人必备。| Your local JAV/video manager.

    GitHub · Go · 932 stars · 94 forks · 0 stars in 24 hours · Trending score 0.98