scrapy repositories

Trending repositories tagged scrapy, ranked by trending score. Updated daily.

  1. 1
    tb0hdan/domains

    World’s single largest Internet domains dataset

    GitHub · Shell · 1,181 stars · 186 forks · +4 stars in 24 hours · Trending score 1.07

  2. 2
    scrapy/itemadapter

    Common interface for data container classes

    GitHub · Python · 71 stars · 28 forks · +1 stars in 24 hours · Trending score 0.76

  3. 3
    netweir/netweir

    An ultra-fast, self-healing, undetectable web scraper. Rust does the work; you write Python.

    GitHub · Rust · 9 stars · 4 forks · +1 stars in 24 hours · Trending score 0.64

  4. 4
    bilalahhmedd/ebay-products-scraper

    A production grade data scraper developed to acquire listing data (product details, images and item-specifications) from ebay.com

    GitHub · Python · 32 stars · 11 forks · +1 stars in 24 hours · Trending score 0.55

  5. 5
    scrapy-plugins/scrapy-zyte-api

    Zyte API integration for Scrapy

    GitHub · Python · 44 stars · 25 forks · +1 stars in 24 hours · Trending score 0.55

  6. 6
    bitmakerla/estela

    estela, an elastic web scraping cluster 🕸

    GitHub · TypeScript · 204 stars · 17 forks · +1 stars in 24 hours · Trending score 0.53

  7. 7
    borys25ol/amazon-location-cookies-service

    Self-hosted API that extracts Amazon location and delivery-country session cookies, solving the AWS WAF challenge with a headless browser

    GitHub · Python · 48 stars · 12 forks · +1 stars in 24 hours · Trending score 0.45

  8. 8
    discourselab/scrapai-cli

    AI-powered web scraping CLI. Describe what you want, get a production-ready Scrapy spider. Write once, reuse forever.

    GitHub · Python · 117 stars · 18 forks · +1 stars in 24 hours · Trending score 0.45

  9. 9
    shengchenyang/AyugeSpiderTools

    使 scrapy 开发不用在意 item,pipeline,middleware 等通用场景下模块的编写,解放开发者的双手。

    GitHub · Python · 103 stars · 17 forks · +1 stars in 24 hours · Trending score 0.41

  10. 10
    juancarlospaco/faster-than-requests

    Faster requests on Python 3

    GitHub · Nim · 1,132 stars · 89 forks · 0 stars in 24 hours · Trending score 0.40

  11. 11
    alltheplaces/alltheplaces

    A set of spiders and scrapers to extract location information from places that post their location on the internet.

    GitHub · Python · 851 stars · 309 forks · 0 stars in 24 hours · Trending score 0.37

  12. 12
    rmax/scrapy-redis

    Redis-based components for Scrapy.

    GitHub · Python · 5,648 stars · 1,579 forks · −1 stars in 24 hours · Trending score 0.37

  13. 13
    scrapy/scrapy-lint

    A linter for Scrapy projects.

    GitHub · Python · 22 stars · 11 forks · 0 stars in 24 hours · Trending score 0.23

  14. 14
    magic890/tripadvisor-scraper

    TripAdvisor scraper

    GitHub · Python · 80 stars · 36 forks · 0 stars in 24 hours · Trending score 0.14

  15. 15
    Suvastutech-Ltd/scrapy-stealth

    Stealthy Crawling. Maximum Results. A pluggable anti-bot and stealth framework for Scrapy.

    GitHub · Python · 17 stars · 9 forks · 0 stars in 24 hours · Trending score 0.14

  16. 16
    tech-engine/goscrapy

    GoScrapy: High perfomance webscraping framework in Go, inspired by Python's Scrapy.

    GitHub · Go · 371 stars · 42 forks · −2 stars in 24 hours · Trending score 0.10

  17. 17
    scrapfly/python-scrapfly

    Official Python SDK for the Scrapfly platform: web scraping, screenshots, AI extraction, crawling, and a remote anti-bot browser. Integrates with Scrapy, LlamaIndex, and LangChain.

    GitHub · Python · 62 stars · 18 forks · −1 stars in 24 hours · Trending score 0.09

  18. 18
    scrapy-plugins/scrapy-splash

    Scrapy+Splash for JavaScript integration

    GitHub · Python · 3,223 stars · 455 forks · 0 stars in 24 hours · Trending score 0.00

  19. 19
    imflyn/decoration-design-crawler

    土巴兔和谷居装修网站爬虫

    GitHub · Python · 108 stars · 37 forks · 0 stars in 24 hours · Trending score 0.00

  20. 20
    RayXBuilds/Movie_Recommend

    历史学习项目:Spark 电影推荐、爬虫、Web 与后台。新版持续开发见 cinematch-recommender。

    GitHub · Java · 3,007 stars · 1,048 forks · −6 stars in 24 hours · Trending score -0.00

  21. 21
    holgerd77/django-dynamic-scraper

    Creating Scrapy scrapers via the Django admin interface

    GitHub · Python · 1,151 stars · 298 forks · −7 stars in 24 hours · Trending score -0.00

  22. 22
    wildovortex96622/Youtube-resocket-viewbot

    🦊 Fast views botter based on proxies

    GitHub · 27 stars · 0 forks · −23 stars in 24 hours · Trending score -0.03