scrapy repositories
Trending repositories tagged scrapy, ranked by trending score. Updated daily.
-
1
tb0hdan/domains
World’s single largest Internet domains dataset
GitHub · Shell · 1,181 stars · 186 forks · +4 stars in 24 hours · Trending score 1.07
-
2
scrapy/itemadapter
Common interface for data container classes
GitHub · Python · 71 stars · 28 forks · +1 stars in 24 hours · Trending score 0.76
-
3
netweir/netweir
An ultra-fast, self-healing, undetectable web scraper. Rust does the work; you write Python.
GitHub · Rust · 9 stars · 4 forks · +1 stars in 24 hours · Trending score 0.64
-
4
bilalahhmedd/ebay-products-scraper
A production grade data scraper developed to acquire listing data (product details, images and item-specifications) from ebay.com
GitHub · Python · 32 stars · 11 forks · +1 stars in 24 hours · Trending score 0.55
-
5
scrapy-plugins/scrapy-zyte-api
Zyte API integration for Scrapy
GitHub · Python · 44 stars · 25 forks · +1 stars in 24 hours · Trending score 0.55
-
6
bitmakerla/estela
estela, an elastic web scraping cluster 🕸
GitHub · TypeScript · 204 stars · 17 forks · +1 stars in 24 hours · Trending score 0.53
-
7
borys25ol/amazon-location-cookies-service
Self-hosted API that extracts Amazon location and delivery-country session cookies, solving the AWS WAF challenge with a headless browser
GitHub · Python · 48 stars · 12 forks · +1 stars in 24 hours · Trending score 0.45
-
8
discourselab/scrapai-cli
AI-powered web scraping CLI. Describe what you want, get a production-ready Scrapy spider. Write once, reuse forever.
GitHub · Python · 117 stars · 18 forks · +1 stars in 24 hours · Trending score 0.45
-
9
shengchenyang/AyugeSpiderTools
使 scrapy 开发不用在意 item,pipeline,middleware 等通用场景下模块的编写,解放开发者的双手。
GitHub · Python · 103 stars · 17 forks · +1 stars in 24 hours · Trending score 0.41
-
10
juancarlospaco/faster-than-requests
Faster requests on Python 3
GitHub · Nim · 1,132 stars · 89 forks · 0 stars in 24 hours · Trending score 0.40
-
11
alltheplaces/alltheplaces
A set of spiders and scrapers to extract location information from places that post their location on the internet.
GitHub · Python · 851 stars · 309 forks · 0 stars in 24 hours · Trending score 0.37
-
12
rmax/scrapy-redis
Redis-based components for Scrapy.
GitHub · Python · 5,648 stars · 1,579 forks · −1 stars in 24 hours · Trending score 0.37
-
13
scrapy/scrapy-lint
A linter for Scrapy projects.
GitHub · Python · 22 stars · 11 forks · 0 stars in 24 hours · Trending score 0.23
-
14
magic890/tripadvisor-scraper
TripAdvisor scraper
GitHub · Python · 80 stars · 36 forks · 0 stars in 24 hours · Trending score 0.14
-
15
Suvastutech-Ltd/scrapy-stealth
Stealthy Crawling. Maximum Results. A pluggable anti-bot and stealth framework for Scrapy.
GitHub · Python · 17 stars · 9 forks · 0 stars in 24 hours · Trending score 0.14
-
16
tech-engine/goscrapy
GoScrapy: High perfomance webscraping framework in Go, inspired by Python's Scrapy.
GitHub · Go · 371 stars · 42 forks · −2 stars in 24 hours · Trending score 0.10
-
17
scrapfly/python-scrapfly
Official Python SDK for the Scrapfly platform: web scraping, screenshots, AI extraction, crawling, and a remote anti-bot browser. Integrates with Scrapy, LlamaIndex, and LangChain.
GitHub · Python · 62 stars · 18 forks · −1 stars in 24 hours · Trending score 0.09
-
18
scrapy-plugins/scrapy-splash
Scrapy+Splash for JavaScript integration
GitHub · Python · 3,223 stars · 455 forks · 0 stars in 24 hours · Trending score 0.00
-
19
imflyn/decoration-design-crawler
土巴兔和谷居装修网站爬虫
GitHub · Python · 108 stars · 37 forks · 0 stars in 24 hours · Trending score 0.00
-
20
RayXBuilds/Movie_Recommend
历史学习项目:Spark 电影推荐、爬虫、Web 与后台。新版持续开发见 cinematch-recommender。
GitHub · Java · 3,007 stars · 1,048 forks · −6 stars in 24 hours · Trending score -0.00
-
21
holgerd77/django-dynamic-scraper
Creating Scrapy scrapers via the Django admin interface
GitHub · Python · 1,151 stars · 298 forks · −7 stars in 24 hours · Trending score -0.00
-
22
wildovortex96622/Youtube-resocket-viewbot
🦊 Fast views botter based on proxies
GitHub · 27 stars · 0 forks · −23 stars in 24 hours · Trending score -0.03