1
High-throughput crawling and scraping for agent data pipelines
$ npx skills add scrapy/scrapy61.8K stars77 qualityClaude Code
High-confidence pick with strong adoption and healthy maintenance signals.
pythoncrawlerrag
by scrapyQuick view
Decision filters
Choose skills by scenario, quality, and trust signals.
9 skills matching "scrapy"
Best blend of quality, stars, freshness, and agent usage
2
Redis-based components for Scrapy.
$ npx skills add rmax/scrapy-redis5.6K stars68 qualityClaude Code
High-confidence pick with strong adoption and healthy maintenance signals.
pythoncrawler
by rmaxQuick view
3
🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量数据去重等功能。更有功能强大的爬虫管理系统feaplat为其提供方便的部署及调度
$ npx skills add Boris-code/feapder3.7K stars67 qualityClaude Code
High-confidence pick with strong adoption and healthy maintenance signals.
pythoncrawler
by Boris-codeQuick view
4
基于搜狗微信搜索的微信公众号爬虫接口
$ npx skills add chyroc/WechatSogou6.3K stars65 qualityClaude Code
High-confidence pick with strong adoption and healthy maintenance signals.
pythoncrawler
by chyrocQuick view
5
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
$ npx skills add crawlab-team/crawlab12.2K stars65 qualityClaude Code
High-confidence pick with strong adoption and healthy maintenance signals.
gocrawler
by crawlab-teamQuick view
6
Write web scrapers in Ruby using a clean, AI-assisted DSL. Kimurai uses AI to figure out where the data lives, then caches the selectors and scrapes with pure Ruby. Get the intelligence of an LLM without the per-request latency or token costs.
$ npx skills add vifreefly/kimuraframework1.1K stars56 qualityClaude Code + Browser agents
High-confidence pick with strong adoption and healthy maintenance signals.
rubyweb-automation
by vifreeflyQuick view
7
:sparkling_heart: High available distributed ip proxy pool, powerd by Scrapy and Redis
$ npx skills add SpiderClub/haipproxy5.5K stars53 qualityClaude Code
Solid option that is likely worth shortlisting for production workflows.Check: Repository looks stale
pythoncrawler
by SpiderClubQuick view
8
Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️
$ npx skills add wkunzhi/Python3-Spider3.4K stars51 qualityClaude Code
Solid option that is likely worth shortlisting for production workflows.Check: Repository looks stale
pythoncrawler
by wkunzhiQuick view
9
Scrapy Fake Useragent
PROMISING · 55Random User-Agent middleware based on fake-useragent
$ npx skills add alecxe/scrapy-fake-useragent688 stars39 qualityClaude Code
Useful candidate, but compare it with alternatives before adopting.Check: Repository looks stale
pythonweb-automation
by alecxeQuick view