#1
Redis-based components for Scrapy.
5.6K starsMay 19, 2026 pushweb-automationPythonCrawler
$ npx skills add rmax/scrapy-redisAlternatives
DotnetSpider alternatives for AI agents.
Compare similar skills by workflow fit, trust score, quality, GitHub adoption, maintenance, and install readiness.
Current skill
DotnetSpider
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
100
Quality
94
Trust
4.1K
Stars
#2
Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.
2.3K starsSep 9, 2024 pushweb-automationC#Crawler
$ npx skills add sjdirect/abot#3
Elegant Scraper and Crawler Framework for Golang
25K starsJun 18, 2026 pushweb-automationGoCrawler
$ npx skills add gocolly/colly#4
小红书数据采集、网站图片、视频资源批量下载工具，颜值超高的数据采集工具（批量下载，视频提取，图片）Telegram:https://t.me/+ZtLSwuIKTo44MDY1
1.4K starsDec 16, 2025 pushweb-automationCrawlerClaude Code
$ npx skills add xisuo67/XHS-Spider#5
Python ProxyPool for web spider
23K starsJun 9, 2026 pushweb-automationPythonCrawler
$ npx skills add jhao104/proxy_pool#6
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
15K starsMay 13, 2026 pushweb-automationPythonCrawler
$ npx skills add codelucas/newspaper#7
A next-generation crawling and spidering framework.
17K starsJun 16, 2026 pushweb-automationGoCrawler
$ npx skills add projectdiscovery/katana#8
A scalable, mature and versatile web crawler based on Apache Storm
980 starsJun 14, 2026 pushweb-automationJavaCrawler
$ npx skills add apache/stormcrawler#9
A scalable web crawler framework for Java.
12K starsDec 20, 2025 pushweb-automationJavaCrawler
$ npx skills add code4craft/webmagic#10
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
6.8K starsJun 18, 2026 pushweb-automationTypeScriptCrawler
$ npx skills add bda-research/node-crawler#11
A lightweight web crawler framework.（Java爬虫框架）
756 starsJun 13, 2026 pushweb-automationJavaCrawler
$ npx skills add xuxueli/xxl-crawler#12
Python API for JMComic | 提供Python API访问禁漫天堂，同时支持网页端和移动端 | 禁漫天堂GitHub Actions下载器🚀
6.0K starsJun 11, 2026 pushweb-automationPythonCrawler
$ npx skills add hect0x7/JMComic-Crawler-Python#13
为所有人准备的AI搞钱团队，帮你把经验和方法跑成一门生意。
8.3K starsJun 5, 2026 pushweb-automationTypeScriptCrawler
$ npx skills add TeamWiseFlow/wiseflow#14
Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机
11K starsMay 9, 2026 pushweb-automationPythonCrawler
$ npx skills add injetlee/Python#15
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
2.4K starsJun 11, 2026 pushweb-automationPHPCrawler
$ npx skills add JayBizzle/Crawler-Detect#16
🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量数据去重等功能。更有功能强大的爬虫管理系统feaplat为其提供方便的部署及调度
3.7K starsJun 1, 2026 pushweb-automationPythonCrawler
$ npx skills add Boris-code/feapderHow to choose
When should you switch?
Use an alternative when it has a clearer install path, higher trust score, fresher maintenance, or better platform fit for your current agent stack. Keep DotnetSpider if it already passes your workflow test and repository review.
Next step
Compare top candidates side by side
Open the compare page, test the install commands in a sandbox, and check each repository before using a skill in production.