#1
A scalable web crawler framework for Java.
12K starsDec 20, 2025 pushweb-automationJavaCrawler
$ npx skills add code4craft/webmagicAlternatives
Stormcrawler alternatives for AI agents.
Compare similar skills by workflow fit, trust score, quality, GitHub adoption, maintenance, and install readiness.
Current skill
Stormcrawler
A scalable, mature and versatile web crawler based on Apache Storm
87
Quality
87
Trust
980
Stars
#2
A lightweight web crawler framework.（Java爬虫框架）
756 starsJun 13, 2026 pushweb-automationJavaCrawler
$ npx skills add xuxueli/xxl-crawler#3
Elasticsearch File System Crawler (FS Crawler)
1.4K starsJun 12, 2026 pushweb-automationJavaCrawler
$ npx skills add dadoonet/fscrawler#4
Easy to use lightweight web crawler（易用的轻量化网络爬虫）
2.5K starsJan 23, 2026 pushweb-automationJavaCrawler
$ npx skills add xtuhcy/gecco#5
Redis-based components for Scrapy.
5.6K starsMay 19, 2026 pushweb-automationPythonCrawler
$ npx skills add rmax/scrapy-redis#6
DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
4.1K starsApr 3, 2026 pushweb-automationC#Crawler
$ npx skills add dotnetcore/DotnetSpider#7
Java API For Chrome and Firefox
806 starsMay 20, 2026 pushweb-automationJavaCrawler
$ npx skills add fanyong920/jvppeteer#8
Run a high-fidelity browser-based web archiving crawler in a single Docker container
1.1K starsJun 10, 2026 pushweb-automationTypeScriptCrawler
$ npx skills add webrecorder/browsertrix-crawler#9
Elegant Scraper and Crawler Framework for Golang
25K starsMay 25, 2026 pushweb-automationGoCrawler
$ npx skills add gocolly/colly#10
Fess is very powerful and easily deployable Enterprise Search Server.
1.1K starsJun 11, 2026 pushweb-automationJavaCrawler
$ npx skills add codelibs/fess#11
A collection of awesome web crawler,spider in different languages
7.2K starsJun 16, 2024 pushweb-automationCrawlerClaude Code
$ npx skills add BruceDone/awesome-crawler#12
Python ProxyPool for web spider
23K starsJun 9, 2026 pushweb-automationPythonCrawler
$ npx skills add jhao104/proxy_pool#13
newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
15K starsMay 13, 2026 pushweb-automationPythonCrawler
$ npx skills add codelucas/newspaper#14
A next-generation crawling and spidering framework.
17K starsJun 15, 2026 pushweb-automationGoCrawler
$ npx skills add projectdiscovery/katana#15
新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。
11K starsJun 14, 2023 pushweb-automationJavaCrawler
$ npx skills add ssssssss-team/spider-flow#16
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
6.8K starsJun 4, 2026 pushweb-automationTypeScriptCrawler
$ npx skills add bda-research/node-crawlerHow to choose
When should you switch?
Use an alternative when it has a clearer install path, higher trust score, fresher maintenance, or better platform fit for your current agent stack. Keep Stormcrawler if it already passes your workflow test and repository review.
Next step
Compare top candidates side by side
Open the compare page, test the install commands in a sandbox, and check each repository before using a skill in production.