Stormcrawler
apache
A scalable, mature and versatile web crawler based on Apache Storm
Web ScrapingReview before use
995GitHub
OPENAGENTSKILL / DIRECTORY
AI Agent Skills
Find a skill for your next task. Explore tools for Codex, Claude Code, Cursor and more.
Results · “distributed-crawler”
1–16 / 72
More filters
Results: 72
Node Crawler
bda-research
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
Web ScrapingReview before use
6.8KGitHub
Crawler Illegal Cases In China
hiddendevj
Collection of China illegal cases about web crawler 本项目用来整理所有中国大陆爬虫开发者涉诉与违规相关的新闻、资料与法律法规。致力于帮助在中国大陆工作的爬虫行业从业者了解我国相关法律，避免触碰数据合规红线。
Web ScrapingReview before use
4.6KGitHub
Awesome Crawler
BruceDone
A collection of awesome web crawler,spider in different languages
Web ScrapingReview before use
7.2KGitHub
JMComic Crawler Python
hect0x7
Python API for JMComic | 提供Python API访问禁漫天堂，同时支持网页端和移动端 | 禁漫天堂GitHub Actions下载器🚀
Web ScrapingReview before use
6.5KGitHub
4.5KGitHub
2.8KGitHub
Work Crawler
kanasimi
Download comics novels 小说漫画下载工具 小説漫画のダウンローダ 小說漫畫下載:腾讯漫画 大角虫漫画 有妖气 咪咕 SF漫画 哦漫画 看漫画 漫画柜 汗汗酷漫 動漫伊甸園 快看漫画 微博动漫 733动漫网 大古漫画网 漫画DB 無限動漫 動漫狂 卡推漫画 动漫之家 动漫屋 古风漫画网 36漫画网 亲亲漫画网 乙女漫…
Web ScrapingReview before use
4.1KGitHub
Crawler Detect
JayBizzle
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
Web ScrapingReview before use
2.4KGitHub
1.4KGitHub
Browsertrix Crawler
webrecorder
Run a high-fidelity browser-based web archiving crawler in a single Docker container
1.1KGitHub
5.6KGitHub
MediaCrawler
NanmiCoder
小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 知乎问答文章 | 评论爬虫。支持多平台社交媒体内容抓取，提供完整的数据采集解决方案。
DataReview before use
56.9KGitHub
Crawlab
crawlab-team
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
Web ScrapingReview before use
12.2KGitHub
Haipproxy
SpiderClub
:sparkling_heart: High available distributed ip proxy pool, powerd by Scrapy and Redis
Web ScrapingReview before use
5.5KGitHub
11.7KGitHub
External platform skills
Browse external directory →Owner-curated external sources. Not filtered by the scores or compatibility controls above; excluded from GitHub rankings and automatic installation.
No external entries match this query.