AI Agent Skills

A scalable, mature and versatile web crawler based on Apache Storm

Web Crawler/Spider for NodeJS + server-side jQuery ;-)

Collection of China illegal cases about web crawler 本项目用来整理所有中国大陆爬虫开发者涉诉与违规相关的新闻、资料与法律法规。致力于帮助在中国大陆工作的爬虫行业从业者了解我国相关法律，避免触碰数据合规红线。

A collection of awesome web crawler,spider in different languages

Python API for JMComic | 提供Python API访问禁漫天堂，同时支持网页端和移动端 | 禁漫天堂GitHub Actions下载器🚀

新浪微博爬虫，用python爬取新浪微博数据，并下载微博图片和微博视频

https://spatie.be/docs/crawler

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

Elasticsearch File System Crawler (FS Crawler)

Run a high-fidelity browser-based web archiving crawler in a single Docker container

Crawl a website starting from a URL, find relevant pages, and extract data – all guided by your natural language prompt.

A scalable web crawler framework for Java.

Easy to use lightweight web crawler（易用的轻量化网络爬虫）

Elegant Scraper and Crawler Framework for Golang

🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量…

:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。