https://spatie.be/docs/crawler

Library for Rapid (Web) Crawler and Scraper Development

Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or filesystem to various data repositories such as search engines.

🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)

K 哥爬虫代码分享，JS 逆向，爬虫进阶。关注公众号：K哥爬虫

A high performance web crawler / scraper in Elixir.

Open-source LLM-friendly web crawler and scraper

Elegant Scraper and Crawler Framework for Golang

小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 知乎问答文章 | 评论爬虫。支持多平台社交媒体内容抓取，提供完整的数据采集解决方案。

GEO-first SEO skill for Claude Code. Comprehensive AI search optimization for any website — citability scoring, AI crawler analysis, brand authority, schema markup, platform-specific optimization, and PDF reports. If you want learn how to sell this to real businesses, check out the skool community

Web Crawler/Spider for NodeJS + server-side jQuery ;-)

Python API for JMComic | 提供Python API访问禁漫天堂，同时支持网页端和移动端 | 禁漫天堂GitHub Actions下载器🚀