XSRFProbe alternatives for AI agents.

Python ProxyPool for web spider

🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量数据去重等功能。更有功能强大的爬虫管理系统feaplat为其提供方便的部署及调度

Every web site provides APIs.

Web crawling framework based on asyncio.

Elegant Scraper and Crawler Framework for Golang

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、微信读书、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

Web Crawler/Spider for NodeJS + server-side jQuery ;-)

:sparkling_heart: High available distributed ip proxy pool, powerd by Scrapy and Redis

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

爬虫逆向案例，已完成：TLS指纹｜瑞数｜震坤行 | 网易易盾 | 微信小程序反编译逆向(百达星系) | 同花顺 | rpc解密 | 加速乐 | 极验滑块验证码 | 巨量算数 | Boss直聘 | 企查查 | 中国五矿 | qq音乐 | 产业政策大数据平台 | 企知道 | 雪球网(acw_sc__v2) | 1688 | 七麦数据 | whggzy | 企名科技 | mohurd | 艺恩数据 | 欧科云链

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

Python API for JMComic | 提供Python API访问禁漫天堂，同时支持网页端和移动端 | 禁漫天堂GitHub Actions下载器🚀

Redis-based components for Scrapy.

:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机