AI Agent Skills

A scalable web crawler framework for Java.

Incredibly fast crawler designed for OSINT.

Easy to use lightweight web crawler（易用的轻量化网络爬虫）

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

Polite, slim and concurrent web crawler.

:zap: A distributed crawler for weibo, building with celery and requests.

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

a new crawler based on python with more function including Network fingerprint search

PHP script to recursively crawl websites and generate a sitemap. Zero dependencies.

A simple and flexible web crawler that follows the robots.txt policies and crawl delays.

HTML to Markdown converter and crawler.

带你了解一下Golang的市场行情

Elegant Scraper and Crawler Framework for Golang

🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量…

:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

This is a wencai crawler.（i问财的策略回测接口的Pythonic工具包）