AI Agent Skills

zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目

Crawl a website starting from a URL, find relevant pages, and extract data – all guided by your natural language prompt.

🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量…

A scalable web crawler framework for Java.

Easy to use lightweight web crawler（易用的轻量化网络爬虫）

Polite, slim and concurrent web crawler.

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

PHP script to recursively crawl websites and generate a sitemap. Zero dependencies.

A Facebook crawler

A simple and flexible web crawler that follows the robots.txt policies and crawl delays.

带你了解一下Golang的市场行情

Elegant Scraper and Crawler Framework for Golang

:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you ca…

This email crawler will visit all pages of a provided website and parse and save emails found to a csv file.

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion