AI Agent Skills

Collect Zhihu answers, articles, videos, comments, and creator evidence with MediaCrawler through search and exact URLs. Use for expert discourse, problem framing, objec…

Crawl a website starting from a URL, find relevant pages, and extract data – all guided by your natural language prompt.

This email crawler will visit all pages of a provided website and parse and save emails found to a csv file.

Edit the theme's robots.txt.liquid to explicitly allow AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, Amazonbot) so AI assistants are per…

带你了解一下Golang的市场行情

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

A scalable web crawler framework for Java.

news-please - an integrated web crawler and information extractor for news that just works

Easy to use lightweight web crawler（易用的轻量化网络爬虫）

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.

Polite, slim and concurrent web crawler.

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

A simple and flexible web crawler that follows the robots.txt policies and crawl delays.

Web crawling built for AI

Elegant Scraper and Crawler Framework for Golang

Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM…