AI Agent Skills

A scalable web crawler framework for Java.

news-please - an integrated web crawler and information extractor for news that just works

Easy to use lightweight web crawler（易用的轻量化网络爬虫）

Turn websites into clean markdown or structured data for retrieval and agents.

Web crawling built for AI

Elegant Scraper and Crawler Framework for Golang

Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM…

🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量…

:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

Incredibly fast crawler designed for OSINT.

Flexible Node.js AI-assisted crawler library

Automate a real browser from the terminal for navigation, form filling, snapshots, screenshots, extraction, and UI-flow debugging.

Browser automation layer for agents that need to interact with websites.

Reliable browser automation and testing engine for web agent tasks.

AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading…

Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application