AI Agent Skills

网页抓取技能——单页抓取与整站爬取的决策与落盘规范。触发：需要把文档站/系列页面抓下来长期引用、离线存档、或为知识库补充原始材料时。

List of libraries, tools and APIs for web scraping and data processing.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Python scraper based on AI

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 知乎问答文章 | 评论爬虫。支持多平台社交媒体内容抓取，提供完整的数据采集解决方案。

Scrapy, a fast high-level web crawling & scraping framework for Python.

Web crawling built for AI

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

A next-generation crawling and spidering framework.

Lighter web automation with Python

Adaptive web scraping for agent data collection

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, a…

Python ProxyPool for web spider

A Chrome DevTools Protocol driver for web automation and scraping.

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Down…