AI Agent Skills

List of libraries, tools and APIs for web scraping and data processing.

Scrapy, a fast high-level web crawling & scraping framework for Python.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Web crawling built for AI

Lighter web automation with Python

Give your AI agent a web browser

Adaptive web scraping for agent data collection

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, a…

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

A Chrome DevTools Protocol driver for web automation and scraping.

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Down…

The API to search, scrape, and interact with the web at scale. 🔥

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and o…

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

A modern ebook manager and reader with sync and backup capacities for Windows, macOS, Linux, Android, iOS and Web

Python ProxyPool for web spider