AI Agent Skills

List of libraries, tools and APIs for web scraping and data processing.

The web scraping open project repository aims to share knowledge and experiences about web scraping with Python

Web crawling built for AI

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM…

Give your AI agent a web browser

Turn websites into clean markdown or structured data for retrieval and agents.

Adaptive web scraping for agent data collection

A command-line utility for taking automated screenshots of websites

Scrapy, a fast high-level web crawling & scraping framework for Python.

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Down…

🤖 Scrape data from HTML websites automatically by just providing examples

A Chrome DevTools Protocol driver for web automation and scraping.

Swiss-army tool for scraping and extracting data from online assets, made for hackers

Web Scraping Framework

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.