AI Agent Skills

List of libraries, tools and APIs for web scraping and data processing.

The web scraping open project repository aims to share knowledge and experiences about web scraping with Python

Open-source LLM-friendly web crawler and scraper for agent workflows.

Adaptive web scraping for agent data collection

Scrapy, a fast high-level web crawling & scraping framework for Python.

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

A Chrome DevTools Protocol driver for web automation and scraping.

Swiss-army tool for scraping and extracting data from online assets, made for hackers

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Down…

Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM…

Web Scraping Framework

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

Getting started with Puppeteer and Chrome Headless for Web Scraping

Simple web scraping for R

The complete web scraping toolkit for PHP.

Lightweight library for scraping web-sites with LLMs