AI Agent Skills

Scrapy, a fast high-level web crawling & scraping framework for Python.

List of libraries, tools and APIs for web scraping and data processing.

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

Open-source LLM-friendly web crawler and scraper for agent workflows.

Adaptive web scraping for agent data collection

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

A Chrome DevTools Protocol driver for web automation and scraping.

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Down…

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and o…

The headless browser for AI agents and web scraping

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥

Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.

Declarative web scraping

Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️‍♂️ when scraping…

Local-first AI job intelligence workbench for scraping roles, ranking fit, and generating tailored application materials.