AI Agent Skills

List of libraries, tools and APIs for web scraping and data processing.

Generic automation framework for acceptance testing and RPA

Scrapy, a fast high-level web crawling & scraping framework for Python.

Elegant Scraper and Crawler Framework for Golang

Web crawling built for AI

Adaptive web scraping for agent data collection

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbo…

A next-generation crawling and spidering framework.

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

FlowGram is an extensible workflow development framework with built-in canvas, form, variable, and materials that helps developers build AI workflow platforms faster and…

A Chrome DevTools Protocol driver for web automation and scraping.

Event-driven Automation Framework for Kubernetes

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Down…

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and o…

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

Cross-platform automation framework for all kinds of apps, built on top of the W3C WebDriver protocol