AI Agent Skills

List of libraries, tools and APIs for web scraping and data processing.

Scrapy, a fast high-level web crawling & scraping framework for Python.

Python API for JMComic | 提供Python API访问禁漫天堂，同时支持网页端和移动端 | 禁漫天堂GitHub Actions下载器🚀

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and o…

The process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Lighter web automation with Python

Python tool for converting files and office documents to Markdown.

Python scraper based on AI

Adaptive web scraping for agent data collection

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

Python binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.

Python ProxyPool for web spider

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

A pure-python PDF library capable of splitting, merging, cropping, and transforming the pages of PDF files

Small python-gtk application, which helps the user to merge or split PDF documents and rotate, crop and rearrange their pages using an interactive and intuitive graphica…