Web scraping skills

AI agent skills for web scraping.

Find reusable crawler, browser automation, HTML-to-markdown, and structured extraction skills that agents can compare before scraping public websites.

Built for builders searching for reliable web scraping skills, crawler skills, and browser automation skills for AI agents.

Matched

16

Stars

892K

Workflow

Crawl

Output

Structured data

Agent jobs

Start from a real workflow, not a keyword.

These pages are built for high-intent search and for agents that need a structured shortlist with install commands, trust signals, audit links, and real outcome evidence before installing third-party code.

01

Scrape public pricing pages into JSON or markdown

02

Crawl documentation sites for RAG ingestion

03

Monitor public web pages and extract changed fields

04

Use browser automation when static HTML is not enough

Ranked shortlist

High-signal skills to inspect first.

Open best list
26K stars

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

100

Quality

91

Trust

59

Proven

browser-automationSep 10, 2026 pushApache-2.0Early agent signal
$ npx skills add apify/crawlee
9.5K stars

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

100

Quality

85

Trust

42

Proven

browser-automationSep 10, 2026 pushApache-2.0Early agent signal
$ npx skills add apify/crawlee-python
139K stars

The API to search, scrape, and interact with the web at scale. 🔥

100

Quality

78

Trust

33

Proven

agent-frameworksJun 26, 2026 pushAGPL-3.0Early agent signal
$ npx skills add firecrawl/firecrawl
70K stars

Adaptive web scraping for agent data collection

100

Quality

82

Trust

—

Proven

web-automationJul 15, 2026 pushUnknownNeeds first agent run
$ npx skills add D4Vinci/Scrapling
16K stars

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥

100

Quality

68

Trust

31

Proven

web-automationJul 2, 2026 pushAGPL-3.0Needs agent review
$ npx skills add getmaxun/maxun
16K stars

The headless browser for AI agents and web scraping

100

Quality

85

Trust

—

Proven

web-automationJun 18, 2026 pushApache-2.0Needs first agent run
$ npx skills add h4ckf0r0day/obscura

Evaluation

How to choose the right skill.

Scope controls for crawl depth and allowed domains

Clean markdown or structured output support

Fresh maintenance and visible repository activity

Clear install command and sandbox-friendly usage

Questions

What is the best web scraping skill for an AI agent?

Start with skills that return clean markdown or structured data, expose crawl limits, and have strong maintenance signals. OpenAgentSkill ranks candidates by task fit, trust, quality, stars, and install readiness.

Can an agent use these skills automatically?

Yes. Agents can call the Resolve API with a scraping task, inspect the ranked shortlist, and fetch install handoffs before running anything.