OpenAgentSkill
GitHub200+

Collect structured data

Web scraping and data extraction skills

Find skills for crawling websites, extracting structured data, monitoring pages, and turning messy web content into agent-ready inputs.

Browse matching skillsRead guideResolve via Agent API

Try this task

I need my agent to scrape websites and extract structured data from pages.

18
Matched
18
Strong trust
2
Install ready
0
Auto allowed
18
500+ stars

Agent should be able to

  • +Crawl target URLs
  • +Extract tables and metadata
  • +Normalize messy page content

Resolve

Let the agent pick

Returns the best skill, alternatives, install handoff, risk summary, and safety gate.

Text plan

LLM-readable output

Plain text version for Codex, Claude Code, Cursor, and custom agent runtimes.

Browse

Human shortlist

Open the filtered registry view for this workflow and compare candidates manually.

Recommended workflow

Turn this use case into a workflow

Scrape, clean, and reuse web data

Web data pipeline workflow

A practical workflow for agents that crawl public pages, extract clean content, normalize data, and hand it to downstream research or RAG workflows.

Workflow map

What to build with these skills

01

Extract product data from websites

02

Monitor competitor pages

03

Turn HTML into clean markdown

04

Feed crawled content into RAG

Best first installs

Start with high-signal skills

18 matched skills

Crawlee

VERIFIED

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

26K stars89 trust93 auditREVIEWED

Install

Human review before install

Risk

Safe to try

Agent fit

OpenAI Agents + Browser agents

Updated

Sep 10, 2026

$ npx skills add apify/crawlee

Crawlee Python

VERIFIED

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

9.5K stars88 trust93 auditREVIEWED

Install

Human review before install

Risk

Safe to try

Agent fit

OpenAI Agents + Browser agents

Updated

Sep 10, 2026

$ npx skills add apify/crawlee-python

Firecrawl

VERIFIED

The API to search, scrape, and interact with the web at scale. 🔥

139K stars85 trust88 auditREVIEWED

Install

Human review before install

Risk

Safe to try

Agent fit

Codex + Claude Code

Updated

Jun 26, 2026

$ npx skills add firecrawl/firecrawl

Skill shortlist

More options for this use case

Browse full marketplace

Scrapling

web-automationCoding

Adaptive web scraping for agent data collection

Review before production · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

70K stars83 trust88 audit72 safety

Maxun

web-automationResearch

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥

Low metadata risk · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

16K stars87 trust90 audit74 safety

Scrapegraph AI

web-automationResearch

Python scraper based on AI

Review before production · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

27K stars87 trust89 audit69 safety

Webapp Testing

browser-automationCoding

Use Playwright to interact with and test local web applications, capture screenshots, debug UI behavior, and inspect browser logs.

Review before production · Review the audit page, then allow agent install in a sandboxed workflow.

180K stars87 trust92 audit76 safety

Browser Use

agent-frameworksCoding

🌐 Make websites accessible for AI agents. Automate tasks online with ease.

Review before production · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

99K stars85 trust88 audit72 safety

Playwright

browser-automationCoding

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

Low metadata risk · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

91K stars87 trust89 audit73 safety

Crawl4AI

web-automationResearch

Open-source LLM-friendly web crawler and scraper

Review before production · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

73K stars88 trust91 audit79 safety

Open Design

agent-skillsResearch

🎨 Local-first, open-source Claude Design alternative. 🖥️ Native desktop app. ⚡ 259+ Skills · ✨ 142+ Design Systems 🖼️ Web · desktop · mobile prototypes · slides · images · videos · HyperFrames 📦 Sandboxed preview · HTML/PDF/PPTX/MP4 export 🤖 Claude Code / OpenClaw / Codex / Cursor / OpenCode / Qwen / Copilot / Hermes / Kimi & 17+ CLIs.

Low metadata risk · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

88K stars89 trust92 audit84 safety

D3

data-analysisData

Bring data to life with SVG, Canvas and HTML. :bar_chart::chart_with_upwards_trend::tada:

Low metadata risk · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

113K stars86 trust88 audit64 safety

30 Seconds Of Code

educationEducation

Coding articles to level up your development skills

Review before production · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

128K stars87 trust89 audit77 safety

Echarts

data-analysisData

Apache ECharts is a powerful, interactive charting and data visualization library for browser

Low metadata risk · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

67K stars89 trust91 audit75 safety

Web Dev For Beginners

educationEducation

24 Lessons, 12 Weeks, Get Started as a Web Developer

Review before production · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

96K stars87 trust89 audit77 safety

Front End Checklist

agent-frameworksCoding

🗂 The essential checklist for modern web development, for humans and AI agents

Review before production · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

73K stars84 trust87 audit75 safety

MinerU

document-processingResearch

Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.

Review before production · Skill source structure is not confirmed in the registry. Inspect the source and identify valid skill instructions before proposing an installation. A repository URL or GitHub stars do not prove installability.

68K stars85 trust87 audit71 safety

Guizang Ppt Skill

agent-skillsPresentation

AI-agent Skill for generating polished HTML slide decks: editorial magazine and Swiss layouts, image prompts, social covers, and a WebGL/low-power presentation runtime.

Low metadata risk · Review the audit page, then allow agent install in a sandboxed workflow.

25K stars90 trust93 audit81 safety

FAQ

How to choose skills for this workflow

These answers are written for both human builders and agents consuming the Registry API.

What are the best AI agent skills for web scraping?

Start by comparing Crawlee, Crawlee Python, Firecrawl. OpenAgentSkill ranks them by workflow fit, GitHub adoption, trust score, safety gate, and install readiness.

Can an AI agent use this page directly?

Yes. Use the linked Registry API prompt to query /api/skills/search with the task: "I need my agent to scrape websites and extract structured data from pages." and retrieve install handoff links for the top results.

Should I install every recommended skill?

No. Start with the highest-fit skill, test it in a sandbox workflow, and add companion skills only when the task needs extra coverage.