Collect structured data

Best AI agent skills for web scraping

Find skills for crawling websites, extracting structured data, monitoring pages, and turning messy web content into agent-ready inputs.

Builders choosing skills for extract product data from websites and monitor competitor pages. Ranked from the OpenAgentSkill index using quality, trust, freshness, adoption, and install readiness.

best AI agent skills for web scraping

Compare top 4 Filter production candidates Resolve this workflow

30

Ranked

964K

Stars

92

Top trust

59

Agent proven

Search intent

Find reusable AI agent skills for crawling websites, extracting structured data, and preparing web content for downstream workflows.

These pages are generated from real registry records. The list below is not a generic article; every row links to a skill profile with install, trust, audit, and risk fields.

Resolve task

Scrape competitor pricing pages into a clean table

Resolve task

Turn public websites into markdown for RAG

Resolve task

Monitor product pages and extract structured changes

#1

Crawlee

53 fitTrust 92Excellent 100Agent Proven 59Audit 93 · Safe to try

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Excellent quality, 26K stars, and a 35 use-case fit score.

Best suited scenario

Crawl target URLs

26K starsSep 10, 2026 pushProduction candidateEarly agent signalTypeScriptPlaywright

Alternatives Compare Audit

$ npx skills add apify/crawlee

#2

Crawlee Python

51 fitTrust 85Excellent 100Agent Proven 42Audit 91 · Safe to try

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

Excellent quality, 9.5K stars, and a 34 use-case fit score.

Best suited scenario

Crawl target URLs

9.5K starsSep 10, 2026 pushStrong shortlistEarly agent signalPythonPlaywright

Alternatives Compare Audit

$ npx skills add apify/crawlee-python

#3

Webapp Testing

49 fitTrust 93Excellent 100Agent Proven 73Audit 92 · Safe to try

Use Playwright to interact with and test local web applications, capture screenshots, debug UI behavior, and inspect browser logs.

Excellent quality, 180K stars, and a 23 use-case fit score.

Best suited scenario

Navigate pages

180K starsOct 9, 2026 pushProduction candidatePromising agent evidenceClaude CodeCodex

Alternatives Compare Audit

$ npx skills add anthropics/skills --skill webapp-testing

#4

Scrapling

44 fitTrust 83Excellent 100Agent Proven —Audit 88 · Needs review

Adaptive web scraping for agent data collection

Excellent quality, 70K stars, and a 26 use-case fit score.

Best suited scenario

Crawl target URLs

70K starsJul 15, 2026 pushStrong shortlistNeeds first agent runPythonWeb Automation

Alternatives Compare Audit

$ npx skills add D4Vinci/Scrapling

#5

Maxun

43 fitTrust 68Excellent 100Agent Proven 31Audit 90 · Safe to try

🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥

Excellent quality, 16K stars, and a 25 use-case fit score.

Best suited scenario

Crawl target URLs

16K starsJul 2, 2026 pushManual reviewNeeds agent reviewTypeScriptBrowser Automation

Alternatives Compare Audit

$ npx skills add getmaxun/maxun

#6

GenericAgent

43 fitTrust 86Excellent 100Agent Proven —Audit 91 · Needs review

Self-evolving agent: grows skill tree from 3.3K-line seed, achieving full system control with 6x less token consumption

Excellent quality, 14K stars, and a 17 use-case fit score.

Best suited scenario

Navigate pages

14K starsSep 30, 2026 pushProduction candidateNeeds first agent runPythonAI Agents

Alternatives Compare Audit

$ npx skills add lsdefine/GenericAgent

#7

Obscura

43 fitTrust 85Excellent 100Agent Proven —Audit 88 · Safe to try

The headless browser for AI agents and web scraping

Excellent quality, 16K stars, and a 25 use-case fit score.

Best suited scenario

Crawl target URLs

16K starsJun 18, 2026 pushStrong shortlistNeeds first agent runRustBrowser Automation

Alternatives Compare Audit

$ npx skills add h4ckf0r0day/obscura

#8

Playwright Browser Skill

43 fitTrust 81Excellent 100Agent Proven —Audit 88 · Safe to try

Automate a real browser from the terminal for navigation, form filling, snapshots, screenshots, extraction, and UI-flow debugging.

Excellent quality, 28K stars, and a 21 use-case fit score.

Best suited scenario

Navigate pages

28K starsSep 8, 2026 pushStrong shortlistNeeds first agent runCodexClaude Code

Alternatives Compare Audit

$ npx skills add openai/skills --skill playwright

#9

Browser Use

42 fitTrust 85Excellent 100Agent Proven —Audit 88 · Safe to try

🌐 Make websites accessible for AI agents. Automate tasks online with ease.

Excellent quality, 99K stars, and a 22 use-case fit score.

Best suited scenario

Crawl target URLs

99K starsJun 13, 2026 pushStrong shortlistNeeds first agent runPythonAI Agents

Alternatives Compare Audit

$ npx skills add browser-use/browser-use

#10

Pydoll

42 fitTrust 88Excellent 100Agent Proven —Audit 93 · Safe to try

Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.

Excellent quality, 7.1K stars, and a 19 use-case fit score.

Best suited scenario

Crawl target URLs

7.1K starsSep 16, 2026 pushProduction candidateNeeds first agent runPythonBrowser Automation

Alternatives Compare Audit

$ npx skills add autoscrape-labs/pydoll

#11

AnyCrawl

42 fitTrust 85Excellent 100Agent Proven 42Audit 91 · Safe to try

AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.

Excellent quality, 3.4K stars, and a 25 use-case fit score.

Best suited scenario

Crawl target URLs

3.4K starsAug 11, 2026 pushStrong shortlistEarly agent signalMDXRAG

Alternatives Compare Audit

$ npx skills add any4ai/AnyCrawl

#12

Open Design

41 fitTrust 95Excellent 100Agent Proven 67Audit 92 · Safe to try

🎨 Local-first, open-source Claude Design alternative. 🖥️ Native desktop app. ⚡ 259+ Skills · ✨ 142+ Design Systems 🖼️ Web · desktop · mobile prototypes · slides · images · videos · HyperFrames 📦 Sandboxed preview · HTML/PDF/PPTX/MP4 export 🤖 Claude Code / OpenClaw / Codex / Cursor / OpenCode / Qwen / Copilot / Hermes / Kimi & 17+ CLIs.

Excellent quality, 88K stars, and a 19 use-case fit score.

Best suited scenario

Navigate local resources

88K starsAug 17, 2026 pushProduction candidateEarly agent signalTypeScriptAI Agents

Alternatives Compare Audit

$ npx skills add nexu-io/open-design

#13

OpenCLI

40 fitTrust 83Excellent 100Agent Proven —Audit 88 · Safe to try

Make Any Website into CLI & Use your logged-in browser by AI agent.

Excellent quality, 27K stars, and a 20 use-case fit score.

Best suited scenario

Crawl target URLs

27K starsJul 12, 2026 pushStrong shortlistNeeds first agent runJavaScriptAI Agents

Alternatives Compare Audit

$ npx skills add jackwener/OpenCLI

#14

Claude Code Best Practice

40 fitTrust 87Excellent 100Agent Proven —Audit 89 · Safe to try

from vibe coding to agentic engineering - practice makes claude perfect

Excellent quality, 61K stars, and a 18 use-case fit score.

Best suited scenario

Crawl target URLs

61K starsJun 25, 2026 pushProduction candidateNeeds first agent runHTMLAI Agents

Alternatives Compare Audit

$ npx skills add shanraisshan/claude-code-best-practice

#15

Guizang Ppt Skill

40 fitTrust 97Excellent 100Agent Proven 74Audit 93 · Safe to try

AI-agent Skill for generating polished HTML slide decks: editorial magazine and Swiss layouts, image prompts, social covers, and a WebGL/low-power presentation runtime.

Excellent quality, 25K stars, and a 18 use-case fit score.

Best suited scenario

Crawl target URLs

25K starsAug 26, 2026 pushProduction candidatePromising agent evidenceHTMLAI Agents

Alternatives Compare Audit

$ npx skills add op7418/guizang-ppt-skill

#16

Archify

39 fitTrust 92Excellent 100Agent Proven 75Audit 91 · Safe to try

Any agent Skill: generate beautiful architecture diagrams with dark/light theme toggle and PNG/JPEG/WebP/SVG export

Excellent quality, 47K stars, and a 16 use-case fit score.

Best suited scenario

Crawl target URLs

47K starsSep 4, 2026 pushProduction candidatePromising agent evidenceJavaScriptAI Agents

Alternatives Compare Audit

$ npx skills add tt-a1i/archify

#17

Google Maps Scraper

39 fitTrust 83Excellent 100Agent Proven 42Audit 88 · Safe to try

Google Maps Scraper & Lead Generation Tool. Extract 50+ data points including business emails, phone numbers, and social profiles. Includes enrichment features, API access, and no recurring fees

Excellent quality, 2.8K stars, and a 21 use-case fit score.

Best suited scenario

Crawl target URLs

2.8K starsJun 3, 2026 pushStrong shortlistEarly agent signalSales AutomationBrowser agents

Alternatives Compare Audit

$ npx skills add omkarcloud/google-maps-scraper

#18

Huginn

39 fitTrust 89Excellent 100Agent Proven —Audit 90 · Safe to try

Create agents that monitor and act on your behalf. Your agents are standing by!

Excellent quality, 49K stars, and a 18 use-case fit score.

Best suited scenario

Crawl target URLs

49K starsJun 13, 2026 pushProduction candidateNeeds first agent runRubyAutomation

Alternatives Compare Audit

$ npx skills add huginn/huginn

#19

Browser Harness

39 fitTrust 86Excellent 100Agent Proven —Audit 90 · Safe to try

Browser Harness | Self-healing harness that enables LLMs to complete any task.

Excellent quality, 17K stars, and a 17 use-case fit score.

Best suited scenario

Navigate pages

17K starsAug 31, 2026 pushProduction candidateNeeds first agent runPythonAI Agents

Alternatives Compare Audit

$ npx skills add browser-use/browser-harness

#20

Camoufox

39 fitTrust 84Excellent 99Agent Proven —Audit 88 · Safe to try

🦊 Anti-detect browser

Excellent quality, 9.5K stars, and a 19 use-case fit score.

Best suited scenario

Navigate pages

9.5K starsJun 16, 2026 pushStrong shortlistNeeds first agent runC++Playwright

Alternatives Compare Audit

$ npx skills add daijro/camoufox

#21

Frontend Slides

39 fitTrust 93Excellent 100Agent Proven 67Audit 89 · Safe to try

Create beautiful slides on the web using a coding agent's frontend skills

Excellent quality, 29K stars, and a 18 use-case fit score.

Best suited scenario

Choose the right deck format

29K starsJun 23, 2026 pushProduction candidateEarly agent signalJavaScriptCoding Agent

Alternatives Compare Audit

$ npx skills add zarazhangrui/frontend-slides

#22

Pinchtab

39 fitTrust 88Excellent 100Agent Proven —Audit 92 · Safe to try

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

Excellent quality, 10K stars, and a 17 use-case fit score.

Best suited scenario

Crawl target URLs

10K starsSep 1, 2026 pushProduction candidateNeeds first agent runGoBrowser Automation

Alternatives Compare Audit

$ npx skills add pinchtab/pinchtab

#23

Dashi Ppt Skill

38 fitTrust 80Excellent 96Agent Proven 47Audit 90 · Safe to try

AI agent skill for generating browser-editable presentations with multiple visual themes and export to HTML, PDF, and PPTX.

Excellent quality, 5.2K stars, and a 19 use-case fit score.

Best suited scenario

Crawl target URLs

5.2K starsJul 30, 2026 pushStrong shortlistNeeds agent reviewJavaScriptClaude Code

Alternatives Compare Audit

$ npx skills add chuspeeism/dashi-ppt-skill

#24

Cua

38 fitTrust 90Excellent 100Agent Proven —Audit 93 · Safe to try

Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full desktops (macOS, Linux, Windows).

Excellent quality, 21K stars, and a 17 use-case fit score.

Best suited scenario

Crawl target URLs

21K starsAug 15, 2026 pushProduction candidateNeeds first agent runHTMLComputer Use

Alternatives Compare Audit

$ npx skills add trycua/cua

#25

Opendataloader Pdf

38 fitTrust 86Excellent 100Agent Proven —Audit 89 · Safe to try

PDF Parser for AI-ready data. Automate PDF accessibility. Open-source.

Excellent quality, 26K stars, and a 17 use-case fit score.

Best suited scenario

Chunk documents

26K starsJun 22, 2026 pushProduction candidateNeeds first agent runJavaRAG

Alternatives Compare Audit

$ npx skills add opendataloader-project/opendataloader-pdf

#26

Visual Explainer

38 fitTrust 89Excellent 100Agent Proven —Audit 92 · Safe to try

Agent skill that generates rich HTML pages or slide decks for diagrams, diff reviews, plan audits, data tables, and project recaps

Excellent quality, 9.6K stars, and a 16 use-case fit score.

Best suited scenario

Crawl target URLs

9.6K starsAug 28, 2026 pushProduction candidateNeeds first agent runHTMLAI Agents

Alternatives Compare Audit

$ npx skills add nicobailon/visual-explainer

#27

Video Use

38 fitTrust 87Excellent 100Agent Proven —Audit 89 · Safe to try

A tool that lets AI agents like Claude Code edit videos by cutting filler words, color grading, adding subtitles, and more, all via natural language commands.

Excellent quality, 17K stars, and a 17 use-case fit score.

Best suited scenario

Inspect source files

17K starsJul 1, 2026 pushProduction candidateNeeds first agent runPythonClaude Code

Alternatives Compare Audit

$ npx skills add browser-use/video-use

#28

Skyvern

38 fitTrust 83Excellent 100Agent Proven —Audit 88 · Needs review

Automate browser workflows with AI agents

Excellent quality, 23K stars, and a 17 use-case fit score.

Best suited scenario

Crawl target URLs

23K starsJul 19, 2026 pushStrong shortlistNeeds first agent runPythonBrowser

Alternatives Compare Audit

$ npx skills add Skyvern-AI/skyvern

#29

Midscene

38 fitTrust 86Excellent 100Agent Proven —Audit 90 · Safe to try

AI-powered, vision-driven UI automation for every platform.

Excellent quality, 14K stars, and a 17 use-case fit score.

Best suited scenario

Navigate pages

14K starsJul 24, 2026 pushProduction candidateNeeds first agent runTypeScriptComputer Use

Alternatives Compare Audit

$ npx skills add web-infra-dev/midscene

#30

Claude Seo

38 fitTrust 88Excellent 100Agent Proven —Audit 92 · Safe to try

Universal SEO skill for Claude Code. 25 sub-skills + 18 sub-agents covering technical SEO, E-E-A-T, schema, GEO/AEO, backlinks, local SEO, maps intelligence, semantic clustering, e-commerce SEO, international SEO, Google APIs, and PDF/Excel reporting. Optional DataForSEO, Firecrawl, and Banana extensions.

Excellent quality, 14K stars, and a 17 use-case fit score.

Best suited scenario

Inspect source files

14K starsJul 27, 2026 pushProduction candidateNeeds first agent runPythonClaude Code

Alternatives Compare Audit

$ npx skills add AgriciDaniel/claude-seo

Selection method

How this list is ranked

OpenAgentSkill scores each candidate against the workflow keywords, then balances fit with GitHub stars, quality signals, trust profile, Agent Proven outcome evidence, maintenance freshness, and whether there is a clear install path.

How does OpenAgentSkill rank web scraping?

The ranking combines workflow fit, quality score, trust profile, GitHub adoption, maintenance freshness, and whether a clear install path exists.

Should I install the top skill immediately?

No. Treat the list as a shortlist, open the skill detail page, inspect the repository and license, then test the install command in a sandbox workflow.

Can my agent consume this ranking through an API?

Yes. Use /api/skills/search with the related task or /api/agent/rankings?slug=best-web-scraping-skills to fetch ranked skill data.