Web scraping skills
Find reusable crawler, browser automation, HTML-to-markdown, and structured extraction skills that agents can compare before scraping public websites.
Built for builders searching for reliable web scraping skills, crawler skills, and browser automation skills for AI agents.
Matched
16
Stars
459K
Workflow
Crawl
Output
Structured data
Agent jobs
These pages are built for high-intent search and for agents that need a structured shortlist before installing third-party code.
01
Scrape public pricing pages into JSON or markdown
02
Crawl documentation sites for RAG ingestion
03
Monitor public web pages and extract changed fields
04
Use browser automation when static HTML is not enough
Task routes
Scrape pricing pages
Extract pricing data from public competitor pages and turn it into clean structured output.
Crawl docs
Turn documentation pages into clean markdown or records that an agent can search and reuse.
Browser workflow
Navigate web apps, fill forms, take screenshots, and verify state through browser automation.
Ranked shortlist
#01
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
100
Quality
100
Trust
74
Fit
$ npx skills add apify/crawleeCrawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
100
Quality
100
Trust
69
Fit
$ npx skills add apify/crawlee-python#03
The API to search, scrape, and interact with the web at scale. 🔥
100
Quality
100
Trust
78
Fit
$ npx skills add firecrawl/firecrawlPython scraper based on AI
100
Quality
100
Trust
74
Fit
$ npx skills add ScrapeGraphAI/Scrapegraph-ai#05
🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
100
Quality
100
Trust
73
Fit
$ npx skills add getmaxun/maxun#06
Open-source LLM-friendly web crawler and scraper
100
Quality
100
Trust
79
Fit
$ npx skills add unclecode/crawl4aiEvaluation
Scope controls for crawl depth and allowed domains
Clean markdown or structured output support
Fresh maintenance and visible repository activity
Clear install command and sandbox-friendly usage
Questions
Start with skills that return clean markdown or structured data, expose crawl limits, and have strong maintenance signals. OpenAgentSkill ranks candidates by task fit, trust, quality, stars, and install readiness.
Yes. Agents can call the Resolve API with a scraping task, inspect the ranked shortlist, and fetch install handoffs before running anything.