AI Agent Skill Repository
Browse reusable skills for Codex, Claude Code, Cursor, finance, research, web scraping, PPT, football analytics, data, marketing, design, and more.
Live registry search
Exact name and slug matches are checked against the live registry before ranked alternatives.
Decision filters
Showing 1-16 of 151 ranked candidates matching "crawl"
Best blend of relevance, quality, freshness, and verified outcomes
Flexible Node.js AI-assisted crawler library
$ npx skills add coder-hxl/x-crawlScenario Testing and QA · I need my agent to test a web app, reproduce bugs, and verify fixes.
Browser agents + CLI · 4 targets
Web data for AI applications
$ npx skills add firecrawl/firecrawlScenario Document processing · I need my agent to read PDFs, extract tables, and turn documents into structured data.
Claude Code + OpenAI Agents · 4 targets
📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.
$ npx skills add InstaPy/instagram-profilecrawlScenario Testing and QA · I need my agent to test a web app, reproduce bugs, and verify fixes.
Browser agents + CLI · 4 targets
Web crawling built for AI
$ npx skills add unclecode/crawl4aiScenario Research agents · I need my agent to research a topic, compare sources, and produce a concise report.
Claude Code + OpenAI Agents · 4 targets
Free desktop SEO crawler - open source alternative to Screaming Frog and similar tools. Crawl websites, analyze links, extract SEO data, and export results without subsc…
$ npx skills add PhialsBasement/LibreCrawlScenario Marketing and growth · I need my agent to research keywords, improve SEO, analyze growth channels, and prepare marketing workflows.
CLI + Codex · 4 targets
Crawl and extract (regular or onion) webpages through TOR network
$ npx skills add MikeMeliz/TorCrawl.pyScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Crawl telegra.ph searching for nudes!
$ npx skills add yaroslaff/nudecrawlerScenario RAG and knowledge · I need my agent to build a RAG workflow over documents and retrieve reliable context.
Browser agents + CLI · 4 targets
Crawl a website starting from a URL, find relevant pages, and extract data – all guided by your natural language prompt.
$ npx skills add oxylabs/ai-crawler-pyScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
PHP script to recursively crawl websites and generate a sitemap. Zero dependencies.
$ npx skills add vezaynk/Sitemap-Generator-CrawlerScenario Marketing and growth · I need my agent to research keywords, improve SEO, analyze growth channels, and prepare marketing workflows.
CLI + Codex · 4 targets
CrawlAI RAG is an AI-powered website intelligence platform that allows users to crawl entire websites, index their content, and ask natural-language questions using Retr…
$ npx skills add AnkitNayak-eth/CrawlAI-RAGScenario RAG and knowledge · I need my agent to build a RAG workflow over documents and retrieve reliable context.
CLI + Codex · 4 targets
A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。
$ npx skills add NaiboWang/EasySpiderScenario Coding agents · I need a coding agent that can understand a repository, edit code, and review pull requests.
CLI + Codex · 4 targets
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
$ npx skills add PuerkitoBio/fetchbotScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
$ npx skills add edoardottt/cariddiScenario Testing and QA · I need my agent to test a web app, reproduce bugs, and verify fixes.
CLI + Codex · 4 targets
[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you ca…
$ npx skills add hu17889/go_spiderScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Scrapy, a fast high-level web crawling & scraping framework for Python.
$ npx skills add scrapy/scrapyScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Crawl BookCorpus
$ npx skills add soskek/bookcorpusScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Page 1
Showing the strongest 16 results to keep the registry fast for humans and agents. Refine by use case, platform, stars, or search query for a narrower shortlist.
Try the agent resolve APIAI Agent Skill Repository
Browse reusable skills for Codex, Claude Code, Cursor, finance, research, web scraping, PPT, football analytics, data, marketing, design, and more.
Live registry search
Exact name and slug matches are checked against the live registry before ranked alternatives.
Decision filters
Showing 1-16 of 151 ranked candidates matching "crawl"
Best blend of relevance, quality, freshness, and verified outcomes
Flexible Node.js AI-assisted crawler library
$ npx skills add coder-hxl/x-crawlScenario Testing and QA · I need my agent to test a web app, reproduce bugs, and verify fixes.
Browser agents + CLI · 4 targets
Web data for AI applications
$ npx skills add firecrawl/firecrawlScenario Document processing · I need my agent to read PDFs, extract tables, and turn documents into structured data.
Claude Code + OpenAI Agents · 4 targets
📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.
$ npx skills add InstaPy/instagram-profilecrawlScenario Testing and QA · I need my agent to test a web app, reproduce bugs, and verify fixes.
Browser agents + CLI · 4 targets
Web crawling built for AI
$ npx skills add unclecode/crawl4aiScenario Research agents · I need my agent to research a topic, compare sources, and produce a concise report.
Claude Code + OpenAI Agents · 4 targets
Free desktop SEO crawler - open source alternative to Screaming Frog and similar tools. Crawl websites, analyze links, extract SEO data, and export results without subsc…
$ npx skills add PhialsBasement/LibreCrawlScenario Marketing and growth · I need my agent to research keywords, improve SEO, analyze growth channels, and prepare marketing workflows.
CLI + Codex · 4 targets
Crawl and extract (regular or onion) webpages through TOR network
$ npx skills add MikeMeliz/TorCrawl.pyScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Crawl telegra.ph searching for nudes!
$ npx skills add yaroslaff/nudecrawlerScenario RAG and knowledge · I need my agent to build a RAG workflow over documents and retrieve reliable context.
Browser agents + CLI · 4 targets
Crawl a website starting from a URL, find relevant pages, and extract data – all guided by your natural language prompt.
$ npx skills add oxylabs/ai-crawler-pyScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
PHP script to recursively crawl websites and generate a sitemap. Zero dependencies.
$ npx skills add vezaynk/Sitemap-Generator-CrawlerScenario Marketing and growth · I need my agent to research keywords, improve SEO, analyze growth channels, and prepare marketing workflows.
CLI + Codex · 4 targets
CrawlAI RAG is an AI-powered website intelligence platform that allows users to crawl entire websites, index their content, and ask natural-language questions using Retr…
$ npx skills add AnkitNayak-eth/CrawlAI-RAGScenario RAG and knowledge · I need my agent to build a RAG workflow over documents and retrieve reliable context.
CLI + Codex · 4 targets
A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。
$ npx skills add NaiboWang/EasySpiderScenario Coding agents · I need a coding agent that can understand a repository, edit code, and review pull requests.
CLI + Codex · 4 targets
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
$ npx skills add PuerkitoBio/fetchbotScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more
$ npx skills add edoardottt/cariddiScenario Testing and QA · I need my agent to test a web app, reproduce bugs, and verify fixes.
CLI + Codex · 4 targets
[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you ca…
$ npx skills add hu17889/go_spiderScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Scrapy, a fast high-level web crawling & scraping framework for Python.
$ npx skills add scrapy/scrapyScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Crawl BookCorpus
$ npx skills add soskek/bookcorpusScenario GitHub automation · I need my agent to triage GitHub issues, review pull requests, and summarize repository changes.
CLI + Codex · 4 targets
Page 1
Showing the strongest 16 results to keep the registry fast for humans and agents. Refine by use case, platform, stars, or search query for a narrower shortlist.
Try the agent resolve API