LLMWebCrawler
Aavache
A Web Crawler based on LLMs implemented with Ray and Huggingface. The embeddings are saved into a vector database for fast clustering and retrieval. Use it for your RAG.
104GitHub
OPENAGENTSKILL / DIRECTORY
AI Agent Skills
Find a skill for your next task. Explore tools for Codex, Claude Code, Cursor and more.
Find a skill for your next task. Preview examples where available.
Results · “crawler”
1–7 / 7
Results: 7
AnyCrawl
any4ai
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading…
3.4KGitHub
geo
zubair-trabzada
GEO-first SEO analysis tool. Optimizes websites for AI-powered search engines (ChatGPT, Claude, Perplexity, Gemini, Google AI Overviews) while maintaining traditional SE…
10.1KGitHub
agentmarkup
agentmarkup
Install, configure, audit, and fix AgentMarkup machine-readable website metadata in JavaScript web repos. Use when adding @agentmarkup/vite, @agentmarkup/astro, @agentma…
24GitHub
orangeo-ai-visibility-skill
OranAi-Ltd
Audit brand AI visibility readiness and prepare OranGEO-style GEO, AEO, LLM SEO, and AI search optimization action plans. Use when asked for a Claude Code skill, Codex s…
133GitHub
geo
TheSmokeDev
GEO-first SEO analysis tool. Optimizes websites for AI-powered search engines (ChatGPT, Claude, Perplexity, Gemini, Google AI Overviews) while maintaining traditional SE…
22GitHub
aeo
alirezarezvani
Answer Engine Optimization (AEO) skill — optimize content to be cited by AI language models (ChatGPT, Perplexity, Claude, Gemini, Mistral) as authoritative sources. Dist…
24.8KGitHub