WebScrapper
nuhmanpk
Powerful Telegram bot for web scraping and crawling. Fast, easy, and loved by thousands!
204GitHub
OPENAGENTSKILL / DIRECTORY
AI Agent Skills
Find a skill for your next task. Explore tools for Codex, Claude Code, Cursor and more.
Find a skill for your next task. Preview examples where available.
Results · “crawling”
17–32 / 36
Results: 36
Oxylabs AI Studio Js
oxylabs
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM…
143GitHub
Kuri
justrach
Browser automation, web crawling, and iOS + Android device control for AI agents. Zig-native, token-efficient CDP snapshots, HAR recording, native adb wire-protocol clie…
331GitHub
Wxpath
rodricios
wxpath - declarative web crawling with XPath; a Web Query Language (WQL)
112GitHub
Geo AI Agent
brightdata
AI-powered tool to audit and optimize website content by crawling URLs, analyzing H1s, and generating actionable GEO recommendations with CrewAI.
164GitHub
Sitemap Generator Cli
lgraubner
Creates an XML-Sitemap by crawling a given site.
338GitHub
Open Semantic Etl
opensemanticsearch
Python based Open Source ETL tools for file crawling, document processing (text extraction, OCR), content analysis (Entity Extraction & Named Entity Recognition) & data…
281GitHub
existing-site-analyzer
sergekostenchuk
Analyze an existing website before improve, redesign, rebuild, or migration decisions. Use when a project has a current URL, local HTML/codebase, screenshots, sitemap, r…
24GitHub
Tushare
waditu
TuShare is a utility for crawling historical data of China stocks
15.3KGitHub
Sotawhat
chiphuyen
Returns latest research results by crawling arxiv papers and summarizing abstracts. Helps you stay afloat with so many new papers everyday.
1.4KGitHub
geo-llmstxt
zubair-trabzada
Analyzes and generates llms.txt files -- the emerging standard for helping AI systems understand website structure and content. Can validate existing llms.txt files or g…
10.1KGitHub
Hydra Link Checker
victoriadrake
Hydra: a multithreaded site-crawling link checker in Python standard library
125GitHub
Lantern
royalicing
251GitHub
geo-knowledge
yaojingang
Build and query an evidence-lined GEO knowledge graph from approved source bundles. Use for entity normalization, relation lineage, conflicting fact preservation, increm…
152GitHub
seo
yaojingang
Turn one natural-language request into an evidence-bounded SEO work plan across technical SEO, crawling and indexing, Search Console incidents, keyword-to-page mapping,…
152GitHub
:mag: Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript
165GitHub