Crawl4AI
VERIFIED
Web crawling built for AI
Downloads 31.0K
Stars 66.0K
Used by 18.0K agents
Rating 4.9/5
Version 0.4.0
Install with one command
$ npx skills add unclecode/crawl4aiOverview
Crawl4AI is an open-source web crawling framework designed specifically for AI agents. It extracts clean, structured data from websites in LLM-friendly formats like markdown. Supports JavaScript rendering, session management, and proxy rotation.
Platform Compatibility
claudeFULL
gpt-4FULL
langchainFULL
crewaiFULL
openclawFULL
Technical Details
- Version
- 0.4.0
- License
- Apache-2.0
- Last Updated
- 5/22/2026
- Published
- 2/21/2026
Frameworks & Tools
ClaudeGPT-4LangChainCrewAIOpenClaw
Author
U
unclecode✓
@unclecode
Trust & Safety
- —Open source (public GitHub repo)
- —AI static analysis passed
- —License: Apache-2.0
- —Manually verified by team
Related Skills
Scrapy
High-throughput crawling and scraping for agent data pipelines
61.8K stars · 0 installsScrapling
Adaptive web scraping for agent data collection
52.7K stars · 0 installsFirecrawl
Turn any website into LLM-ready markdown or structured data
122.8K stars · 25.0K installsBrowser Use
Make AI agents interact with websites using natural language
95.0K stars · 28.0K installs