Awesome Web Scraping
lorien
List of libraries, tools and APIs for web scraping and data processing.
OPENAGENTSKILL / DIRECTORY
Find a skill for your next task. Explore tools for Codex, Claude Code, Cursor and more.
Find a skill for your next task. Preview examples where available.
1–16 / 112
Results: 112
lorien
List of libraries, tools and APIs for web scraping and data processing.
robotframework
Generic automation framework for acceptance testing and RPA
microsoft
A framework for building, orchestrating and deploying AI agents and multi-agent workflows with support for Python and .NET.
scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
gocolly
Elegant Scraper and Crawler Framework for Golang
Mobile Security Framework (MobSF) is an automated, all-in-one mobile application (Android/iOS/Windows) pen-testing, malware analysis and security assessment framework ca…
unclecode
Web crawling built for AI
ohmyzsh
🙃 A delightful community-driven (with 2,500+ contributors) framework for managing your zsh configuration. Includes 300+ optional plugins (rails, git, macOS, hub, docker…
D4Vinci
Adaptive web scraping for agent data collection
pathwaycom
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
crewAIInc
Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling com…
refinedev
A React Framework for building internal tools, admin panels, dashboards & B2B apps with unmatched flexibility.
microsoft
A programming framework for agentic AI
RasaHQ
💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbo…
camel-ai
🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org
projectdiscovery
A next-generation crawling and spidering framework.