Collect structured data
Find skills for crawling websites, extracting structured data, monitoring pages, and turning messy web content into agent-ready inputs.
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
Excellent quality, 9.1K stars, and a 36 use-case fit score.
$ npx skills add apify/crawlee-python🔥 Search, scrape, and clean the web for AI agents.
Excellent quality, 123K stars, and a 34 use-case fit score.
$ npx skills add firecrawl/firecrawlOpen-source LLM-friendly web crawler and scraper
Excellent quality, 66K stars, and a 31 use-case fit score.
$ npx skills add unclecode/crawl4aiTransform Web Content into LLM-Ready Data
Excellent quality, 1.8K stars, and a 30 use-case fit score.
$ npx skills add watercrawl/WaterCrawl👾 Fast and simple video download library and CLI tool written in Go
Excellent quality, 31K stars, and a 30 use-case fit score.
$ npx skills add iawia002/luxElegant Scraper and Crawler Framework for Golang
Excellent quality, 25K stars, and a 30 use-case fit score.
$ npx skills add gocolly/collyA Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.
Strong quality, 2.8K stars, and a 29 use-case fit score.
$ npx skills add NikolaiT/GoogleScraperAdaptive web scraping for agent data collection
Excellent quality, 53K stars, and a 29 use-case fit score.
$ npx skills add D4Vinci/Scraplingnewspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
Excellent quality, 15K stars, and a 29 use-case fit score.
$ npx skills add codelucas/newspaper🤖 Scrape data from HTML websites automatically by just providing examples
Promising quality, 1.4K stars, and a 29 use-case fit score.
$ npx skills add lorey/mlscraper🤖 AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.
Strong quality, 641 stars, and a 28 use-case fit score.
$ npx skills add ScrapeGraphAI/scrapecraftscrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
Excellent quality, 4.1K stars, and a 28 use-case fit score.
$ npx skills add gosom/google-maps-scraperA collection of awesome web crawler,spider in different languages
Strong quality, 7.2K stars, and a 27 use-case fit score.
$ npx skills add BruceDone/awesome-crawlerFree Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.
Excellent quality, 3.0K stars, and a 27 use-case fit score.
$ npx skills add oxylabs/amazon-scraperDistributed crawler powered by Headless Chrome
Strong quality, 5.6K stars, and a 27 use-case fit score.
$ npx skills add yujiosaka/headless-chrome-crawler:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
Excellent quality, 2.7K stars, and a 27 use-case fit score.
$ npx skills add jae-jae/QueryListA visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/网页爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。
Excellent quality, 44K stars, and a 27 use-case fit score.
$ npx skills add NaiboWang/EasySpiderNode.js scraper to get data from Google Play
Excellent quality, 2.9K stars, and a 27 use-case fit score.
$ npx skills add facundoolano/google-play-scraperDeclarative web scraping
Excellent quality, 6.0K stars, and a 27 use-case fit score.
$ npx skills add MontFerret/ferretMovie metadata scraper
Strong quality, 3.6K stars, and a 27 use-case fit score.
$ npx skills add sqzw-x/mdcxStructured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM agents with fresh data. AI Studio python SDK for intelligent web data gathering.
Excellent quality, 2.9K stars, and a 27 use-case fit score.
$ npx skills add oxylabs/oxylabs-ai-studio-pyTo extract main article from given URL with Node.js
Excellent quality, 1.9K stars, and a 27 use-case fit score.
$ npx skills add extractus/article-extractorNewPipe's core library for extracting data from streaming sites
Excellent quality, 1.9K stars, and a 27 use-case fit score.
$ npx skills add TeamNewPipe/NewPipeExtractorWebsite Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.
Excellent quality, 2.1K stars, and a 27 use-case fit score.
$ npx skills add goclone-dev/gocloneLearn step-by-step how to scrape Google Trends data and make a result comparison using Python and Oxylabs SERP API. Extract keywords, their popularity, breakdown by region, related queries, and more.
Excellent quality, 2.6K stars, and a 27 use-case fit score.
$ npx skills add oxylabs/how-to-scrape-google-trendsThe headless Chrome/Chromium driver on top of Puppeteer. Take screenshots, generate PDFs, extract text and HTML with a production-ready API.
Excellent quality, 1.8K stars, and a 27 use-case fit score.
$ npx skills add microlinkhq/browserlessThe process of extracting product data from Amazon using Python, including titles, ratings, prices, images, and descriptions.
Strong quality, 2.9K stars, and a 27 use-case fit score.
$ npx skills add oxylabs/how-to-scrape-amazon-product-dataPython binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.
Excellent quality, 1.6K stars, and a 27 use-case fit score.
$ npx skills add rushter/selectolaxGive your AI the power to browse, scrape, and extract structured data from complex websites — with faster execution, lower cost, and more reliable results.
Excellent quality, 1.4K stars, and a 26 use-case fit score.
$ npx skills add browser-act/skillsAgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracting data quickly, precisely, and at scale. Includes REST API, Python and JavaScript SDKs, browser debugger.
Excellent quality, 1.4K stars, and a 26 use-case fit score.
$ npx skills add tinyfish-io/agentql