AI Agent Skills

Open-source LLM-friendly web crawler and scraper for agent workflows.

Turn websites into clean markdown or structured data for retrieval and agents.

Crawl a website starting from a URL, find relevant pages, and extract data – all guided by your natural language prompt.

Crawl and extract (regular or onion) webpages through TOR network

Flexible Node.js AI-assisted crawler library

AI crawler access analysis. Checks robots.txt, meta tags, and HTTP headers to determine which AI crawlers can access the site. Provides a complete access map and recomme…

Every web site provides APIs.

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you ca…

Jekyll-based static site for The Programming Historian

Automate a real browser from the terminal for navigation, form filling, snapshots, screenshots, extraction, and UI-flow debugging.

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

Forges reusable Skill packages (SKILL.md + scripts) from website exploration via browser-act — no re-exploration later. Use when: user wants a reusable Skill for any web…

scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more…

Full-site crawling, scraping, and site mapping via Firecrawl MCP. Use when user says "crawl site", "map site", "full crawl", "find all pages", "broken links", "site stru…