Browsertrix Crawler
webrecorder
Run a high-fidelity browser-based web archiving crawler in a single Docker container
1.1KGitHub
OPENAGENTSKILL / DIRECTORY
AI Agent Skills
Find a skill for your next task. Explore tools for Codex, Claude Code, Cursor and more.
Find a skill for your next task. Preview examples where available.
Results · “scrapy-crawler”
1–16 / 21
Results: 21
Crawlergo
Qianlitp
A powerful browser crawler for web vulnerability scanners
3KGitHub
Headless Chrome Crawler
yujiosaka
Distributed crawler powered by Headless Chrome
5.6KGitHub
Crawler
crwlrsoft
Library for Rapid (Web) Crawler and Scraper Development
368GitHub
Scrapy Playwright
scrapy-plugins
🎭 Playwright integration for Scrapy
1.4KGitHub
Oxylabs AI Studio Py
oxylabs
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM…
2.9KGitHub
Web Scraping
lkuffo
Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup
390GitHub
Crawley
s0rg
The unix-way web crawler
341GitHub
Email Crawler Lead Generator
superryeti
This email crawler will visit all pages of a provided website and parse and save emails found to a csv file.
182GitHub
Ache
VIDA-NYU
ACHE is a web crawler for domain-specific search.
483GitHub
Oxylabs AI Studio Js
oxylabs
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM…
143GitHub
X Crawl
coder-hxl
Flexible Node.js AI-assisted crawler library
1.9KGitHub
Wayback Machine Scraper
sangaline
A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
480GitHub
Fundus
flairNLP
A very simple news crawler with a funny name
461GitHub
Chromium For Spider
myvyang
dynamic crawler for web vulnerability scanner
251GitHub
Tracker Radar Collector
duckduckgo
🕸 Modular, multithreaded, puppeteer-based crawler
164GitHub