media-crawler-douyin
tsingyuai
Collect Douyin competitive evidence with MediaCrawler using keyword search, exact video detail, comments, media, and creator profiles. Use for trend, hook, format, audie…
2KGitHub
OPENAGENTSKILL / DIRECTORY
AI Agent Skills
Find a skill for your next task. Explore tools for Codex, Claude Code, Cursor and more.
Find a skill for your next task. Preview examples where available.
Results · “scrapy-crawler”
49–64 / 85
Results: 85
media-crawler-kuaishou
tsingyuai
Collect Kuaishou competitive evidence with MediaCrawler using keyword search, exact video detail, comments, media, and creator profiles. Use for trend, format, audience-…
2KGitHub
media-crawler-tieba
tsingyuai
Collect Baidu Tieba thread and user evidence with MediaCrawler using keyword or bar discovery, exact thread detail, replies, and creator pages. Use for community pain-po…
2KGitHub
media-crawler-weibo
tsingyuai
Collect Weibo posts and creator evidence with MediaCrawler using search, exact post IDs, comments, optional media, and creator IDs. Use for discourse, trend, messaging,…
2KGitHub
media-crawler-zhihu
tsingyuai
Collect Zhihu answers, articles, videos, comments, and creator evidence with MediaCrawler through search and exact URLs. Use for expert discourse, problem framing, objec…
2KGitHub
QueryList
jae-jae
:spider: The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
2.7KGitHub
Web Scraping
lkuffo
Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup
390GitHub
Go Spider
hu17889
[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you ca…
1.8KGitHub
Crawley
s0rg
The unix-way web crawler
341GitHub
Email Crawler Lead Generator
superryeti
This email crawler will visit all pages of a provided website and parse and save emails found to a csv file.
182GitHub
Ache
VIDA-NYU
ACHE is a web crawler for domain-specific search.
483GitHub
Oxylabs AI Studio Js
oxylabs
Structured data gathering from any website using AI-powered scraper, crawler, and browser automation. Scraping and crawling with natural language prompts. Equip your LLM…
143GitHub
Krawl
BlessedRebuS
Krawl is a customizable, lightweight, cloud-native web deception server and anti-crawler that creates fake web applications with low-hanging vulnerabilities using realis…
546GitHub
Edit the theme's robots.txt.liquid to explicitly allow AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, OAI-SearchBot, Amazonbot) so AI assistants are per…
182GitHub
Freshonions Torscraper
dirtyfilthy
Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion
535GitHub
Scavenger
rndinfosecguy
Crawler (Bot) searching for credential leaks on paste sites.
810GitHub