News Please alternatives for AI agents.

Python ProxyPool for web spider

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

Python API for JMComic | 提供Python API访问禁漫天堂，同时支持网页端和移动端 | 禁漫天堂GitHub Actions下载器🚀

NewPipe's core library for extracting data from streaming sites

Redis-based components for Scrapy.

Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by Elasticsearch

Elasticsearch File System Crawler (FS Crawler)

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

Crawl and extract (regular or onion) webpages through TOR network

Fess is very powerful and easily deployable Enterprise Search Server.

🚀🚀🚀feapder is an easy to use, powerful crawler framework | feapder是一款上手简单，功能强大的Python爬虫框架。内置AirSpider、Spider、TaskSpider、BatchSpider四种爬虫解决不同场景的需求。且支持断点续爬、监控报警、浏览器渲染、海量数据去重等功能。更有功能强大的爬虫管理系统feaplat为其提供方便的部署及调度

Every web site provides APIs.

A community-driven way to read and chat with AI bots - powered by chatGPT.

All In One Web Recon

Videodl: A lightweight video downloader written in pure python. (轻量级视频下载器，优先高清无水印，支持抖音，快手，小红书，B站，TikTok，YouTube，FIFA+，优酷，腾讯，爱奇艺，1905电影网，乐视，芒果，咪咕，PPTV，搜狐，Facebook，Twitter，新浪微博，今日头条，网易公开课，全民K歌，CCTV央视频，酷狗音乐MV，新片场，知乎，百度贴吧，TED等海量流媒体平台)

基于搜狗微信搜索的微信公众号爬虫接口