跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
浏览器与自动化

agent-ready-ai-rules

Sub-skill de agent-ready-cloudflare: Implement AI Bot Rules in robots.txt

fabricioctelles · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add fabricioctelles/skills --skill agent-ready-ai-rules

技能说明


name: agent-ready-ai-rules description: > Sub-skill de agent-ready-cloudflare: Implement AI Bot Rules in robots.txt

Implement AI Bot Rules in robots.txt

Add explicit User-agent entries for AI crawlers in your robots.txt per RFC 9309.

Requirements

  • Add User-agent blocks for AI-specific bots: GPTBot, OAI-SearchBot, Claude-Web, Google-Extended, Amazonbot, anthropic-ai, Bytespider, CCBot, Applebot-Extended
  • Set Allow and/or Disallow rules matching your content policy
  • A wildcard User-agent: * block alone is not sufficient — explicit AI bot entries are required

Cloudflare

AI Crawl Control lets you manage AI bot rules from the dashboard without editing robots.txt manually.

Validate

POST https://isitagentready.com/api/scan
Content-Type: application/json

{"url": "https://YOUR-SITE.com"}

Check that checks.botAccessControl.robotsTxtAiRules.status is "pass".