Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Browser & automation

agent-ready-ai-rules

Sub-skill de agent-ready-cloudflare: Implement AI Bot Rules in robots.txt

fabricioctelles · Static checks on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add fabricioctelles/skills --skill agent-ready-ai-rules

About this skill


name: agent-ready-ai-rules description: > Sub-skill de agent-ready-cloudflare: Implement AI Bot Rules in robots.txt

Implement AI Bot Rules in robots.txt

Add explicit User-agent entries for AI crawlers in your robots.txt per RFC 9309.

Requirements

  • Add User-agent blocks for AI-specific bots: GPTBot, OAI-SearchBot, Claude-Web, Google-Extended, Amazonbot, anthropic-ai, Bytespider, CCBot, Applebot-Extended
  • Set Allow and/or Disallow rules matching your content policy
  • A wildcard User-agent: * block alone is not sufficient — explicit AI bot entries are required

Cloudflare

AI Crawl Control lets you manage AI bot rules from the dashboard without editing robots.txt manually.

Validate

POST https://isitagentready.com/api/scan
Content-Type: application/json

{"url": "https://YOUR-SITE.com"}

Check that checks.botAccessControl.robotsTxtAiRules.status is "pass".