跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
安全

ai-foundry-posture-check

Assess security, reliability, and cost posture of Azure OpenAI and Microsoft Foundry deployments. Detects critical anti-patterns: API keys instead of Managed Identity, public endpoints without firewall, disabled content filtering, missing diagnostic settings, deprecated model versions, over-provisioned PTU, absence of AI Gateway (APIM) for rate limiting, shared dev/prod deployments, quota exhaustion risk, 429 throttling rates, runaway token consumption, and single-region deployment. Use when asked about AI workload security, OpenAI best practices, Foundry health check, model deployment review, AI cost optimization, quota monitoring, or throttling analysis.

ricmmartins · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add ricmmartins/azure-sre-agent-skills --skill ai-foundry-posture-check

技能说明


name: ai-foundry-posture-check description: > Assess security, reliability, and cost posture of Azure OpenAI and Microsoft Foundry deployments. Detects critical anti-patterns: API keys instead of Managed Identity, public endpoints without firewall, disabled content filtering, missing diagnostic settings, deprecated model versions, over-provisioned PTU, absence of AI Gateway (APIM) for rate limiting, shared dev/prod deployments, quota exhaustion risk, 429 throttling rates, runaway token consumption, and single-region deployment. Use when asked about AI workload security, OpenAI best practices, Foundry health check, model deployment review, AI cost optimization, quota monitoring, or throttling analysis. tools:

  • RunAzCliReadCommands
  • execute_kusto_query

AI Foundry & OpenAI Posture Check

Purpose

Assess the security, reliability, and cost efficiency of Azure OpenAI and Microsoft Foundry deployments. Detects the most common anti-patterns that startups make when building AI-powered products — from exposed endpoints to runaway token costs.

Based on the Azure Well-Architected Framework for AI workloads and Microsoft Foundry operational best practices.

When to use this skill

  • User asks "is our OpenAI deployment secure?"
  • User asks about AI cost optimization or token consumption
  • Review before going to production with an AI feature
  • User asks about content filtering, model versions, or rate limiting
  • Periodic AI workload health check

Pre-check

Confirm with the user:

  • Which Azure OpenAI / Cognitive Services accounts to assess (or "all in subscription")
  • Whether they use PTU (Provisioned Throughput) or Standard deployments
  • Whether they have production AI workloads already live

Assessment procedure

Step 0: Discover AI resources

az cognitiveservices account list \
  --subscription <sub-id> \
  --query "[?kind=='OpenAI' || kind=='AIServices'].{name:name, kind:kind, rg:resourceGroup, location:location, sku:sku.name}" \
  -o table

If no results, try:

az cognitiveservices account list \
  --subscription <sub-id> \
  --query "[].{name:name, kind:kind, rg:resourceGroup, location:location}" \
  -o table

If no Cognitive Services accounts exist, report "No Azure OpenAI or AI Foundry resources found" and end assessment.

For each account found, run the following checks:


🔐 CATEGORY 1 — Security (Critical)

Check 1.1 — Managed Identity enabled (not API keys only)

az cognitiveservices account show \
  --name <account> --resource-group <rg> \
  --query "{identity:identity.type, disableLocalAuth:properties.disableLocalAuth}" \
  -o json
FindingSeverityScore
identity.type = SystemAssigned/UserAssigned AND disableLocalAuth = true✅ Pass12 pts
identity.type set but disableLocalAuth = false⚠️ Partial6 pts
identity.type = None or null❌ Fail0 pts

📖 https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/managed-identity

Check 1.2 — Network isolation (firewall or private endpoint)

az cognitiveservices account show \
  --name <account> --resource-group <rg> \
  --query "{publicAccess:properties.publicNetworkAccess, defaultAction:properties.networkAcls.defaultAction, privateEndpoints:properties.privateEndpointConnections}" \
  -o json
FindingSeverityScore
publicNetworkAccess = Disabled + private endpoints exist✅ Pass13 pts
defaultAction = Deny + IP/VNet rules configured✅ Pass13 pts
defaultAction = Allow (open to internet)❌ Fail0 pts

📖 https://learn.microsoft.com/en-us/azure/ai-services/cognitive-services-virtual-networks

Check 1.3 — Content filtering (RAI policies)

az rest --method get \
  --url "https://management.azure.com/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<account>/raiPolicies?api-version=2024-10-01"

Then check deployments have a policy assigned:

az cognitiveservices account deployment list \
  --name <account> --resource-group <rg> \
  --query "[].{name:name, model:properties.model.name, raiPolicy:properties.raiPolicyName}" \
  -o table
FindingSeverityScore
All deployments have RAI policy assigned with filters enabled✅ Pass10 pts
Some deployments missing policy or using minimal filters⚠️ Partial5 pts
No RAI policies or content filtering disabled❌ Fail0 pts

📖 https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/content-filters

⚙️ CATEGORY 2 — Reliability & Operations

Check 2.1 — Model version pinned and not deprecated

az cognitiveservices account deployment list \
  --name <account> --resource-group <rg> \
  --query "[].{name:name, model:properties.model.name, version:properties.model.version}" \
  -o table

Cross-reference with available models:

az cognitiveservices account list-models \
  --name <account> --resource-group <rg> \
  --query "[].{model:model.name, version:model.version, lifecycle:model.lifecycleStatus}" \
  -o table
FindingSeverityScore
All deployments on GA versions with explicit version pinned✅ Pass7 pts
Deployments on versions nearing retirement (< 3 months)⚠️ Attention3 pts
Deployments on Legacy/Deprecated versions❌ Fail0 pts

📖 https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/model-retirements

Check 2.2 — Diagnostic settings enabled

# Get resource ID
RESOURCE_ID=$(az cognitiveservices account show \
  --name <account> --resource-group <rg> --query "id" -o tsv)

az monitor diagnostic-settings list --resource $RESOURCE_ID -o json
FindingSeverityScore
Diagnostic settings sending RequestResponse + Audit to Log Analytics✅ Pass
阅读完整来源 ↗