跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
其他技能

observability

Structured logging, distributed tracing, and alerting for AI systems and traditional services. You can't fix what you can't see.

DevelopersGlobal · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add DevelopersGlobal/ai-agent-skills --skill observability

技能说明


name: observability description: Structured logging, distributed tracing, and alerting for AI systems and traditional services. You can't fix what you can't see. category: harden applies-to: [claude, gemini, cursor, copilot, any] version: 1.0.0

Overview

Observability is the ability to understand the internal state of a system from its external outputs. For AI systems this is especially critical: agents make decisions that are hard to interpret without detailed telemetry.

The three pillars: Logs (what happened), Traces (how long and where), Metrics (aggregate health).

When to Use

  • Before deploying any new service to production
  • When adding AI agent capabilities to an existing system
  • When debugging production issues
  • When designing multi-agent pipelines

Process

Step 1: Structured Logging

  1. All logs must be structured (JSON, not free text). Fields: timestamp, level, service, traceId, message, context.
  2. Log levels used correctly:
    • ERROR: Something failed that requires immediate attention
    • WARN: Something unexpected happened but the system recovered
    • INFO: Normal significant events (requests received, jobs completed)
    • DEBUG: Detailed diagnostic information (off in production by default)
  3. Never log secrets, PII, or auth tokens.
  4. For AI systems, log: prompt inputs (sanitized), model outputs, token counts, latency, model version.

Verify: Logs are structured JSON. No secrets in logs. AI interactions logged.

Step 2: Distributed Tracing

  1. Every request gets a unique traceId generated at the entry point.
  2. traceId is propagated through all downstream calls (HTTP headers, message queues, agent calls).
  3. Each service/agent creates a span for its work, with: start time, end time, parent span ID.
  4. Use OpenTelemetry as the standard instrumentation library.

Verify: You can trace a single request across all services/agents in a single view.

Step 3: Metrics

  1. Define and track key metrics:
    • RED metrics: Rate (requests/sec), Errors (error rate %), Duration (latency p50/p95/p99)
    • AI-specific: Token usage, prompt cost, model latency, hallucination rate, retrieval precision
  2. Dashboards: one dashboard per service with RED metrics, one dashboard for AI system health.

Verify: RED metrics are tracked for every service. AI-specific metrics tracked for AI systems.

Step 4: Alerting

  1. Alerts must be actionable — every alert should have a runbook.
  2. Alert on symptoms (high error rate, high latency), not just causes.
  3. AI-specific alerts: token budget exceeded, model error rate spike, retrieval failure rate spike.
  4. On-call rotation: someone is responsible for every alert at all times.

Verify: Every alert has a runbook. On-call rotation defined.

Common Rationalizations (and Rebuttals)

ExcuseRebuttal
"We'll add monitoring after launch"You'll be fighting fires blind. Add it before.
"Console.log is enough"In production, console.log is noise. Structured logs with context are signals.
"The AI model handles it internally"Model internals are a black box. You must observe the inputs and outputs.

Verification

  • Structured JSON logging on all services
  • No secrets in logs
  • Distributed tracing with trace ID propagation
  • RED metrics tracked for all services
  • AI-specific metrics tracked (tokens, cost, latency)
  • Alerts configured with runbooks

References

  • production-deployment skill
  • multi-agent-orchestration skill
  • OpenTelemetry documentation