法律与合规
agent-audit-logging
Implement comprehensive audit logging and reporting for multi-agent systems. Covers event capture, structured logging, traceability, compliance reporting, forensic analysis, and real-time monitoring dashboards for agent actions and decisions.
cosmicstack-labs · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add cosmicstack-labs/mercury-agent-skills --skill agent-audit-logging技能说明
name: agent-audit-logging description: 'Implement comprehensive audit logging and reporting for multi-agent systems. Covers event capture, structured logging, traceability, compliance reporting, forensic analysis, and real-time monitoring dashboards for agent actions and decisions.' metadata: author: cosmicstack-labs version: 1.0.0 category: ai-ml tags: - audit-logging - compliance - observability - forensics - agent-tracing - reporting - governance
Agent Audit Log Reporting
Overview
When agents make decisions, take actions, and spend money, every step must be traceable. Audit logs answer questions like: "What did the agent do?", "Why did it do that?", "Who asked for it?", and "Can we prove it followed the rules?" This skill covers event sourcing, structured logging, traceability chains, compliance reporting, and forensic analysis for production multi-agent systems.
Core Concepts
Why Audit Logging Matters
| Need | Without Audit | With Audit |
|---|---|---|
| Debugging | "The agent did something wrong, but what?" | Full replay of decisions |
| Compliance | No evidence of rule following | Verifiable compliance trail |
| Billing | "Why did we spend $5K today?" | Per-task cost attribution |
| Security | Can't detect injection or abuse | Pattern detection on logs |
| Improvement | Guess what went wrong | Data-driven optimization |
| Accountability | "Was this the agent or the user?" | Clear provenance |
What to Log
| Event | Details | Priority |
|---|---|---|
| Invocation | Task received, agent, timestamp | Required |
| Reasoning | Agent's chain-of-thought | Required |
| Tool Calls | Tool name, params, result, latency | Required |
| Decisions | Branch taken, confidence, rationale | Required |
| LLM Response | Raw model output | High |
| Errors | Error type, stack trace, recovery action | Required |
| Handoffs | Source, target, context summary | Required |
| Human Interventions | Override, confirmation, escalation | Required |
| Token Usage | Prompt/completion counts | High |
| User Feedback | Rating, correction, follow-up | Medium |
Step-by-Step Implementation
Step 1: Define the Audit Event Schema
from dataclasses import dataclass, field, asdict
from typing import Any, Optional
from enum import Enum
import json
import time
import uuid
class EventType(Enum):
INVOCATION = "agent.invocation"
REASONING = "agent.reasoning"
TOOL_CALL = "agent.tool_call"
TOOL_RESULT = "agent.tool_result"
DECISION = "agent.decision"
LLM_RESPONSE = "agent.llm_response"
ERROR = "agent.error"
HANDOFF = "agent.handoff"
HUMAN_INTERVENTION = "agent.human_intervention"
TOKEN_USAGE = "agent.token_usage"
@dataclass
class AuditEvent:
"""Structured audit event for any agent action."""
# Identity
event_id: str = None
event_type: EventType = None
agent_name: str = ""
task_id: str = ""
session_id: str = ""
# What happened
action: str = ""
params: dict = field(default_factory=dict)
result: Any = None
# Context
reasoning: str = ""
confidence: float = 0.0
source: str = "" # User, system, or parent agent
# Traceability
parent_event_id: Optional[str] = None
trace_id: str = ""
# Metadata
timestamp: float = None
duration_ms: float = 0.0
token_count: int = 0
model: str = ""
version: str = ""
# Error
error: Optional[str] = None
error_type: Optional[str] = None
def __post_init__(self):
if self.event_id is None:
self.event_id = str(uuid.uuid4())
if self.timestamp is None:
self.timestamp = time.time()
if not self.trace_id:
self.trace_id = self.event_id
def serialize(self) -> dict:
"""Serialize to dictionary for storage."""
data = asdict(self)
data["event_type"] = self.event_type.value
data["timestamp"] = self.timestamp
return data
Step 2: Build the Audit Logger
class AuditLogger:
"""Structured audit logger with multiple backends."""
def __init__(self, storage_backend, buffer_size: int = 100):
self.storage = storage_backend
self.buffer = []
self.buffer_size = buffer_size
self._lock = threading.Lock()
def log(self, event: AuditEvent):
"""Log an audit event (buffered for performance)."""
with self._lock:
self.buffer.append(event)
if len(self.buffer) >= self.buffer_size:
self.flush()
def flush(self):
"""Flush buffered events to storage."""
with self._lock:
if not self.buffer:
return
events = self.buffer.copy()
self.buffer.clear()
# Write batch
asyncio.create_task(
self.storage.batch_write([
e.serialize() for e in events
])
)
async def log_invocation(self, agent_name: str, task: str,
session_id: str, trace_id: str = None):
"""Log an agent invocation."""
self.log(AuditEvent(
event_type=EventType.INVOCATION,
agent_name=agent_name,
action="invocation",
params={"task": task},
session_id=session_id,
trace_id=trace_id or str(uuid.uuid4()),
source="user"
))
async def log_tool_call(self, agent_name: str, tool_name: str,
params: dict, trace_id: str,
parent_id: str = None):
"""Log a tool call."""
self.log(AuditEvent(
event_type=EventType.TOOL_CALL,
agent_name=agent_name,
action=f"tool_call:{tool_name}",
params=params,
阅读完整来源 ↗