跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
安全

corrections-audit

Use to analyze correction trends, surface recurring patterns, and graduate repeat corrections to guardrails or anti-patterns.

haabe · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add haabe/mycelium --skill corrections-audit

技能说明


name: corrections-audit description: "Use to analyze correction trends, surface recurring patterns, and graduate repeat corrections to guardrails or anti-patterns." metadata: instruction_budget: "40" framework_dependency: "mycelium" framework_dependency_note: "This skill is designed to run within the Mycelium framework (https://github.com/haabe/mycelium). Standalone use will skip the canvas state, theory gates, and harness behavior the skill assumes. Install: /plugin install mycelium@haabe-mycelium."

Corrections Audit Skill

Analyze corrections.md for trends, recurring patterns, and actionable insights.

Attribution: run the script, do not count by hand

python3 "${CLAUDE_PLUGIN_ROOT}/scripts/check_correction_attribution.py" --root . --snapshot

Added 2026-08-03. This audit's headline number — who caught the mistake — was computed by hand and had been produced exactly twice, six weeks apart, each time because a human remembered to look. It is the only number that answers the question the correction loop exists to answer: more harness, or more context.

The script reports the escape rate and its denominator, every time. At the time of writing 14 of 72 entries carry a catcher, so the rate covers 19% of the corpus — quoting it without that is a claim about the whole wearing borrowed clothes, which is the failure this repo spent 2026-08-02/03 removing from its own checks. If it prints NO RATE AVAILABLE, that is not 0%: it means nothing in the corpus says who caught anything.

--snapshot appends the reading to .claude/evals/metrics/corrections/<date>.json, the same layout the other metric adapters use. The level is not the signal. Trend caught_by_hook_or_check rising in absolute terms; the ratio can also be improved by logging fewer user-caught mistakes, which is why this script never gates on it.

Attribution is a HARD RULE at write time, not a suggestion — see engine/agent-operating-contract.md. It was advisory until 2026-08-03 and 72 of 100 entries carry no catcher as a result. Do not backfill those in bulk: who caught a mistake six weeks ago is not recoverable by inference, and a guessed catcher corrupts the only number this loop produces.

For entries written before the rule, add one phrase per new entry — caught by user, caught by hook, caught by review, self-caught. The parser reads the prose forms already present rather than requiring a new field, so nothing existing needs rewriting.

When to Use

  • Loop 2 (Incremental) cadence: after every 3+ corrections are logged
  • When the same correction category appears 3+ times
  • During /mycelium:diamond-assess if corrections gate has findings
  • Before starting a new diamond at the same scale as a previously corrected one

Workflow

  1. Load corrections AND warnings AND clusters: Read .claude/memory/corrections.md, .claude/memory/warnings-log.md, AND .claude/memory/cluster-instances.md (the cluster log graduated 2026-05-08 — canonical record of recurring-pattern instances and their graduation status; without it, "the cluster has graduated N times" has no auditable backing).

    • If corrections + warnings both empty AND no clusters logged: report "No corrections, warnings, or clusters to audit" and stop
    • Treat all three as inputs to the same pattern analysis. Corrections capture agent-introduced failures; warnings capture framework-state debt; cluster-instances capture cross-cluster pattern accumulation with explicit graduation criteria. Same recurring-pattern shape, different vantage points.
  2. Categorize by frequency:

    • Group corrections by Category (bias, security, engineering, process, communication)
    • Group by Scope (discovery, delivery, orchestration, quality)
    • Count occurrences per group
  3. Detect recurring patterns:

    • Same category appears 3+ times -> candidate for guardrail graduation
    • Same scope appears 3+ times -> candidate for domain-level CLAUDE.md update
    • Same mistake repeats after prevention was documented -> prevention strategy failed, needs escalation
  4. Check origin distribution (APEX alignment):

    • Count corrections by Origin (ai-generated, human-written, ai-assisted)
    • If ai-generated corrections dominate (>60%): flag for prompt/context improvement, BUT see detection_origin cross-check below before acting on this interpretation
    • If human-written corrections dominate (>60%): flag for process/training improvement
    • If ai-assisted is high: check if the AI contribution or the human contribution caused the issue

4b. Cross-check with detection_origin (when field is present — see .claude/memory/README.md):

  • Count corrections by Detection_origin if present (user / agent_self / hook / evaluator / eval_runner / external_review)
  • Critical disambiguation: if Origin is heavily ai-generated AND detection_origin is heavily user, the apparent AI-quality signal is actually a HARNESS-DETECTION GAP. The AI is generating failures and the user is the only entity catching them. The right intervention is more harness checks (hooks, evaluators), NOT more AI context.
  • If detection_origin is dominantly user (>70%): flag for harness-detection gap. Suggest where new hooks or evaluators could catch the failure modes earlier.
  • If detection_origin is well-distributed across mechanisms: harness coverage is healthy; trust the Origin signal at face value.
  • Surfaced 2026-05-03 (mycelium-roadmap dogfood): without this cross-check, the audit's "100% ai-generated → improve prompt context" framing would have driven the wrong intervention. Real signal was "AI generates, user catches" — fixed by shipping the framework-guard hook (harness-detection layer), not by improving prompts.
  1. Root-cause recurring corrections (5 Whys): For each correction that appears 3+ times, apply 5 Whys to find the systemic root:
    • Why did this happen? -> Why did that happen? -> .
阅读完整来源 ↗