corrections-audit
Use to analyze correction trends, surface recurring patterns, and graduate repeat corrections to guardrails or anti-patterns.
haabe · Static checks on file
Installation
Read the source and dependencies before running a command. Copying does not install or execute anything.
npx skills add haabe/mycelium --skill corrections-auditAbout this skill
name: corrections-audit description: "Use to analyze correction trends, surface recurring patterns, and graduate repeat corrections to guardrails or anti-patterns." metadata: instruction_budget: "40" framework_dependency: "mycelium" framework_dependency_note: "This skill is designed to run within the Mycelium framework (https://github.com/haabe/mycelium). Standalone use will skip the canvas state, theory gates, and harness behavior the skill assumes. Install: /plugin install mycelium@haabe-mycelium."
Corrections Audit Skill
Analyze corrections.md for trends, recurring patterns, and actionable insights.
Attribution: run the script, do not count by hand
python3 "${CLAUDE_PLUGIN_ROOT}/scripts/check_correction_attribution.py" --root . --snapshot
Added 2026-08-03. This audit's headline number — who caught the mistake — was computed by hand and had been produced exactly twice, six weeks apart, each time because a human remembered to look. It is the only number that answers the question the correction loop exists to answer: more harness, or more context.
The script reports the escape rate and its denominator, every time. At the
time of writing 14 of 72 entries carry a catcher, so the rate covers 19% of the
corpus — quoting it without that is a claim about the whole wearing borrowed
clothes, which is the failure this repo spent 2026-08-02/03 removing from its own
checks. If it prints NO RATE AVAILABLE, that is not 0%: it means nothing in the
corpus says who caught anything.
--snapshot appends the reading to .claude/evals/metrics/corrections/<date>.json, the same layout the other metric adapters use. The level is not the signal. Trend caught_by_hook_or_check rising in absolute terms; the ratio can also be improved by logging fewer user-caught mistakes, which is why this script never gates on it.
Attribution is a HARD RULE at write time, not a suggestion — see engine/agent-operating-contract.md. It was advisory until 2026-08-03 and 72 of 100 entries carry no catcher as a result. Do not backfill those in bulk: who caught a mistake six weeks ago is not recoverable by inference, and a guessed catcher corrupts the only number this loop produces.
For entries written before the rule, add one phrase per new entry — caught by user,
caught by hook, caught by review, self-caught. The parser reads the prose
forms already present rather than requiring a new field, so nothing existing
needs rewriting.
When to Use
- Loop 2 (Incremental) cadence: after every 3+ corrections are logged
- When the same correction category appears 3+ times
- During
/mycelium:diamond-assessif corrections gate has findings - Before starting a new diamond at the same scale as a previously corrected one
Workflow
-
Load corrections AND warnings AND clusters: Read
.claude/memory/corrections.md,.claude/memory/warnings-log.md, AND.claude/memory/cluster-instances.md(the cluster log graduated 2026-05-08 — canonical record of recurring-pattern instances and their graduation status; without it, "the cluster has graduated N times" has no auditable backing).- If corrections + warnings both empty AND no clusters logged: report "No corrections, warnings, or clusters to audit" and stop
- Treat all three as inputs to the same pattern analysis. Corrections capture agent-introduced failures; warnings capture framework-state debt; cluster-instances capture cross-cluster pattern accumulation with explicit graduation criteria. Same recurring-pattern shape, different vantage points.
-
Categorize by frequency:
- Group corrections by
Category(bias, security, engineering, process, communication) - Group by
Scope(discovery, delivery, orchestration, quality) - Count occurrences per group
- Group corrections by
-
Detect recurring patterns:
- Same category appears 3+ times -> candidate for guardrail graduation
- Same scope appears 3+ times -> candidate for domain-level CLAUDE.md update
- Same mistake repeats after prevention was documented -> prevention strategy failed, needs escalation
-
Check origin distribution (APEX alignment):
- Count corrections by
Origin(ai-generated, human-written, ai-assisted) - If ai-generated corrections dominate (>60%): flag for prompt/context improvement, BUT see
detection_origincross-check below before acting on this interpretation - If human-written corrections dominate (>60%): flag for process/training improvement
- If ai-assisted is high: check if the AI contribution or the human contribution caused the issue
- Count corrections by
4b. Cross-check with detection_origin (when field is present — see .claude/memory/README.md):
- Count corrections by
Detection_originif present (user / agent_self / hook / evaluator / eval_runner / external_review) - Critical disambiguation: if Origin is heavily ai-generated AND detection_origin is heavily
user, the apparent AI-quality signal is actually a HARNESS-DETECTION GAP. The AI is generating failures and the user is the only entity catching them. The right intervention is more harness checks (hooks, evaluators), NOT more AI context. - If detection_origin is dominantly
user(>70%): flag for harness-detection gap. Suggest where new hooks or evaluators could catch the failure modes earlier. - If detection_origin is well-distributed across mechanisms: harness coverage is healthy; trust the Origin signal at face value.
- Surfaced 2026-05-03 (mycelium-roadmap dogfood): without this cross-check, the audit's "100% ai-generated → improve prompt context" framing would have driven the wrong intervention. Real signal was "AI generates, user catches" — fixed by shipping the framework-guard hook (harness-detection layer), not by improving prompts.
- Root-cause recurring corrections (5 Whys):
For each correction that appears 3+ times, apply 5 Whys to find the systemic root:
- Why did this happen? -> Why did that happen? -> .