Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Development

night-market-diagnostics-toolkit

Run and interpret repo diagnostic scripts (ratchets, validators, token stats). Use when measuring health. Do not use to run tests; use night-market-operations.

athola · AI review on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add athola/claude-night-market --skill night-market-diagnostics-toolkit

About this skill


name: night-market-diagnostics-toolkit description: Run and interpret repo diagnostic scripts (ratchets, validators, token stats). Use when measuring health. Do not use to run tests; use night-market-operations.

Night Market Diagnostics Toolkit

Measure instead of eyeballing. Every claim about the health of this repo ("the skill graph is clean", "descriptions fit the budget", "no compromised dependencies") has a script that produces a number or a PASS/FAIL. This skill catalogs those scripts, their exact invocations, and how to read their output.

Conventions for everything below:

  • Run all commands from the repo root (/home/alext/claude-night-market).
  • Plain python3 is enough for every script in the table except mutation testing, which needs uv.
  • All checks here are read-only. None of them mutate the repo.

The ratchet-baseline concept

Several checks are ratchets. A ratchet check counts a category of existing debt (for example, SKILL.md files missing an Exit Criteria section), compares the live count against a frozen number in a baseline JSON file, and:

  • passes while the live count is at or below the baseline (pre-existing debt is tolerated),
  • fails the moment a NEW violation pushes the count above the baseline (debt may not grow),
  • nudges you to lower the baseline number when the live count drops (locking in the win so the debt cannot silently return).

The goal is a baseline that only shrinks, eventually to zero. The two ratchet baselines live at scripts/skill_graph_baseline.json (keys max_dangling_bugs, max_uncalled_libraries) and scripts/skill_exit_criteria_baseline.json (key max_missing_exit_criteria). When a ratchet prints a "dropped to N (baseline M). Lower ... to lock the win." line, edit the baseline JSON down to N in the same PR.

Tool table

Jargon used in the table: "dangling Skill() ref" means a Skill(plugin:name) reference in a skill/command/agent file whose target skill does not exist. "Uncalled library" means a library-role skill no other skill invokes (the risk targeted by .claude/rules/shared-utility-consumer-rule.md).

ToolInvocationWhat it measuresHow to interpretWhen to run
Plugin structure validatorpython3 plugins/abstract/scripts/validate_plugin.py plugins/<name>plugin.json validity, kebab-case naming, directory layoutExit 0 with "Plugin validation passed". Any FAIL line names the broken fileBefore committing plugin changes. make validate-all loops it over every plugin
Skill-graph drift ratchetpython3 scripts/check_skill_graph_drift.pyDangling Skill() refs and uncalled libraries vs skill_graph_baseline.jsonExit 0 at/below baseline. Nonzero exit lists the new dangling refs. "Lower ... to lock the win" means shrink the baselineAfter adding, renaming, or deleting skills or Skill() refs
Exit-criteria drift ratchetpython3 scripts/check_skill_exit_criteria_drift.pySKILL.md files under plugins/ missing an ## Exit Criteria heading vs skill_exit_criteria_baseline.jsonSame ratchet semantics as aboveAfter adding or editing any SKILL.md
Pinned-version checkerpython3 scripts/check_pinned_versions.pyGitHub-sourced tool pins vs the latest upstream release (needs network)"N pin(s) current" is healthy. A "holding X at ..." line documents an intentional hold with its reason (for example bandit 1.8.6 for the Python 3.9 hook floor)In pre-commit, and whenever a CI tool-setup step breaks
Capabilities syncbash scripts/capabilities-sync-check.shplugin.json registrations vs the generated capabilities reference in book/src/"PASSED: All capabilities are in sync" plus counts. On drift, run /sanctum:sync-capabilities --fixAfter changing any skill, command, or agent registration
Supply-chain scanpython3 scripts/supply_chain_scan.pyLockfiles vs the known-compromised-versions blocklist, plus known malicious artifactsTwo [OK] lines is clean. Any hit must be resolved before releaseWhen adding dependencies, during incidents, before releases
Token statspython3 plugins/abstract/scripts/context_optimizer.py stats plugins/ --format jsonBytes and estimated tokens per SKILL.md, bucketed small/medium/largeSkills over ~5,000 estimated tokens breach the quality-gates file limit and are split candidatesWhen a skill feels bloated, before modularizing
Description budgetpython3 plugins/abstract/scripts/validate_budget.pySum of all skill/command description characters vs the 90,000-char ecosystem ceiling (ADR-0004, 160 chars per description)Prints used/ceiling and headroom. Exit 0 means within budgetAfter editing any frontmatter description
skrills validateskrills validate --skill-dir plugins --target claudeSkill frontmatter validity per target framework (Claude, Codex, Copilot)"Validated N skills: ..." then an Errors (n): list with file:line. Fix every listed errorSkill audits. make validate-skills falls back to uv run python scripts/check_plugin_hooks.py when skrills is absent
skrills analyzeskrills analyze --skill-dir pluginsSkill token usage and dependency structureLarge-token outliers are split candidatesBudget planning. make analyze-skills falls back to scripts/generate_dependency_map.py
Mutation testingcd plugins/<name> && uv run mutmut run --paths-to-mutate=scripts/,src/ --tests-dir=tests/Whether the test suite kills injected code mutations (surviving mutants = untested behavior)Exit 0 = no survivors. Exit 2 = survivors found (CI treats this as pass-with-report). Any other exit = crash, investigateWeekly CI (Sunday) or manually before hardening a test suite
Markdown link checkerpython3 scripts/check-markdown-links.py [file.md ...]Broken relative links and anchors. No args = scan the whole repoExit 1 prints each broken link as `file: l
Read the complete source ↗