night-market-completion-integrity-campaign
Bind loop 'done' to unfakeable gates. Use to harden egregore/herald loops or promote completion_integrity. Not for QA gates; use night-market-validation-and-qa.
athola · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add athola/claude-night-market --skill night-market-completion-integrity-campaign技能说明
name: night-market-completion-integrity-campaign description: Bind loop 'done' to unfakeable gates. Use to harden egregore/herald loops or promote completion_integrity. Not for QA gates; use night-market-validation-and-qa.
Night Market Completion-Integrity Campaign
This is an executable campaign, phased and decision-gated, against the
hardest live problem in this repository: making "done" in autonomous
loops mean something the agent cannot fake, and earning the promotion
of completion_integrity from default-off to default-on. Every command
below was run against the repo on 2026-07-02 (v1.9.15) unless marked
candidate. Run the phases in order. Each phase ends at a gate with
expected observations and branch instructions.
Definitions
| Term | Meaning |
|---|---|
| Autonomous loop | A session that keeps working without a human turn: egregore's orchestrator (/egregore:summon), the externally installed ralph-wiggum loop, herald's auto-continue Stop hook |
| Stop hook | A Claude Code hook fired when the agent tries to end its turn. It prints {"decision": "approve"} (allow stop) or {"decision": "block", "reason": ...} (keep working) |
| Completion integrity | The property that a loop's "done" signal is bound to a verifier the agent does not control, instead of the agent's own say-so |
| False stop | The loop halts while verifiable work remains, or accepts a claimed completion that a check would have rejected |
| False continue | The loop keeps working (blocks the stop) on a turn that was genuinely finished |
| Verdict | The egregore quality-gate outcome: pass, pass-with-warnings, or fix-required (computed after a 3-attempt auto-fix loop) |
Problem statement
Three assets exist today. None of them, alone, binds "done" to an unfakeable gate:
| Asset | What it does | Default | Key commits |
|---|---|---|---|
plugins/egregore/scripts/config.py field PipelineConfig.completion_integrity | When true, a fix-required verdict counts as a step failure (the item cannot reach completed with blocking findings) and merge is held for human review regardless of auto_merge | False | 83281337 (gate), cd903cbf (raw-JSON opt-in test) |
plugins/herald/hooks/double_shot_latte.py (Stop hook) | Deterministic continue/stop judge over the last assistant message, with an opt-in LLM second shot for the single ambiguous outcome | Deterministic only | 268cff89 (timeout cap + gating) |
plugins/imbue/skills/proof-of-work/modules/verifier-integrity.md | The theory: a gate earns trust only if the spec is validated separately from the code and the check is proven able to fail | Prose guidance | 29081fda |
The load-bearing weakness, verified by reading commit 83281337: the
completion_integrity flag is real code with a tested load path, but
its enforcement lives in agent instructions
(plugins/egregore/agents/orchestrator.md, the egregore:quality-gate
skill, and skills/summon/modules/pipeline.md). No Python code path
blocks a pipeline transition. The manifest that records item status
(.egregore/manifest.json) is written by the same agent the gate is
supposed to bind. The campaign exists to close that gap with evidence.
Evidence bar
Success is measured, never judged by eye. The standing rules are
drawn from two 2026-07-01 research passes, whose evidence now lives
in .claude/rules/prefer-invariants-over-fallbacks.md (harness-loop
findings) and
plugins/imbue/skills/proof-of-work/modules/verifier-integrity.md
(verifier findings), plus the load-bearing claims inlined below:
- State the numbers a hypothesis predicts BEFORE running the measurement. A threshold chosen after seeing the data is not a gate.
- One mechanism must explain all observations, including the negative ones.
- Never let the generator be its own judge: LLM self-verification is
measurably unreliable and self-critique can degrade output, so
verdicts come from an independent verifier (prover-verifier
separation, arXiv 2402.08115; see
plugins/imbue/skills/proof-of-work/modules/verifier-integrity.md). - A green check proves spec satisfaction, never correctness. Every gate you add must itself be proven able to go red (verifier-integrity Guard 2: mutation or revert test).
Campaign map
| Phase | Question it answers | Gate to pass |
|---|---|---|
| P0 | Do today's gate and judge suites pass as documented? | Exact pass counts reproduced |
| P1 | What does the gate log when enabled on real work? | Opt-in path exercised on a bounded item, logs captured |
| P2 | What are the false-stop and false-continue rates? | Pre-registered numbers met |
| P3 | Can the loop satisfy the gate without doing the work? | Refutation attempts run, holes documented or closed |
| P4 | Is the default flip earned? | Change control passed |
P0: baseline the gate and judge suites
Run every suite from inside its plugin directory. Root pytest sets
norecursedirs = plugins/*, so running from the repo root silently
collects nothing (or raises ImportPathMismatchError). Every cd
in this skill is relative to the repo root: start each command block
from there.
cd plugins/egregore
uv run pytest tests/ -q
Expected (2026-07-02): 477 passed in about 2 seconds, preceded by a
coverage table.
cd plugins/egregore
uv run pytest tests/test_config.py tests/test_quality_gate.py -q
Expected: 27 passed. tests/test_config.py alone is 13 passed and
includes the two completion-integrity guards:
test_completion_integrity_opt_in_roundtrip and
test_completion_integrity_loads_from_raw_json (the real user opt-in
path, added in cd903cbf. It also guards the field against silent
removal, because _filter_fields would drop the key).
cd plugins/herald
uv run pytest tests/ -q
Expected: 105 passed in under 2 seconds. This suite contains
test_llm_timeout_fits_within_hook_timeout, which asserts
`LLM_TIMEOUT_SE