blue-team
Use when the user has concrete failing cases in code or a guardrail/classifier/filter/prompt/API they own — a red-team failure catalogue OR a CI/CD test-failure report (failing pytest/JUnit tests) — and wants the target patched until those failures are closed without breaking what already works. It points straight at the failed cases (normalize any source with tools/ingest.py), fixes one root-cause class per iteration, and re-checks with tools/verify.py — oracle mode against a red-team oracle, or tests mode against the test suite — keeping a patch only if it closes a class while nothing that passed before regresses, else reverting; loops until every class is closed (dry) or the budget runs out, then opens a pull request with the patch set. The defensive fixer half of a find→fix setup. Not for discovering new failures (that is red-team), and not for editing the oracle, tests, or holdout that define ground truth.
gaasher · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add gaasher/Agent-Loop-Skills --skill blue-team技能说明
name: blue-team description: > Use when the user has concrete failing cases in code or a guardrail/classifier/filter/prompt/API they own — a red-team failure catalogue OR a CI/CD test-failure report (failing pytest/JUnit tests) — and wants the target patched until those failures are closed without breaking what already works. It points straight at the failed cases (normalize any source with tools/ingest.py), fixes one root-cause class per iteration, and re-checks with tools/verify.py — oracle mode against a red-team oracle, or tests mode against the test suite — keeping a patch only if it closes a class while nothing that passed before regresses, else reverting; loops until every class is closed (dry) or the budget runs out, then opens a pull request with the patch set. The defensive fixer half of a find→fix setup. Not for discovering new failures (that is red-team), and not for editing the oracle, tests, or holdout that define ground truth. compatibility: Requires Python 3.9+; git + the gh CLI for the pull-request handoff (degrades to a patch series). metadata: version: "0.1.0"
Blue Team
A defensive fixer loop — the inverse of red-team. The artifact is the target, now writable;
the feedback signal is two-part, like optimize-loop: a gate that must hold (nothing that passed
before regresses) and a metric that must drop (the count of open failure classes, toward zero). You
point it at a set of concrete failed cases and fix them one root-cause class at a time. Each
iteration you patch one class, then run tools/verify.py, and keep the patch only if it closes the class
with no regression, else revert. You loop until every class is closed (dry) or the budget runs out,
then hand the patch set off as a pull request. This is the fix half of a find→fix setup (see
Pairing).
The failed cases come from a real source; tools/ingest.py normalizes any of them into one catalogue:
oraclemode — ared-teamfailures.jsonl: each case is an input where the target's verdict disagrees with a ground-truth oracle. A case is closed when target and oracle now agree; a regression is a benign<holdout>input that newly disagrees (most often a new over-block).testsmode — a CI/CD test-failure report (pytest --junitxml/ JUnit XML, or a list of failing node ids): each case is a failing test. A case is closed when its test now passes; a regression is any other test that was passing and now fails.
When to use
Use to fix a concrete set of failing cases in code or a guardrail/classifier/filter/prompt/API the user
owns — a red-team catalogue, or the failing tests from a CI run — driving the open-class count to zero
without breaking what worked. A class is the root-cause group the loop closes as a unit (a red-team
technique, or a CI failure area / test class).
Default: pick the mode that matches the source (oracle for red-team, tests for CI/CD). Escape hatch:
in oracle mode with no separate functional test suite, the <holdout> alone is the regression guard;
in tests mode the suite's own previously-passing tests are the guard. Not for finding new failures (run
red-team), and not for editing the ground truth (the oracle, the tests, or the holdout).
Setup
Resolve bindings interactively. If loop.run.yaml exists, load it, confirm the values in one line, and
skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is available) infer a likely
value per binding and recommend it; on other hosts ask each as a quoted prompt. Then write
loop.run.yaml and confirm before creating any other files. Two worked configs:
examples/run.example.yaml (oracle mode) and examples/tests.run.yaml (tests mode).
| binding | meaning | default | how to infer |
|---|---|---|---|
<source> | where the failed cases come from: oracle (red-team) or tests (CI/CD) | — | red-team failures.jsonl → oracle; failing pytest/JUnit → tests |
<target_files> | the file(s) the loop may edit to fix the target | — | the source/guardrail/classifier behind the failures |
<catalogue> | the failed cases to close, JSONL; build it with tools/ingest.py (see below) | <sandbox_root>/catalogue.jsonl | red-team's <failures_log>, or a JUnit report |
<oracle_cmd> | (oracle mode) ground-truth verdict (frozen), same stdin→verdict contract as red-team | — | a reference checker / policy impl |
<holdout> | (oracle mode) benign inputs that must keep passing (regression guard) | <sandbox_root>/holdout.jsonl | known-good inputs the oracle agrees on |
<test_cmd> | (tests mode) runs the suite and writes a JUnit XML; regressions read from it | — | pytest --junitxml=<junit> (or any runner that emits JUnit) |
<junit> | (tests mode) path to the JUnit XML <test_cmd> writes | <sandbox_root>/junit.xml | — |
<iter_strategy> | branches (one commit per kept fix → feeds the PR) or snapshots (folder per iter) | branches | dirty / non-git tree → snapshots |
<pr_branch> | branch the fixes land on and the PR opens from | blue-team/<tag> | today's date as <tag> |
<sandbox_root> | where snapshots + the ledger live | ./sandbox | — |
<budget> | max iterations | 8 | — |
<patience> | give up on one class after N failed attempts → mark it a residual | 3 | — |
<skill_dir> is this skill's installed folder; substitute the real path when writing loop.run.yaml.
Build the catalogue first with tools/ingest.py, which normalizes any source into {id, ..., class}:
python3 <skill_dir>/tools/ingest.py --from red-team --in <failures.jsonl> --out <catalogue> # oracle mode
python3 <skill_dir>/tools/ingest.py --from junit --in <report.xml> --out <catalogue> # tests mode
The signal each iteration is tools/verify.py, in the mode matching <source>:
# oracle mode — <target_cmd> runs <target_files>, e.g. "python3 ./guardrail.py"
python3 <s
阅读完整来源 ↗