iterate-ml-experiment
Owns the iteration loop on top of an ML workspace: the `journal/JOURNAL.md` index and the per-experiment `journal/NN_short_name.md` design notes that must be drafted and approved by the user **before** `experiments/NN_short_name.py` is created. Drives the propose → iterate → approve → implement → record loop; dispatches to `iterate-from-skore` / `iterate-from-user` for sourcing. TRIGGER — any of: - A session opens in an ML workspace (whether or not `journal/` exists yet — missing/placeholder → bootstrap mode). - User says "what's next", "resume", "where were we", "let's iterate", "propose next", "first baseline". - About to create a new `experiments/NN_*.py` (the matching `journal/NN_*.md` must exist and be approved first). - User wants to record an outcome from a finished run. - User asks to compare past experiments or review what's been tried ("compare X and Y", "where are we?"). SKIP when: no `journal/` yet AND no workspace scaffold (route to `organize-ml-workspace`); the work is me
probabl-ai · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add probabl-ai/skills --skill iterate-ml-experiment技能说明
name: iterate-ml-experiment
description: >
Owns the iteration loop on top of an ML workspace: the
journal/JOURNAL.md index and the per-experiment
journal/NN_short_name.md design notes that must be drafted and
approved by the user before experiments/NN_short_name.py is
created. Drives the propose → iterate → approve → implement →
record loop; dispatches to iterate-from-skore /
iterate-from-user for sourcing.
TRIGGER — any of:
- A session opens in an ML workspace (whether or not
journal/exists yet — missing/placeholder → bootstrap mode). - User says "what's next", "resume", "where were we", "let's iterate", "propose next", "first baseline".
- About to create a new
experiments/NN_*.py(the matchingjournal/NN_*.mdmust exist and be approved first). - User wants to record an outcome from a finished run.
- User asks to compare past experiments or review what's been tried ("compare X and Y", "where are we?").
SKIP when: no journal/ yet AND no workspace scaffold (route to
organize-ml-workspace); the work is mechanical inside
pipeline.py / evaluate.py / data.py with no journal-level
implication (owned by build-ml-pipeline /
evaluate-ml-pipeline); the user asks for a symbol lookup
(python-api); the user is diagnosing a single skore report
without a "what next" framing (evaluate-ml-pipeline).
HOW TO USE: read journal/JOURNAL.md first, classify the turn via
the Mode picker (table near the top), then read only the
matching section. Sibling skills open just-in-time when a step
requires them — do not pre-read all sibling skills at session start.
Design notes are the only artifact this skill writes; read,
compare, and overview modes don't write.
Iterate ML Experiment
The loop on top of experiments/: what to try next, why, what
counts as a result, how the trail is recorded. Pipeline / evaluation
mechanics live in sibling skills.
Next-step pointers — flow at a glance
session open
│
├── JOURNAL.md missing / placeholder ──► § 0 Bootstrap
│ │
│ ├─► G-EDA (explore-ml-data: run | skip)
│ │
│ └─► design note → G-DESIGN → § 3 implement
│
├── "what's next?" with ≥1 done row ───► § 1 → § 2 (sourcing) → § 3 implement
│
├── "run finished" ─────────────────────► § 4 record outcome
│ │
│ └─► dispatch audit-ml-pipeline
│
└── "status?" / "compare X Y" ──────────► references/maintenance_modes.md
Always re-emit the Pre-flight checklist with evidence before declaring the turn done.
First action — read state + emit read-set tracker
Open each sibling SKILL.md just-in-time when a step calls for
it (e.g. open evaluate-ml-pipeline before § 3's CV-strategy
step). Do not pre-read all at session start.
Sibling skills (just-in-time):
- organize-ml-workspace, data-science-python-stack,
python-env-manager, python-api, python-code-style,
explore-ml-data, build-ml-pipeline, evaluate-ml-pipeline,
test-ml-pipeline, smoke-test-ml-pipeline,
iterate-from-skore / iterate-from-user
Then before answering:
- Read
journal/JOURNAL.md. Missing/placeholder → bootstrap (§ 0). This is the canonical project digest (Status, Data understanding (EDA), History, Backlog). - Check
Workspace decisionsblock for pre-recorded gates (tabular, env_manager, package, skore_mode, cv_splitter) — a recorded decision skips itsAskUserQuestion. - Emit the Pre-flight checklist with each box filled.
- Use the Mode picker to find which section to read.
Mode picker — read this before navigating the body
You read one mode section per turn. Match the user's signal, then jump.
| Signal / workspace state | Mode | Section |
|---|---|---|
JOURNAL.md missing / placeholder / 0 History rows | Bootstrap | § 0 |
journal/ not scaffolded (no src/, no experiments/) | Bootstrap → handoff first | → organize-ml-workspace, then § 0 |
| "what's next?" / "let's iterate" / "propose next" — with ≥1 done row | Iterate (propose) | §§ 1–3 + Dispatch table |
| "the run finished" / "log the result" / "we got X = …" | Iterate (record) | § 4 |
| "where are we?" / "status?" / "what have we tried?" | Project overview | references/maintenance_modes.md § "Project overview" |
| "compare X and Y" / "X vs Y" | Compare (read-only) | references/maintenance_modes.md § "Compare past experiments" |
| "let's pivot the goal" / "actually we care about " | Goal pivot | references/maintenance_modes.md § "Goal pivots" |
| "abandon X" / "drop X" | Abandoned | references/maintenance_modes.md § "Abandoned experiments" |
| Re-do a prior experiment under different conditions | Re-run | references/maintenance_modes.md § Re-runs |
If two modes seem to match ("compare X and Y, then propose"), pick the read mode first, stop. Re-entering § 1 is a separate turn.
Stop conditions — read before anything else
- No design note, no script. Never create or edit
experiments/NN_*.pyuntiljournal/NN_*.mdexists, is filled, and the user has explicitly approved it. JOURNAL.mdis read at session start, not improvised. Don't reconstruct history fromexperiments/filenames orgit log— those don't carry the why.- Strategy is picked, not assumed. Name the sourcing strategy
in every proposal (
skore/user/my-pick/B<N>). Don't silently default. Exception: bootstrap — baseline is forced by workspace defaults; no strategy dispatch. - Approval is explicit. "approved" / "yes" / "go" / "looks
good" from the user is the gate. Ambiguous → re-ask via
AskUserQuestion. - **Outcomes are recorded, not n