Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Design & UI

iterate-from-skore

Source the next ML experiment proposal by **reading the audit digest** at `scratch/audit/<stem>/audit.md` (produced by `audit-ml-pipeline` at § 4 record-outcome). For every row in the digest's `## Checks summary` whose `severity` is `issue` or `tip`, follow the row's `documentation_url` to draft a Backlog row whose `Item` is the mitigation the docs recommend. The `## Metrics summary` provides context for the human summary paragraph but does not drive Backlog rows on its own. Returns the enriched Backlog rows + a one-paragraph summary back to `iterate-ml-experiment`, which writes the rows into `JOURNAL.md` and re-presents the sourcing menu so the user can promote a `B<N>` row. Stops at "Backlog enriched, summary returned"; never writes a per-experiment design note, never picks the "winning" finding — the user picks via `B<N>`. TRIGGER when: `iterate-ml-experiment` is picking a sourcing strategy and the user picks `skore` from the menu; the user says "mine the report", "what does skore s

probabl-ai · Static checks on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add probabl-ai/skills --skill iterate-from-skore

About this skill


name: iterate-from-skore description: > Source the next ML experiment proposal by reading the audit digest at scratch/audit/<stem>/audit.md (produced by audit-ml-pipeline at § 4 record-outcome). For every row in the digest's ## Checks summary whose severity is issue or tip, follow the row's documentation_url to draft a Backlog row whose Item is the mitigation the docs recommend. The ## Metrics summary provides context for the human summary paragraph but does not drive Backlog rows on its own. Returns the enriched Backlog rows + a one-paragraph summary back to iterate-ml-experiment, which writes the rows into JOURNAL.md and re-presents the sourcing menu so the user can promote a B<N> row. Stops at "Backlog enriched, summary returned"; never writes a per-experiment design note, never picks the "winning" finding — the user picks via B<N>.

TRIGGER when: iterate-ml-experiment is picking a sourcing strategy and the user picks skore from the menu; the user says "mine the report", "what does skore see?", "fill the backlog from the diagnostic"; the previous experiment has finished and the user wants the report converted into actionable backlog items.

SKIP when: the previous experiment hasn't run yet (no audit digest on disk); the user has a concrete modelling idea (use iterate-from-user); the task is the mechanics of running / opening a report — route to evaluate-ml-pipeline; the user wants a narrative read of one specific section of the report (route to evaluate-ml-pipeline).

HOW TO USE: read the existing scratch/audit/<stem>/audit.md digest as text — do NOT re-open the skore Project, do NOT call report.* accessors. For each row in the ## Checks summary section whose severity is issue or tip, follow the documentation_url (via WebFetch) and draft one Backlog row citing audit:<stem>:checks.<code>. Dedupe against rows already in JOURNAL.md Backlog by source citation. Return the candidate rows + a one-paragraph human summary. The parent skill writes the rows to JOURNAL.md and re-shows the sourcing menu.

Iterate from skore

Source: the audit digest at scratch/audit/<stem>/audit.md, produced by audit-ml-pipeline at § 4 record-outcome. Output: a set of Backlog-candidate rows + a short human summary, handed back to iterate-ml-experiment. The parent skill writes the rows to JOURNAL.md Backlog and re-presents the sourcing menu so the user can promote one via B<N>.

What this skill consumes

The digest carries two sections that matter here:

  • ## Checks summary — a DataFrame whose rows each have code, severity (passed / issue / tip), and documentation_url. Each issue / tip row → one Backlog candidate, with the documentation_url driving the Item text.
  • ## Metrics summary — task-appropriate headline metrics (regression / classification / multiclass). Used to ground the human summary paragraph ("the run achieved X but the SKD003 check flagged Y"). Does not drive Backlog rows on its own.

Nothing else. The audit template intentionally stops at these two sections; deeper accessors (residuals, importance, calibration, …) are out of scope here.

Why read the digest (not re-walk the Project)

The audit already opened the Project, loaded the report, called the two accessors, and rendered the output as markdown. Re-doing that work here would duplicate the cost of materialising Display objects, risk drift between two walks, and require the agent environment this skill should not need. Reading the digest as text is cheaper and deterministic.

Output contract (read this before the body)

This skill never writes journal/ files (including JOURNAL.md) — the parent owns those. It returns two artifacts as conversation text:

  1. Backlog-candidate rows — one row per actionable check from the digest. Each row carries:

    • Item: one-line experiment idea derived from the check's documentation_url content. Phrase as an experiment idea, not as a metric reading.
    • Source: audit:<stem>:checks.<code> (e.g. audit:01_baseline:checks.SKD003). The citation is load-bearing for dedup.
  2. Summary — one paragraph for the user: how many findings were surfaced, the top 2-3 by severity, the headline numbers from the metrics summary as context. Keep it dense.

If the parent's Backlog already contains a row with the same Source citation, drop the candidate — do not duplicate. The summary should note the number of dropped duplicates ("4 new findings; 2 were already in Backlog from prior mining").

Empty-checks outcome

If the digest's checks summary has no issue / tip rows (only passed), return zero candidate rows and a summary that says so explicitly: "the report looks clean on the checks surface; no actionable findings on this turn." The parent will note this in JOURNAL.md Status and the user picks user next.

Inaccessible-digest fallback

If the digest at scratch/audit/<stem>/audit.md cannot be read (file missing, audit never executed, audit errored), do not fabricate findings from memory and do not re-run probes. Return zero rows and a summary that explains the access failure. The parent surfaces the gap to the user; recovery is owned by audit-ml-pipeline (re-run the audit runner, fix the auth, …).

Stop conditions

  • Don't write journal/ files. That includes JOURNAL.md. This skill returns rows as conversation text; the parent writes them.
  • Don't re-open the skore Project from this skill. The audit already did. Reading the digest as text is the contract — see § "Why read the digest". If the digest is missing, re-execute the audit runner via audit-ml-pipeline; never call project.get(...) from iterate-from-skore.
  • Only ## Checks summary rows drive Backlog candidates. The metrics summary is context for t
Read the complete source ↗