Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Security

explore-ml-data

Owns data understanding BEFORE any model is designed. Places and executes `data/eda.py` (a jupytext `# %%` script) via the shared in-process runner, reads the streamed digest, then writes a persisted `data/eda.md` report (plus linked `data/eda_<table>.html` skrub `TableReport` pages) and the `## Data understanding (EDA)` section of `journal/JOURNAL.md`. The point is to surface the dataset facts — shape, dtypes, missingness, cardinality, target balance / skew, datetime / group structure, feature associations — that JUSTIFY the later learner / splitter / metric decisions, so the user understands *why* the modelling choices are made. Uses `skrub.TableReport` for dataframe overviews and the shared runner `audit-ml-pipeline/scripts/run_cells.py`. Stops at "EDA executed, `data/eda.md` + HTML written, JOURNAL EDA section updated." Never designs the model, never edits `src/<pkg>/`, never modifies the user's raw data files. TRIGGER — any of: - `iterate-ml-experiment` § 0 bootstrap, BEFORE the b

probabl-ai · AI review on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add probabl-ai/skills --skill explore-ml-data

About this skill


name: explore-ml-data description: > Owns data understanding BEFORE any model is designed. Places and executes data/eda.py (a jupytext # %% script) via the shared in-process runner, reads the streamed digest, then writes a persisted data/eda.md report (plus linked data/eda_<table>.html skrub TableReport pages) and the ## Data understanding (EDA) section of journal/JOURNAL.md. The point is to surface the dataset facts — shape, dtypes, missingness, cardinality, target balance / skew, datetime / group structure, feature associations — that JUSTIFY the later learner / splitter / metric decisions, so the user understands why the modelling choices are made. Uses skrub.TableReport for dataframe overviews and the shared runner audit-ml-pipeline/scripts/run_cells.py. Stops at "EDA executed, data/eda.md + HTML written, JOURNAL EDA section updated." Never designs the model, never edits src/<pkg>/, never modifies the user's raw data files.

TRIGGER — any of:

  • iterate-ml-experiment § 0 bootstrap, BEFORE the baseline design note — the G-EDA gate fires here (run / skip).
  • The user asks to "explore the data", "do an EDA", "profile the dataset", "what does the data look like", "understand the data".
  • A new or changed data source needs (re-)understanding before the next experiment.

SKIP when: the workspace isn't scaffolded / bootstrapped yet — iterate-ml-experiment § 0 owns bootstrap ordering and will dispatch here at the G-EDA step; don't run standalone ahead of scaffolding (route to iterate-ml-experiment / organize-ml- workspace); there is no data to explore yet; the user wants to inspect a finished run's skore report rather than the raw dataset (audit-ml-pipeline); the user is past data understanding and wants pipeline / evaluation mechanics (build-ml-pipeline / evaluate-ml-pipeline); a pure symbol lookup (python-api); EDA is already recorded (data/eda.md + the JOURNAL EDA section exist) and the user is not asking to refresh it.

HOW TO USE: run the Detection step (does data/eda.md + the JOURNAL EDA section already exist?), emit the Pre-flight checklist as visible text, read the Stop conditions, then place data/eda.py from templates/eda.py, execute it via the shared runner, read the digest, and author data/eda.md + the JOURNAL EDA section. Always resolve skrub / pandas / polars symbols via python-api, never from memory.

Explore ML Data

Understand the dataset before designing a model. One project-level EDA per workspace: an executable data/eda.py, a persisted data/eda.md narrative, rich data/eda_<table>.html reports, and a short JOURNAL section that links them. The findings feed the baseline design note's learner / splitter / metric choices.

Next-step pointers — where you go after this skill

You came here for…→ next
Bootstrap, before the first baseline→ back to iterate-ml-experiment § 0; the EDA findings inform the auto-drafted 01_baseline.md
User free-text ("explore the data")→ surface the findings; no further dispatch unless the user asks to model
Re-understand a changed data source→ re-run, overwrite data/eda.*, refresh the JOURNAL EDA section

Always re-emit the Pre-flight checklist with evidence before declaring the turn done.

Where this sits in the loop

EDA is a bootstrap-time gate (G-EDA) owned by this skill and fired by iterate-ml-experiment § 0 before the baseline design note. Ordering matters: the dataset facts (class balance, datetime / group columns, missingness, cardinality) are exactly what justifies the splitter (G-CV-SPLITTER), the metric default, and the learner default. Running EDA after the model is designed defeats the purpose.

scaffold → JOURNAL → goal from data/README.md
   │
   └─► G-EDA (run | skip)  ◄── this skill
         │ run
         └─► data/eda.py → execute → data/eda.md + HTML + JOURNAL §EDA
   │
   └─► auto-draft 01_baseline.md  (cites the EDA findings)

Where things live — visual map

Two locations are kept separate: the raw data source (read-only, may live anywhere) and the EDA deliverables (always under <project>/data/).

PathDurabilityWho writes itWhat it holds
raw data source (data/, raw/, an absolute path, external)user-owned, READ-ONLYthe userThe dataset. EDA reads it; never modifies it. May be anywhere — not assumed to be data/
data/eda.pyDurable (committed)This skill, once per workspaceThe jupytext # %% EDA cells. Source of truth. Openable as a notebook for the rich view
data/eda.mdDurable (committed)This skill (authored from the digest)The prose narrative: findings + modelling implications that the baseline note cites
data/eda_<table>.htmlDurable (committed)data/eda.py via TableReport.write_html(...)The rich, interactive skrub report per table — for the human
scratch/eda/eda.mdEphemeral (gitignored), optionalrun_cells.py when given a 2nd argPer-cell digest the agent reads. Same content as stdout
journal/JOURNAL.md § Data understanding (EDA)Durable (committed)This skill2–4 line summary + link to data/eda.md

Mnemonic: the raw data is read-only and lives wherever the user keeps it; data/eda.py is source; data/eda.md + the HTML are the durable deliverables, always under data/; scratch/eda/ and stdout are the ephemeral run digest.

Read-only-against-raw-data contract

The central rule. Surfaced as the first Stop condition below.

Allowed — this skill writes ONLY (deliverables always under <project>/data/, created if absent):

  • data/eda.py — the EDA script (created / overwritten in place).
  • data/eda.md — the authored narrative.
  • data/eda_<table>.html — the skrub TableReport pages.
  • scratch/eda/ — the ephemeral digest.
  • `jou
Read the complete source ↗