build-ml-pipeline
Declare the pipeline from data source to predictor as a **skrub DataOps graph** (not as a bare `sklearn.Pipeline`). Every step is either a pure-Python function (stateless) attached via `.skb.apply_func`, or a sklearn-compatible estimator (stateful) attached via `.skb.apply`. Stops at the declared object — no fit, split, tuning, persistence, or evaluation. TRIGGER — any of: - Writing or editing code that declares any link in the chain *data source → predictor*: loaders, preprocessing, encoders / imputers / scalers, feature steps, composition objects (`Pipeline`, `ColumnTransformer`, skrub `tabular_pipeline`, `nn.Module`), or the final estimator. - A pure-Python data-processing function destined for the pipeline path (cleans / derives / reshapes) — whether wrapped via `FunctionTransformer`, `skrub.@deferred` / `skrub.var`, a custom `BaseEstimator` subclass, or just called in the training path before the estimator. - A step is added, removed, swapped, or reordered inside an existing pipel
probabl-ai · Static checks on file
Installation
Read the source and dependencies before running a command. Copying does not install or execute anything.
npx skills add probabl-ai/skills --skill build-ml-pipelineAbout this skill
name: build-ml-pipeline
description: >
Declare the pipeline from data source to predictor as a skrub
DataOps graph (not as a bare sklearn.Pipeline). Every step is
either a pure-Python function (stateless) attached via
.skb.apply_func, or a sklearn-compatible estimator (stateful)
attached via .skb.apply. Stops at the declared object — no fit,
split, tuning, persistence, or evaluation.
TRIGGER — any of:
- Writing or editing code that declares any link in the chain
data source → predictor: loaders, preprocessing, encoders /
imputers / scalers, feature steps, composition objects
(
Pipeline,ColumnTransformer, skrubtabular_pipeline,nn.Module), or the final estimator. - A pure-Python data-processing function destined for the
pipeline path (cleans / derives / reshapes) — whether wrapped
via
FunctionTransformer,skrub.@deferred/skrub.var, a customBaseEstimatorsubclass, or just called in the training path before the estimator. - A step is added, removed, swapped, or reordered inside an existing pipeline declaration.
- A bare
sklearn.Pipeline/make_pipelineis being used as the top-level — fire to redirect into a skrub DataOps graph. - The user asks to build / declare / set up a pipeline / classifier / regressor for X.
SKIP when: .fit(...) calls / training loops / Trainer.fit /
epoch loops; train/test split or cross-validation splitting;
hyperparameter search; persistence (joblib.dump, checkpointing);
evaluation / metrics / scoring; inference over a pre-trained
model; pure EDA; library-choice questions with no concrete
declaration in play.
HOW TO USE: consult before the first declarative line and on
every structural edit (added/swapped step, changed input columns,
changed estimator family). Don't re-consult for cosmetic edits.
First, read the Stop conditions and emit the Pre-flight
checklist as visible text before any code. Always invoke
python-api to confirm skrub / sklearn symbol names and
signatures before typing — don't guess from memory.
Build ML Pipeline (Declaration)
Declarative shape of a Python ML pipeline from data source to predictor.
Terms used in this skill
Read these once; they're referenced throughout.
- X marker — the
.skb.mark_as_X()call that anchors the predict-time slice. Everything upstream runs identically at fit and predict; everything downstream is per-prediction work. - Predict grid — the rows you want predictions for at predict
time. For IID flat tables: the loaded frame itself. For
time-series / panels: a
(group, time)set. - Cold-start row — a predict-grid row that has no in-slice history available (typical for lags at the start of the slice).
- Predict-time replay — re-binding the graph to a fresh source
identifier at predict (e.g.
learner.predict({"data_dir": …})). - Cross-row step — a feature whose output for a row reads values from other rows (lag, rolling window, group aggregation, side-table join by time/group, drop_nulls on a shifted column).
- Layers 1 / 2 / 3 — source / predict-grid + X-marker / features after the marker. Defined in Rule 2.
Next-step pointers — where you go after this skill
| You came here for… | → next |
|---|---|
| Declared pipeline → CV strategy | → evaluate-ml-pipeline (the G-CV-SPLITTER gate, rule 3) |
| Declared pipeline → smoke test | → test-ml-pipeline → smoke-test-ml-pipeline |
| Symbol lookup mid-declaration | → python-api (Shape 1 / 1b / 3) |
| Missing skrub/sklearn import | → python-env-manager § install |
Modified pipeline.py / features.py / data.py | → python-code-style (ruff + NumPyDoc) |
Always re-emit the Pre-flight checklist with evidence before declaring the turn done.
Canonical pipeline shape — IID flat-table
The 90% case. Copy + adapt; replace TARGET_COL and the regressor.
import skrub
from sklearn.ensemble import HistGradientBoostingRegressor
from <pkg>.data import TARGET_COL, load_raw
def build_learner(data_dir_preview=None):
"""Return the unfit learner (skrub SkrubLearner)."""
data_dir = (
skrub.var("data_dir", value=str(data_dir_preview))
if data_dir_preview is not None
else skrub.var("data_dir")
)
# Layer 1 + 2: load + mark X / y on the source frame.
# No cross-row feature steps → marker sits here.
data = data_dir.skb.apply_func(load_raw)
X = data.drop(columns=[TARGET_COL]).skb.mark_as_X()
y = data[TARGET_COL].skb.mark_as_y()
# Layer 3: estimator at the tail. Feature engineering (if any)
# chains between mark_as_X and the final .skb.apply.
predictions = X.skb.apply(
HistGradientBoostingRegressor(random_state=0), y=y
)
return predictions.skb.make_learner()
For history-dependent / panel / cold-start cases (≠ IID):
→ references/layer_examples.md § history-dependent.
For loader-baked-shift counter-example (what NOT to do):
→ references/layer_examples.md § counter-example.
Stop conditions — read before anything else
Each Stop condition: rule → symptom → recovery. Scan top to bottom; any match means STOP.
S1. Missing dependency
- Rule:
import skrubraising meanspython-env-manageris next, not a substitute library. - Symptom:
ModuleNotFoundError: No module named 'skrub'. - Recovery: invoke
python-env-managerfor the install command. Do NOT substitute withsklearn.Pipeline/make_pipeline/FunctionTransformer— that silently rewrites this skill out of the project.
S2. Symbol from memory is forbidden
- Rule: every skrub / scikit-learn / skore name must come from
a
python-apilookup this turn. - Symptom: you type
tabular_learner(renamed in 0.7+),mark_as_y(col)(signature dropped the positional in 0.9+), or any name "you remember". - Recovery: invoke
python-api. Recognition is not a lookup;