Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Development

build-ml-pipeline

Declare the pipeline from data source to predictor as a **skrub DataOps graph** (not as a bare `sklearn.Pipeline`). Every step is either a pure-Python function (stateless) attached via `.skb.apply_func`, or a sklearn-compatible estimator (stateful) attached via `.skb.apply`. Stops at the declared object — no fit, split, tuning, persistence, or evaluation. TRIGGER — any of: - Writing or editing code that declares any link in the chain *data source → predictor*: loaders, preprocessing, encoders / imputers / scalers, feature steps, composition objects (`Pipeline`, `ColumnTransformer`, skrub `tabular_pipeline`, `nn.Module`), or the final estimator. - A pure-Python data-processing function destined for the pipeline path (cleans / derives / reshapes) — whether wrapped via `FunctionTransformer`, `skrub.@deferred` / `skrub.var`, a custom `BaseEstimator` subclass, or just called in the training path before the estimator. - A step is added, removed, swapped, or reordered inside an existing pipel

probabl-ai · Static checks on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add probabl-ai/skills --skill build-ml-pipeline

About this skill


name: build-ml-pipeline description: > Declare the pipeline from data source to predictor as a skrub DataOps graph (not as a bare sklearn.Pipeline). Every step is either a pure-Python function (stateless) attached via .skb.apply_func, or a sklearn-compatible estimator (stateful) attached via .skb.apply. Stops at the declared object — no fit, split, tuning, persistence, or evaluation.

TRIGGER — any of:

  • Writing or editing code that declares any link in the chain data source → predictor: loaders, preprocessing, encoders / imputers / scalers, feature steps, composition objects (Pipeline, ColumnTransformer, skrub tabular_pipeline, nn.Module), or the final estimator.
  • A pure-Python data-processing function destined for the pipeline path (cleans / derives / reshapes) — whether wrapped via FunctionTransformer, skrub.@deferred / skrub.var, a custom BaseEstimator subclass, or just called in the training path before the estimator.
  • A step is added, removed, swapped, or reordered inside an existing pipeline declaration.
  • A bare sklearn.Pipeline / make_pipeline is being used as the top-level — fire to redirect into a skrub DataOps graph.
  • The user asks to build / declare / set up a pipeline / classifier / regressor for X.

SKIP when: .fit(...) calls / training loops / Trainer.fit / epoch loops; train/test split or cross-validation splitting; hyperparameter search; persistence (joblib.dump, checkpointing); evaluation / metrics / scoring; inference over a pre-trained model; pure EDA; library-choice questions with no concrete declaration in play.

HOW TO USE: consult before the first declarative line and on every structural edit (added/swapped step, changed input columns, changed estimator family). Don't re-consult for cosmetic edits. First, read the Stop conditions and emit the Pre-flight checklist as visible text before any code. Always invoke python-api to confirm skrub / sklearn symbol names and signatures before typing — don't guess from memory.

Build ML Pipeline (Declaration)

Declarative shape of a Python ML pipeline from data source to predictor.

Terms used in this skill

Read these once; they're referenced throughout.

  • X marker — the .skb.mark_as_X() call that anchors the predict-time slice. Everything upstream runs identically at fit and predict; everything downstream is per-prediction work.
  • Predict grid — the rows you want predictions for at predict time. For IID flat tables: the loaded frame itself. For time-series / panels: a (group, time) set.
  • Cold-start row — a predict-grid row that has no in-slice history available (typical for lags at the start of the slice).
  • Predict-time replay — re-binding the graph to a fresh source identifier at predict (e.g. learner.predict({"data_dir": …})).
  • Cross-row step — a feature whose output for a row reads values from other rows (lag, rolling window, group aggregation, side-table join by time/group, drop_nulls on a shifted column).
  • Layers 1 / 2 / 3 — source / predict-grid + X-marker / features after the marker. Defined in Rule 2.

Next-step pointers — where you go after this skill

You came here for…→ next
Declared pipeline → CV strategy→ evaluate-ml-pipeline (the G-CV-SPLITTER gate, rule 3)
Declared pipeline → smoke test→ test-ml-pipeline → smoke-test-ml-pipeline
Symbol lookup mid-declaration→ python-api (Shape 1 / 1b / 3)
Missing skrub/sklearn import→ python-env-manager § install
Modified pipeline.py / features.py / data.py→ python-code-style (ruff + NumPyDoc)

Always re-emit the Pre-flight checklist with evidence before declaring the turn done.

Canonical pipeline shape — IID flat-table

The 90% case. Copy + adapt; replace TARGET_COL and the regressor.

import skrub
from sklearn.ensemble import HistGradientBoostingRegressor

from <pkg>.data import TARGET_COL, load_raw


def build_learner(data_dir_preview=None):
    """Return the unfit learner (skrub SkrubLearner)."""
    data_dir = (
        skrub.var("data_dir", value=str(data_dir_preview))
        if data_dir_preview is not None
        else skrub.var("data_dir")
    )

    # Layer 1 + 2: load + mark X / y on the source frame.
    # No cross-row feature steps → marker sits here.
    data = data_dir.skb.apply_func(load_raw)
    X = data.drop(columns=[TARGET_COL]).skb.mark_as_X()
    y = data[TARGET_COL].skb.mark_as_y()

    # Layer 3: estimator at the tail. Feature engineering (if any)
    # chains between mark_as_X and the final .skb.apply.
    predictions = X.skb.apply(
        HistGradientBoostingRegressor(random_state=0), y=y
    )
    return predictions.skb.make_learner()

For history-dependent / panel / cold-start cases (≠ IID): → references/layer_examples.md § history-dependent.

For loader-baked-shift counter-example (what NOT to do): → references/layer_examples.md § counter-example.

Stop conditions — read before anything else

Each Stop condition: rule → symptom → recovery. Scan top to bottom; any match means STOP.

S1. Missing dependency

  • Rule: import skrub raising means python-env-manager is next, not a substitute library.
  • Symptom: ModuleNotFoundError: No module named 'skrub'.
  • Recovery: invoke python-env-manager for the install command. Do NOT substitute with sklearn.Pipeline / make_pipeline / FunctionTransformer — that silently rewrites this skill out of the project.

S2. Symbol from memory is forbidden

  • Rule: every skrub / scikit-learn / skore name must come from a python-api lookup this turn.
  • Symptom: you type tabular_learner (renamed in 0.7+), mark_as_y(col) (signature dropped the positional in 0.9+), or any name "you remember".
  • Recovery: invoke python-api. Recognition is not a lookup;
Read the complete source ↗