mlops-validation
Add the validation layers that gate a merge — ty typing, Ruff linting, pytest coverage, structured logging, and the trivy, pip-audit, and gitleaks scans. Use when hardening code quality or wiring the mise run check task.
Médéric HURIER (Fmind) · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add MLOps-Courses/mlops-coding-skills --skill mlops-validation技能说明
name: mlops-validation description: Add the validation layers that gate a merge — ty typing, Ruff linting, pytest coverage, structured logging, and the trivy, pip-audit, and gitleaks scans. Use when hardening code quality or wiring the mise run check task. license: MIT metadata: author: Médéric HURIER (Fmind) source: github.com/MLOps-Courses/mlops-coding-skills/tree/main/mlops-validation created: 2026-01-25 updated: 2026-10-12
MLOps Validation
Goal
To ensure software quality, reliability, and security through automated validation layers. This skill enforces Strict Typing (ty), Unified Linting (ruff), Comprehensive Testing (pytest), Structured Logging, and Supply-Chain Scanning (pip-audit, gitleaks, trivy).
Prerequisites
- Language: Python 3.14
- Manager:
uv; Tasks:mise - Context: Ensuring code quality before merge/deploy.
Instructions
Preserve existing work and repository visibility. Prepare local changes first; commit, push, change remote settings, publish, or deploy only with user authorization.
1. The Task Vocabulary
Every check below is a mise task, so the same command runs locally, in the git hook, and in CI. Never invoke the underlying tool by hand in a hook or a workflow — the task is the single definition.
| Task | Tool | What it proves |
|---|---|---|
check:format | dprint, validate-pyproject, ruff format --check, uv lock --check | Files and manifests are canonical and the lockfile is current. |
check:lint | ruff check | No lint violation, including the S (security) rules. |
check:types | ty check | Type annotations are consistent. |
check:vuln | pip-audit | No known CVE in the resolved Python dependencies. |
check:leaks | gitleaks | No secret in the staged change or the recent history. |
check:scan | trivy | No misconfiguration, leaked secret, or forbidden license in the tree. |
check:actions | actionlint, zizmor | Workflows are valid and not vulnerable. |
test | pytest | Behavior is covered and correct. |
mise run check runs the check:* tasks in parallel; mise run all chains format -> check -> test -> build and is the only gate anyone needs to remember.
2. Static Analysis (Typing & Linting)
Catch errors before they run.
- Typing:
- Tool:
ty(Astral type checker; pre-1.0, pin a compatible range such asty>=0.0.69,<0.1). The mandated checker — do not usemypy. - Rule: No
Any(unless absolutely necessary). Fully typed function signatures. - Pragmatism:
tydoes not yet model every dynamic library (MLflow, pandera, Pydantic). Prefer a narrow ignore at the affected boundary, with an explanation and runtime validation. Retain existing project compatibility exceptions only while the library/checker limitation still applies. - DataFrames: Use
panderaschemas to validate DataFrame structures/types. - Classes: Use
pydanticfor data modeling and runtime validation.
- Tool:
- Linting & Formatting:
- Tool:
ruff0.16+ (replaces black, isort, pylint, flake8, bandit). - Rule: Zero tolerance for linter errors. Use
noqasparingly and with justification. - Config: Centralize in
pyproject.toml, with an explicit[tool.ruff.lint] selectlist. Ruff 0.16 expanded the default rule set from 59 to 413 rules, so a project that relies on the default set changes behavior on upgrade while an explicitselectlist does not. - Markdown: Ruff 0.16 also formats Python code blocks inside
.mdfiles. Expect a one-time reformat of your documentation on the first run, and commit it.
- Tool:
3. Testing Strategy
Verify behavior and prevent regressions.
-
Tool:
pytest(9.x). -
Structure: Mirror
src/intests/.src/pkg/mod.py -> tests/test_mod.py -
Fixtures: Use
tests/conftest.pyfor shared setup (mock data, temp paths). -
Coverage: Measure with
pytest-covand set--cov-fail-underto the level the suite actually reaches, so any drop is a visible regression rather than slack under a round number. -
Pattern: Use Given-When-Then in comments.
from sklearn.impute import SimpleImputer def test_imputer_uses_training_statistics() -> None: # Given: Training observations with mean 2. imputer = SimpleImputer(strategy="mean").fit([[1.0], [3.0]]) # When: A fitted transformer processes a missing held-out value. output = imputer.transform([[float("nan")]]) # Then: Training statistics fill the held-out value. assert output.tolist() == [[2.0]] -
MLflow in tests: Point the tests at a SQLite tracking store, not the deprecated file store — but build it once. Creating a fresh MLflow SQLite database runs the full Alembic migration chain (measured at roughly 7 seconds per database), so a session-scoped fixture should migrate one template database and each test should
shutil.copyfileit into its owntmp_path(roughly 0.07 seconds). Migrating per test turned a 34