跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
设计与 UI

qa-metrics

Define, track, and act on QA metrics: test coverage percentage, flakiness rate, defect escape rate, MTTR, test execution time trends, automation ROI, quality gates, and SLAs for test suites. Includes metric formulas, realistic targets by company stage, and the action to take when each metric goes red. Use when: "QA metrics," "test metrics," "quality KPIs," "test health," "flakiness rate," "defect escape rate." Not for: building the dashboard UI (Allure/Grafana) — use qa-dashboard; measuring coverage gaps and mutation score — use coverage-analysis. Related: qa-dashboard, coverage-analysis, ci-cd-integration, release-readiness, quality-postmortem.

kindlmann · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add petrkindlmann/qa-skills --skill qa-metrics

技能说明


name: qa-metrics description: >- Define, track, and act on QA metrics: test coverage percentage, flakiness rate, defect escape rate, MTTR, test execution time trends, automation ROI, quality gates, and SLAs for test suites. Includes metric formulas, realistic targets by company stage, and the action to take when each metric goes red. Use when: "QA metrics," "test metrics," "quality KPIs," "test health," "flakiness rate," "defect escape rate." Not for: building the dashboard UI (Allure/Grafana) — use qa-dashboard; measuring coverage gaps and mutation score — use coverage-analysis. Related: qa-dashboard, coverage-analysis, ci-cd-integration, release-readiness, quality-postmortem. license: MIT metadata: author: kindlmann version: "2.0" category: metrics

Quick Route

Your situationMetric to reach forSection / reference
No metrics yet, where to startcode coverage, flakiness, defect escapeCore Principles + references/rollout.md
Bugs escaping despite high coveragedefect escape rate + mutation scoreCoverage Metrics, Defect Metrics
Tests flaky, devs ignore CIflakiness rate, pass rateTest Health Metrics
Slow pipeline, stale feedbacksuite duration, parallelism efficiencyExecution Metrics
Justifying automation spendautomation ROIProcess Metrics
Picking targets for our stagecompany-stage tableSetting Realistic Targets
Leadership wants delivery + qualityDORA + escape rateEngineering Quality Metrics (DORA)
Building the dashboard view(go to qa-dashboard)—

Discovery Questions

Check .agents/qa-project-context.md first — if it exists, use it as the foundation and skip anything already answered there. Then:

Current State

  • What metrics are you tracking today? (Even "we glance at CI pass rate sometimes" counts.)
  • Where does your test data live? (CI system, coverage reports, bug tracker, spreadsheets, nowhere)
  • Do you have any dashboards already? Who looks at them, and how often?
  • What tooling runs your CI pipeline? (GitHub Actions, GitLab CI, Jenkins, CircleCI)

Stakeholders

  • Who are the stakeholders for quality metrics? (engineering, product, leadership, customers)
  • What does each care about? Engineers want flakiness; leadership wants escape trends; product wants release confidence.
  • Who will own each metric? If nobody owns it, nobody acts on it.

Quality Problems

  • What quality problems need visibility? (regressions, slow pipelines, flaky tests, coverage gaps, incidents)
  • What broke in production recently? Would a metric have caught it earlier?
  • What decisions are you making without data today?

Goals

  • What does "healthy test suite" mean for your team?
  • Any compliance or contractual quality requirements? (SLAs, SOC2, ISO)
  • Quick wins vs. full observability — what is the appetite for metrics infrastructure?

Core Principles

1. Metrics Should Drive Action, Not Just Dashboards

A metric without an action plan is decoration. For every metric, define: what threshold triggers action, what the action is, and who takes it. If flakiness crosses 5%, the on-call engineer investigates the top 3 flaky tests that week. No ambiguity.

2. Leading vs Lagging — Track Both, Act on Leading

Defect escape rate and MTTR are lagging: you learn after users were hurt. Flakiness, coverage delta, and skipped-test count are leading: they predict escapes before they happen. Lagging metrics tell leadership whether quality is moving; leading metrics are where engineers spend their daily attention because they can still change the outcome.

3. Trend Over Snapshot

A single number is nearly useless. Coverage at 72% means nothing; coverage trending 68% → 72% over three sprints tells a story. Display metrics as time series and evaluate direction, not absolute position.

4. Every Metric Needs a Target and an Owner

A target makes a metric actionable; an owner makes it accountable. Without both it becomes background noise. Set targets from your team's maturity (see the targets table) and assign owners who can actually move the number.

5. Vanity Metrics Waste Everyone's Time

"We have 4,000 tests" sounds impressive until 800 are disabled and 200 flaky. Count what matters: tests that run, pass reliably, and catch real bugs. The ultimate metric is whether users hit bugs that hurt the business — work backward from there. Defect escape rate connects directly to user experience; lines of test code connects to nothing.


Essential QA Metrics

Each metric: definition, formula, recommended target, why it matters, and the action when it goes red.

Test Coverage Metrics

How much of our system is verified by automated tests? (leading)

Code Coverage Percentage

The percentage of code exercised by automated tests (line, branch, or statement level).

Line coverage   = (lines executed by tests / total executable lines) × 100
Branch coverage = (branches executed by tests / total branches) × 100

Targets: Line: 70-85% for app code. Branch: 60-75%. Critical paths (payments, auth): 90%+.

Why it matters: Coverage identifies blind spots — code never exercised by tests is where bugs hide undetected.

When it goes red: Coverage drops on PR: block merge or flag. Low in critical module: create targeted tasks. Plateaus: check for dead code vs. genuinely untested logic.

Warning: Coverage measures execution, not assertion quality. Pair it with mutation testing (StrykerJS v9.6+, mutmut) for a truer picture. Stryker's Vitest runner tracks recent Vitest releases — check

阅读完整来源 ↗