ai-qa-review
Review EXISTING test code for quality, smells, and testability issues. Detects test smells across six dimensions — readability, reliability, diagnostic value, design, AI-generated, and coverage — analyzes testability of application code, and backs the qualitative smells with mutation testing. Use when: "review my tests," "test quality audit," "test smells," "testability analysis," "are these tests any good." Not for: generating new tests — use `ai-test-generation`. Not for: testing AI features in your product — use `ai-system-testing`. Related: unit-testing, shift-left-testing, coverage-analysis, ai-test-generation.
petrkindlmann · AI review on file
Installation
Read the source and dependencies before running a command. Copying does not install or execute anything.
npx skills add petrkindlmann/qa-skills --skill ai-qa-reviewAbout this skill
name: ai-qa-review
description: >-
Review EXISTING test code for quality, smells, and testability issues. Detects
test smells across six dimensions — readability, reliability, diagnostic value,
design, AI-generated, and coverage — analyzes testability of application code,
and backs the qualitative smells with mutation testing.
Use when: "review my tests," "test quality audit," "test smells," "testability
analysis," "are these tests any good." Not for: generating new tests — use
ai-test-generation. Not for: testing AI features in your product — use
ai-system-testing.
Related: unit-testing, shift-left-testing, coverage-analysis, ai-test-generation.
license: MIT
metadata:
author: kindlmann
version: "2.0"
category: ai-qa
Before starting: Check for .agents/qa-project-context.md in the project root. It contains test framework conventions, naming patterns, and project-specific quality standards that calibrate review feedback.
Quick Route
Three distinct entry paths. Pick the row, then jump to the named section.
| Situation | Path | Jump to |
|---|---|---|
| PR with changed test files | Run the changed files, score them, check the diff against the PR checklist | Verification → PR Review Checklist |
| Whole suite needs a health pass | Quantify, sample, find the 3-5 systemic smells, propose lint/mutation gates | Batch Audit Process |
| Application code, "why is this hard to test?" | Flag DI / side-effect / pure-function / interface problems with before/after | Testability Analysis |
All three share the same smell vocabulary (the six buckets below) and the same Verification commands.
Discovery Questions
First, read .agents/qa-project-context.md if present and skip any question it already answers.
- Review scope: Reviewing test code for quality, application code for testability, or both? Each triggers a different Quick Route path.
- Framework conventions: What test framework (Jest, Vitest, Playwright, pytest)? Conventions differ —
describe/itnesting, fixture usage, assertion style — and the Verification commands are per-framework. - PR review or batch audit? A PR review runs and scores only the changed files. A batch audit scans the entire suite for systemic patterns.
- Existing quality standards: Does the team have documented test conventions? Check for
.eslintrctest rules,CONTRIBUTING.mdtest guidelines, or a test style guide. - Known pain points: Recurring flaky tests, slow suites, unclear failures? These prioritize which smells to focus on first.
Core Principles
-
Test code is production code. Apply the same quality standards: readability, maintainability, single responsibility. Test code that is hard to read is hard to trust.
-
Review what is asserted, not just what is executed. Coverage proves a line ran; it says nothing about whether a wrong value would be caught. A 95%-coverage suite of
toBeTruthyassertions catches almost nothing. Mutation score (see Verification) measures the thing coverage can't. -
Testability review prevents test debt. Reviewing application code for testability catches design problems before they force awkward test workarounds. If code is hard to test, it is usually hard to maintain.
-
Codify patterns, not just knowledge. Turn recurring review feedback into lint rules, custom ESLint plugins, or shared fixtures. Reviews that repeat the same feedback indicate missing automation.
-
Smells are symptoms, not verdicts. A test smell indicates a potential problem; context decides whether it is actually harmful. A long test for a complex workflow may be appropriate. A mock-heavy test for a boundary may be correct.
-
Actionable feedback only. Every review comment must include what is wrong, why it matters, and how to fix it. "This test is bad" is not actionable. "This test uses sleep-based waiting which causes flakiness — replace with an explicit wait condition" is.
Test Smell Buckets
Six dimensions. Each smell is categorized by the dimension it affects and links to a specific review action. Full SMELL/FIX code for every catalogued smell lives in references/smell-examples.md — keep the inline pointers prominent; the before/after code is the load-bearing part.
Readability Smells
Problems that make tests hard to understand at a glance.
Obscure Setup
What it looks like: 30+ lines of object construction with irrelevant fields drowning the test intent. The reader cannot tell which fields matter for the assertion.
Fix: Extract to factories. Only test-relevant data should appear in the test body: buildOrder({ items: [buildItem({ weight: 2.5, quantity: 2 })] }) instead of constructing full user/product/order objects inline.
Review action: Request factory extraction.
Mystery Guest
What it looks like: loadFixture('report.json') — the test depends on external data the reader cannot see. They must open another file to understand the assertion.
Fix: Inline the test-relevant data or use descriptively named fixtures. The reader should understand the test without opening other files.
Review action: Request inline data or descriptive fixture names.
Duplicate Assertions
What it looks like: Multiple tests assert the same behavior with varying specificity (toBe('Alice'), toHaveProperty('name'), toBeDefined()). Three tests, one behavior.
Review action: Request consolidation. Keep the most specific assertion. Redundant tests increase maintenance cost without increasing c
Read the complete source ↗