release-qa
QA an epic before release on the local environment. Check the environment without changing it, run the browser e2e suite and sort every failure, do exploratory testing in a real browser, file reproduced bugs in the tracker, and give a release call. Use when the user asks for release QA, a bug-hunting pass, or "is this epic ready to release".
Beakpoint-Insights · Static checks on file
Installation
Read the source and dependencies before running a command. Copying does not install or execute anything.
npx skills add Beakpoint-Insights/roost --skill release-qaAbout this skill
name: release-qa description: QA an epic before release on the local environment. Check the environment without changing it, run the browser e2e suite and sort every failure, do exploratory testing in a real browser, file reproduced bugs in the tracker, and give a release call. Use when the user asks for release QA, a bug-hunting pass, or "is this epic ready to release".
Release QA
You are QA for one epic. Find out whether "ready for release" is true, and file what isn't. Don't fix anything: no edits to app or test code, no status changes on existing tickets.
First, load the team config with roost-config. You run in the session of the repo that holds the e2e suite (the one with e2e in repos).
No app to QA in a browser? When the config has no stack, or no repo with e2e (a plugin or a library, say), steps 1 to 3 don't apply. You run in the lead repo's session (planning.lead_repo) instead, and QA is:
- Every repo's own checks on the epic's newest code (its newest branches or open PRs, merged or not; the same commits for all three checks), as each repo's
testssays (for this plugin, for example,claude plugin validate --strict .and its scripts' checks). - The epic's tests, run the way the tests adapter says (for
tests.kind: evals, every case linked to the epic's stories), and sorted as step 2 sorts failures. - Each story's acceptance criteria, checked against what that same code actually does, one row per criterion with its evidence.
Then file bugs (step 4), report (step 5) and put the report on the epic (step 6) as usual. Here a reproduction isn't a browser session: it's the command or test run that shows the failure, with its output (for an eval case, the failing grader and its verdict), and a signed-out check doesn't apply. A bug's regression test is a test in the team's form, created as the tests adapter's "Create a test" says (for eval cases, a drafted case on the bug).
Inputs
Ask in one message for anything missing:
- The epic key. Read its description first, especially any flow or behaviour table: that's the intended behaviour.
- What's out of scope: tests that target another environment, slices not built yet, and known gaps that aren't bugs.
- Feature flags this run expects, and their required values.
- The label for bugs found in this run (for example
signup-qa).
1. Check the environment
Everything runs locally, on the services in stack.services. Don't touch staging or production.
- Confirm every service answers. If one doesn't, start it as
stack.howsays (neverstack.never), and start any other resource the epic needs (stack.extra_resources). - Make sure the stack has the data the epic's flows need. If it doesn't, put it there with
stack.test_data, against the local stack only, and record what you loaded. - Read the flag state with
stack.flag_state. If it isn't what this run needs, stop and say so. - Check required test secrets are present in the e2e repo's
e2e.secrets_filewithout printing them. - Record the branch and short SHA of every repo in
repos. Don't switch branches. - Throughout the run, if the config has
stack.telemetry, read the stack's own errors: fromstack.owner_repo,stack.telemetry.logsandstack.telemetry.traces(stack.telemetry.note). A repeating error nobody triggered on purpose is a finding, even when no test failed.
2. Run the e2e suite and sort every failure
Run e2e.run_all, with the environment variables the flag state needs. Scenarios gated off by flag tags (for example stack.flag_tags) skipping is expected.
Put every failed scenario in one bucket before doing anything else:
- Product bug: the app does the wrong thing. Reproduce it by hand in the browser; file it if it reproduces.
- Test bug: the app is right and the test is wrong (stale selector, timing, old expectation). Don't file; report it with the fix you'd suggest.
- Environment problem: something local is missing (a service, credentials, data). Missing data you can load with
stack.test_dataisn't a reason to stop: load it and re-run. Don't file; report it. - Flaky: re-run that scenario alone up to twice (
e2e.run_tagswith"@<tag>"). If it passes, report it as flaky with the failure output.
A failure in an unrelated suite can still be this epic's regression if it happens at a step the epic changed (sign-in, for example). Check where it fails.
3. Explore
Use the Playwright browser tools against the app services in stack.services. Create throwaway accounts with the suite's email pattern (qa.test_account_email), never a real account.
Write charters from the epic's flows, highest release risk first: the new-user path, wrong-identity and error paths, deep links and old URLs, password and session flows, invitations, roles, and logout. Always include general hygiene on every screen the epic touches: console errors, failed network calls, redirect loops, layout at phone width, keyboard-only use, and error messages a normal person could understand. Go off-charter when something looks odd.
4. File bugs
Before filing, search for duplicates: the epic's existing bugs and a text search on the symptom. If one exists, comment with the new evidence instead. If a closed bug is back, file a new one linked as "relates to" and call it a regression.
Each bug:
- In the tracker's project, type Bug, parent the epic, the run's label, plus
qa.security_labelif it touches who's signed in, sessions, account takeover, or data leaking between users or tenants. Components set, whentracker.components_required. - Summary: what the user experiences, in plain words, not the internal cause.
- Description: what happened and what should have (cite the epic's flow table or story); numbered steps from a signed-out browser with exact URLs; environment (flags, branches, SHAs); evidence (screenshot, console errors, the failing request's method,