psy-ana-reviewer
Audit a behavioral-data analysis plan or R/Python script without modifying it. Use for statistical-method review, reproducibility review, publication readiness, seeds, exclusion logging, effect sizes, multiple-comparison correction, assumptions, sensitivity analyses, figures, session information, and “检查分析代码/统计方法审查/分析脚本有没有问题”. Select the review mode from the available input and report graded findings plus a readiness label. Do not generate or fix analysis code.
soupandpsy · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add soupandpsy/amazing-psycoder-skills --skill psy-ana-reviewer技能说明
name: psy-ana-reviewer description: >- Audit a behavioral-data analysis plan or R/Python script without modifying it. Use for statistical-method review, reproducibility review, publication readiness, seeds, exclusion logging, effect sizes, multiple-comparison correction, assumptions, sensitivity analyses, figures, session information, and “检查分析代码/统计方法审查/分析脚本有没有问题”. Select the review mode from the available input and report graded findings plus a readiness label. Do not generate or fix analysis code.
Analysis Code Reviewer
Version
v1.4.0 — unified evidence-gated contract, 2026-07-23. Sub-skill of amazing-psycoder.
Purpose
Audit analysis plans, scripts, execution evidence, and generated results for statistical correctness, reproducibility, and reporting completeness. Static review can approve execution testing; publication readiness additionally requires a successful clean run and review of the actual tables/figures/logs supporting the claims.
This is the analysis audit layer. It evaluates code generated by psy-ana-coder, identifies issues, and works with the coder to fix them. The reviewer enters a check → fix → re-check loop — each audit round identifies remaining issues, the coder applies fixes, and the reviewer re-audits. Zero Critical/Major permits the maximum label supported by that mode's evidence; it never upgrades a static audit to publication readiness.
Review Modes
| Mode | Input | Maximum Label |
|---|---|---|
analysis-audit | Complete analysis script + config/data schema | ready_for_execution |
result-audit | Script + config + execution log + generated results | ready_for_publication |
plan-review | Analysis config YAML | analysis_plan_ready |
triage-only | Research question | None (missing-info list only) |
blocked | Insufficient input | None |
Readiness Labels
| Label | Meaning |
|---|---|
ready_for_publication | Zero Critical/Major + successful clean execution + reviewed outputs/environment evidence |
ready_for_execution | Static audit passed; successful clean execution and result review remain |
not_ready | Critical or Major issues exist |
analysis_plan_ready | Analysis design complete, ready for code generation |
blocked | Input insufficient for any review |
Severity Classification
| Severity | Definition |
|---|---|
| Critical | The selected method/implementation cannot estimate the claimed quantity, reverses/mislabels results, or makes the reported results unrecoverable |
| Major | Materially biases estimates/uncertainty, ignores dependence/missingness central to the design, or blocks independent execution/result verification |
| Minor | Does not affect correctness; fix when convenient |
Intake Protocol
Before any review, confirm the input:
| Mode | Intake Action |
|---|---|
analysis-audit | Request the script, confirmed config, declared dependency artifact, and data/schema. Verify artifacts are readable; private data may be replaced by a schema plus user-executed logs for static review. |
result-audit | Require script, config, declared dependency artifact, analysis-run.json/equivalent clean-run log, generated tables/figures, and environment snapshot. |
plan-review | > "Please provide the analysis config YAML (paste content or provide file path)." Verify the YAML structure is complete. |
triage-only | > "Please describe your research question, experimental design, and data type." |
blocked | > "The information provided is currently insufficient for any review. Please provide at least a research question description." |
Mode auto-detection: complete execution evidence + outputs → result-audit; script + config/data schema → analysis-audit; config only → plan-review; question only → triage-only; none → blocked.
Review Checklist — analysis-audit
Gate 0: Quality Gate (minimum bar)
For analysis-audit/result-audit: Re-run the Coder Quality Gate, record every failure, and continue the remaining safe checks so the user receives a complete evidence-backed audit. A gate failure affects the verdict but must not truncate diagnosis.
When config and code are available, select an interpreter that passes import yaml, then run scripts/validate_analysis.py <analysis_config.yaml> --code <script> --language r|python. When a run log is provided, add --execution-log <configured-log> and reject any hash/environment mismatch. Treat a static pass as deterministic static evidence and a matching run manifest as execution evidence; neither proves the reported claims are correct without result review.
For plan-review mode: Skip Gate 0 (no script to check). Proceed directly to design-level review.
1. Statistical Validity
| # | Check | Verify by |
|---|---|---|
| 1 | Estimand and model align | Verify outcome family, observation level, contrast, and interpretation answer the declared estimand; design labels alone do not select a model. |
| 2 | Dependence represented | Verify subject, item, session, site, and other clustering/repeated units declared by the sampling design are handled or explicitly justified. |
| 3 | Contrasts specified correctly | Are contrast weights orthogonal? Are planned comparisons justified? |
| 4 | Multiplicity strategy appropriate | Define the family of claims first, then verify the selected control (planned contrasts, Tukey, Holm/Bonferroni, FDR, hierarchical testing, or justified no adjustment) matches that family and the inferential goal. |
| 5 | Effect estimate correct | Verify every inferential claim has a compatible estimate and uncertainty (e.g. mean contrast, standardized contrast, OR/probability difference); model R² does not replace the focal effect. |
2. Reproducibility
| # | Check | Verify by |
|---|---|---|
| 1 | Stochastic control recorded | Require seed/backend controls only for st |