experiment-planner
Use when exploring, continuing, or accepting a deep-learning or computer-science research idea. Converts claims into pilot-first experiment matrices covering diagnostics, failure analysis, authorized execution, evaluation, and paper-story viability.
sidiangongyuan · 已有静态检查记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add sidiangongyuan/codex-skills-library --skill experiment-planner技能说明
name: experiment-planner description: Use when exploring, continuing, or accepting a deep-learning or computer-science research idea. Converts claims into pilot-first experiment matrices covering diagnostics, failure analysis, authorized execution, evaluation, and paper-story viability. license: MIT
Experiment Planner
Overview
Use this skill when the user needs to explore a research idea, continue an authorized experiment workflow, or accept and interpret results. It turns claims into a testable story and experiment plan. It is an adapter over existing research-agent ideas, not a replacement for the user's writing, review, rebuttal, figure, evidence, or GitHub release skills.
Core Boundaries
- Default domain: general deep learning and computer science research. Adapt to collaborative perception, 3D perception, or autonomous driving only when the task context calls for it.
- For discussion-only planning, keep output in chat unless the user asks for a
saved artifact. When the user asks to implement or run experiments in a
repository, persist the pre-run result contract: update the paper's final
LaTeX tables when a manuscript is in scope; otherwise update the existing
experiment-planning document or create
experiment-plan.mdat the repository root. - Do not launch long experiments, deploy GPU jobs, modify code, or retry failed runs unless the user explicitly asks for execution or a still-valid approval covers the ordinary continuation. A bounded same-config recovery stays within that approval; do not silently expand its scope.
- Do not replace
paper-writing,paper-review-panel,rebuttal-response-skills,paper-visual-craft, orgithub-project-release; hand off to them only after the research plan or results are ready. - Treat external projects as references, not installed dependencies. Read
references/source-map.mdbefore discussing provenance or upgrading this skill from upstream sources. - During approved execution, a still-valid earlier user authorization covers ordinary implementation, local bug fixes, queueing in-scope runs, and evaluation. It does not authorize a new experiment family, dataset, seed policy, or paper-only TODO; those remain explicit opt-ins.
Entry Modes
Choose the mode that matches the current request:
- Exploration: turn a new research direction into a falsifiable claim. Use the complete matrix for a substantial new study; a small pilot can use the minimal question/comparison/protocol-plus-budget/endpoint/decision contract.
- Approved execution: continue within the frozen contract and the scope of a still-valid user authorization. Do not silently expand the experiment family, dataset, seed policy, or resource budget.
- Acceptance: compare observed artifacts with the frozen contract and record whether the relevant gate passes, fails, or is inconclusive.
For a continuation, load the latest plan, results, and pass receipt when available. Reuse still-valid decisions about the claim, baseline, metrics, budget, seed, and protocol. Do not restart the initial grill, literature pass, or full matrix unless a material change or unresolved blocker makes it necessary; ask only questions that affect the next decision.
Default Workflow
- Grill consensus: for a substantial new exploration, use
$grill-mestyle interaction to clarify problem, motivation, proposed claim, baseline/control, compute budget, success criteria, and unacceptable shortcuts. For a small pilot, record only the minimal contract. For a continuation, reuse the settled contract and ask only changed or blocking questions. - Literature inspiration: for a substantial new exploration or a changed
novelty claim, use
$research-evidencefor related papers, novelty risk, prior experiment patterns, and unsupported claims. Use$search-firstwhen the task may need existing code, datasets, tools, or implementations. A small pilot may defer this pass when it does not change the decision. Reuse prior literature decisions when the claim and scope are unchanged. - Story viability check: decide whether the idea can support a clean paper story: important problem, credible gap, specific method difference, feasible validation, and claims that will not outrun the evidence.
- Claim freeze: freeze the smallest verifiable claim before planning runs. Avoid changing the story repeatedly while experiments are running.
- Contract freeze: before scheduling a substantial study or full runs, define the final main-result, ablation, and necessary diagnostic tables. For every metric, record its plain-language definition, unit, direction, aggregation, and any delta reference. For a small pilot, freeze only the endpoint, comparison, protocol-plus-budget, and decision gate. Use the in-scope project's explicit missing-value schema and do not write claims from placeholder cells.
- Idea validation first: design the smallest pilot/smoke/sanity experiment that can falsify or support the core hypothesis. If multiple GPUs are idle, parallelize only independent exploration runs with clear ownership.
- Minimum sufficient matrix: only after the pilot passes, add the main result and claim-critical ablations. Add robustness, diagnostics, efficiency, qualitative results, or failure analysis only when they support a paper claim or answer a credible reviewer question; do not add them for symmetry.
- Subagent coordination: keep the main session responsible for planning,
task decomposition, and final result acceptance. Use
explorerfor read-only repo/config/protocol investigation. Useworkerfor implementation with explicit file or module ownership. Do not manually override subagent model or reasoning settings unless the user explicitly requests it. - Impact-based validation: prioritize checks by their ability to block the next d