experiment-bridge
Workflow 1.5: Bridge between idea discovery and auto review. Reads EXPERIMENT_PLAN.md, implements experiment code, deploys to GPU, collects initial results. Use when user says \"实现实验\", \"implement experiments\", \"bridge\", \"从计划到跑实验\", \"deploy the plan\", or has an experiment plan ready to execute.
wanshuiyin · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill experiment-bridge技能说明
name: experiment-bridge description: "Workflow 1.5: Bridge between idea discovery and auto review. Reads EXPERIMENT_PLAN.md, implements experiment code, deploys to GPU, collects initial results. Use when user says "实现实验", "implement experiments", "bridge", "从计划到跑实验", "deploy the plan", or has an experiment plan ready to execute." argument-hint: "[experiment-plan-path-or-topic]" allowed-tools: Bash(*), Read, Write, Edit, Grep, Glob, Skill, mcp__codex__codex, mcp__codex__codex-reply
Workflow 1.5: Experiment Bridge
Implement and deploy experiments from plan: $ARGUMENTS
Overview
This skill bridges Workflow 1 (idea discovery + method refinement) and Workflow 2 (auto review loop). It takes the experiment plan and turns it into running experiments with initial results.
Workflow 1 output: This skill: Workflow 2 input:
refine-logs/EXPERIMENT_PLAN.md → implement → GPT-5.6-Sol review → deploy → collect → initial results ready
refine-logs/EXPERIMENT_TRACKER.md code (cross-model) /run-experiment for /auto-review-loop
refine-logs/FINAL_PROPOSAL.md
Constants
- CODE_REVIEW = true — GPT-5.6-Sol xhigh reviews experiment code before deployment. Catches logic bugs before wasting GPU hours. Set
falseto skip. - AUTO_DEPLOY = true — Automatically deploy experiments after implementation + review. Set
falseto manually inspect code before deploying. - SANITY_FIRST = true — Run the sanity-stage experiment first (smallest, fastest) before launching the rest. Catches setup bugs early.
- MAX_PARALLEL_RUNS = 4 — Maximum number of experiments to deploy in parallel (limited by available GPUs).
- BASE_REPO = false — GitHub repo URL to use as base codebase. When set, clone the repo first and implement experiments on top of it. When
false(default), write code from scratch or reuse existing project files. - COMPACT = false — When
true, (1) readidea-stage/IDEA_CANDIDATES.mdinstead of fullidea-stage/IDEA_REPORT.mdif available, (2) append experiment results toEXPERIMENT_LOG.mdafter collection.
Override:
/experiment-bridge "EXPERIMENT_PLAN.md" — compact: true, base repo: https://github.com/org/project
Inputs
This skill expects one or more of:
refine-logs/EXPERIMENT_PLAN.md(best) — claim-driven experiment roadmap from/experiment-planrefine-logs/EXPERIMENT_TRACKER.md— run-by-run execution tablerefine-logs/FINAL_PROPOSAL.md— method description for implementation contextidea-stage/IDEA_CANDIDATES.md— compact idea summary (preferred whenCOMPACT: true) (fall back to./IDEA_CANDIDATES.mdif not found)idea-stage/IDEA_REPORT.md— full brainstorm output (fall back to./IDEA_REPORT.mdif not found)
If none exist, ask the user what experiments to implement.
Workflow
Phase 1: Parse the Experiment Plan
Read EXPERIMENT_PLAN.md and extract:
- Run order and milestones — which experiments run first (sanity → baseline → main → ablation → polish)
- For each experiment block:
- Dataset / split / task
- Compared systems and variants
- Metrics to compute
- Setup details (backbone, hyperparameters, seeds)
- Success criterion
- Priority (MUST-RUN vs NICE-TO-HAVE)
- Compute budget — total estimated GPU-hours
- Method details from
FINAL_PROPOSAL.md— what exactly to implement
Present a brief summary:
📋 Experiment plan loaded:
- Milestones: [N] (sanity → baseline → main → ablation)
- Must-run experiments: [N]
- Nice-to-have: [N]
- Estimated GPU-hours: [X]
Proceeding to implementation.
Research-contract fallback: if idea-stage/docs/research_contract.md does
not exist yet (idea selected outside /idea-discovery, or an older run),
create it now from templates/RESEARCH_CONTRACT_TEMPLATE.md using the selected
idea + claims from the experiment plan. Downstream /result-to-claim and
/ablation-planner read this file as the claims source, and session recovery
(docs/SESSION_RECOVERY_GUIDE.md) depends on it existing.
Phase 2: Implement Experiment Code
If BASE_REPO is set — clone the repo first:
git clone <BASE_REPO> base_repo/
# Read the repo's README, understand its structure, find entry points
# Implement experiments by modifying/extending this codebase
For each milestone (in order), write the experiment scripts:
-
Check existing code — scan the project (or cloned
base_repo/) for existing experiment scripts, model code, data loaders. Reuse as much as possible. -
Implement missing pieces:
- Training scripts with proper argparse (all hyperparameters configurable)
- Evaluation scripts computing the specified metrics
- Data loading / preprocessing if needed
- Baseline implementations if not already present
- Fixed random seeds for reproducibility
- Results saved to JSON/CSV for later analysis
- Proper logging (wandb if configured in CLAUDE.md)
-
Follow the plan's run order — implement sanity-stage experiments first, then baselines, then main method, then ablations.
-
Self-review before deploying:
- Are all hyperparameters from EXPERIMENT_PLAN.md reflected in argparse?
- Is the random seed fixed and controllable?
- Are results saved in a parseable format (JSON/CSV)?
- Does the code match FINAL_PROPOSAL.md's method description?
Phase 2.5: Cross-Model Code Review (when CODE_REVIEW = true)
Skip this step if CODE_REVIEW is false.
Before deploying, send the experiment code to GPT-5.6-Sol xhigh for review:
mcp__codex__codex:
model: gpt-5.6-sol
config: {"model_reasoning_effort": "xhigh"}
prompt: |
Review the following experiment implementation for correctness.
## Experiment Plan:
[paste key sections from EXPERIMENT_PLAN.md]
## Method Description:
[paste from FINAL_PROPOSAL.md]
## Implementation:
[paste the experiment
阅读完整来源 ↗