跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
开发与测试

experiment-bridge

Workflow 1.5: Bridge between idea discovery and auto review. Reads EXPERIMENT_PLAN.md, implements experiment code, deploys to GPU, collects initial results. Use when user says \"实现实验\", \"implement experiments\", \"bridge\", \"从计划到跑实验\", \"deploy the plan\", or has an experiment plan ready to execute.

wanshuiyin · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill experiment-bridge

技能说明


name: experiment-bridge description: "Workflow 1.5: Bridge between idea discovery and auto review. Reads EXPERIMENT_PLAN.md, implements experiment code, deploys to GPU, collects initial results. Use when user says "实现实验", "implement experiments", "bridge", "从计划到跑实验", "deploy the plan", or has an experiment plan ready to execute." argument-hint: "[experiment-plan-path-or-topic]" allowed-tools: Bash(*), Read, Write, Edit, Grep, Glob, Skill, mcp__codex__codex, mcp__codex__codex-reply

Workflow 1.5: Experiment Bridge

Implement and deploy experiments from plan: $ARGUMENTS

Overview

This skill bridges Workflow 1 (idea discovery + method refinement) and Workflow 2 (auto review loop). It takes the experiment plan and turns it into running experiments with initial results.

Workflow 1 output:                    This skill:                                    Workflow 2 input:
refine-logs/EXPERIMENT_PLAN.md   →   implement → GPT-5.6-Sol review → deploy → collect → initial results ready
refine-logs/EXPERIMENT_TRACKER.md     code        (cross-model)    /run-experiment     for /auto-review-loop
refine-logs/FINAL_PROPOSAL.md

Constants

  • CODE_REVIEW = true — GPT-5.6-Sol xhigh reviews experiment code before deployment. Catches logic bugs before wasting GPU hours. Set false to skip.
  • AUTO_DEPLOY = true — Automatically deploy experiments after implementation + review. Set false to manually inspect code before deploying.
  • SANITY_FIRST = true — Run the sanity-stage experiment first (smallest, fastest) before launching the rest. Catches setup bugs early.
  • MAX_PARALLEL_RUNS = 4 — Maximum number of experiments to deploy in parallel (limited by available GPUs).
  • BASE_REPO = false — GitHub repo URL to use as base codebase. When set, clone the repo first and implement experiments on top of it. When false (default), write code from scratch or reuse existing project files.
  • COMPACT = false — When true, (1) read idea-stage/IDEA_CANDIDATES.md instead of full idea-stage/IDEA_REPORT.md if available, (2) append experiment results to EXPERIMENT_LOG.md after collection.

Override: /experiment-bridge "EXPERIMENT_PLAN.md" — compact: true, base repo: https://github.com/org/project

Inputs

This skill expects one or more of:

  1. refine-logs/EXPERIMENT_PLAN.md (best) — claim-driven experiment roadmap from /experiment-plan
  2. refine-logs/EXPERIMENT_TRACKER.md — run-by-run execution table
  3. refine-logs/FINAL_PROPOSAL.md — method description for implementation context
  4. idea-stage/IDEA_CANDIDATES.md — compact idea summary (preferred when COMPACT: true) (fall back to ./IDEA_CANDIDATES.md if not found)
  5. idea-stage/IDEA_REPORT.md — full brainstorm output (fall back to ./IDEA_REPORT.md if not found)

If none exist, ask the user what experiments to implement.

Workflow

Phase 1: Parse the Experiment Plan

Read EXPERIMENT_PLAN.md and extract:

  1. Run order and milestones — which experiments run first (sanity → baseline → main → ablation → polish)
  2. For each experiment block:
    • Dataset / split / task
    • Compared systems and variants
    • Metrics to compute
    • Setup details (backbone, hyperparameters, seeds)
    • Success criterion
    • Priority (MUST-RUN vs NICE-TO-HAVE)
  3. Compute budget — total estimated GPU-hours
  4. Method details from FINAL_PROPOSAL.md — what exactly to implement

Present a brief summary:

📋 Experiment plan loaded:
- Milestones: [N] (sanity → baseline → main → ablation)
- Must-run experiments: [N]
- Nice-to-have: [N]
- Estimated GPU-hours: [X]

Proceeding to implementation.

Research-contract fallback: if idea-stage/docs/research_contract.md does not exist yet (idea selected outside /idea-discovery, or an older run), create it now from templates/RESEARCH_CONTRACT_TEMPLATE.md using the selected idea + claims from the experiment plan. Downstream /result-to-claim and /ablation-planner read this file as the claims source, and session recovery (docs/SESSION_RECOVERY_GUIDE.md) depends on it existing.

Phase 2: Implement Experiment Code

If BASE_REPO is set — clone the repo first:

git clone <BASE_REPO> base_repo/
# Read the repo's README, understand its structure, find entry points
# Implement experiments by modifying/extending this codebase

For each milestone (in order), write the experiment scripts:

  1. Check existing code — scan the project (or cloned base_repo/) for existing experiment scripts, model code, data loaders. Reuse as much as possible.

  2. Implement missing pieces:

    • Training scripts with proper argparse (all hyperparameters configurable)
    • Evaluation scripts computing the specified metrics
    • Data loading / preprocessing if needed
    • Baseline implementations if not already present
    • Fixed random seeds for reproducibility
    • Results saved to JSON/CSV for later analysis
    • Proper logging (wandb if configured in CLAUDE.md)
  3. Follow the plan's run order — implement sanity-stage experiments first, then baselines, then main method, then ablations.

  4. Self-review before deploying:

    • Are all hyperparameters from EXPERIMENT_PLAN.md reflected in argparse?
    • Is the random seed fixed and controllable?
    • Are results saved in a parseable format (JSON/CSV)?
    • Does the code match FINAL_PROPOSAL.md's method description?

Phase 2.5: Cross-Model Code Review (when CODE_REVIEW = true)

Skip this step if CODE_REVIEW is false.

Before deploying, send the experiment code to GPT-5.6-Sol xhigh for review:

mcp__codex__codex:
  model: gpt-5.6-sol
  config: {"model_reasoning_effort": "xhigh"}
  prompt: |
    Review the following experiment implementation for correctness.

    ## Experiment Plan:
    [paste key sections from EXPERIMENT_PLAN.md]

    ## Method Description:
    [paste from FINAL_PROPOSAL.md]

    ## Implementation:
    [paste the experiment
阅读完整来源 ↗