exploratory-autoresearch
Use when the user wants an autonomous ML research loop that explores the space broadly rather than hill-climbing one approach. A temperature scheduler replaces the usual hypothesis step: it forces several wild, diverse swings (full rewrites, different architectures/regimes) early, then enters an adaptive phase that picks swing / merge / exploit per iteration — with a hard stagnation guard that bans further small-step exploits once they run too long, forcing a pivot back to a swing or merge. Tracks an approaches.md registry and a move_type per iteration; analyses every run before the next move. One change per iteration; loops forever until interrupted. Not for the standard analysis-first ml-autoresearch (which lets analysis alone choose each change), one-off training runs, or sweeps.
gaasher · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add gaasher/Agent-Loop-Skills --skill exploratory-autoresearch技能说明
name: exploratory-autoresearch description: > Use when the user wants an autonomous ML research loop that explores the space broadly rather than hill-climbing one approach. A temperature scheduler replaces the usual hypothesis step: it forces several wild, diverse swings (full rewrites, different architectures/regimes) early, then enters an adaptive phase that picks swing / merge / exploit per iteration — with a hard stagnation guard that bans further small-step exploits once they run too long, forcing a pivot back to a swing or merge. Tracks an approaches.md registry and a move_type per iteration; analyses every run before the next move. One change per iteration; loops forever until interrupted. Not for the standard analysis-first ml-autoresearch (which lets analysis alone choose each change), one-off training runs, or sweeps. compatibility: Requires Python 3.9+ metadata: version: "0.1.0"
Exploratory Autoresearch Loop
This loop runs hot. Like the standard ml-autoresearch, every experiment is followed by a diagnostic
analysis pass. Unlike it, the type of change at each iteration is set by a temperature
scheduler, not the agent's intuition: it forces wide, diverse swings early (full rewrites,
fundamentally different architectures and training regimes), then drops into an adaptive phase that
chooses between swing (a fresh wild approach), merge (combine two registered approaches), or
exploit (a focused tweak of the best). A stagnation guard bans exploit once it has run
<stagnation_limit> times in a row, forcing a pivot back to swing or merge so the loop never gets
stuck hill-climbing. The feedback signal is <metric> read from the run log; an approaches.md
registry and a move_type per iteration are what make the scheduler work.
You are the researcher. Do not pause to ask for permission once the loop is running.
When to use
Use for an open-ended ML campaign where you want forced breadth before refinement — the scheduler
guarantees you sample several distinct families before converging, and the stagnation guard prevents
endless small steps. Default to <swing_budget> = 3 and <stagnation_limit> = 3; raise <swing_budget>
for wider initial exploration. Not for the standard analysis-first ml-autoresearch (use that when you
want the analysis alone to drive each change, with no forced-swing scheduler), not for a single training
run or a fixed sweep, and not for tasks with no measurable scalar metric.
Setup
Resolve bindings interactively. If loop.run.yaml exists in the working dir, load it, confirm the
values in one line, and skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is
available — record <host> = claude-code) infer a likely value for each binding from the project and
present it as the recommended option; on other hosts (<host> = other) ask each as a quoted plain-text
prompt. Then write loop.run.yaml (format: examples/run.example.yaml) and confirm every value with
the user before creating any other files. For branches strategy, create
git checkout -b autoresearch/<run_tag> (tag from today's date; branch must not exist). For time
gating, write <sandbox_root>/run_with_timeout.sh (timeout $(( <budget> * 60 )) <entrypoint> "$@") and
use it as the run command, hard-killing at 2 × <budget> min; for epochs, patch the epoch cap in an
<editable_files> file.
| binding | meaning | default | how to infer |
|---|---|---|---|
<metric> / <metric_direction> | scalar to optimize + minimize/maximize | — | scan editable files + README for metric names |
<run_cmd> / <entrypoint> | command that runs one experiment end to end | — | pyproject.toml / .venv / README |
<editable_files> | files fair game to edit (never the eval harness) | — | model / config / train scripts; exclude data, logs, env, harness |
<sandbox_root> | where snapshots + ledgers live | ./sandbox | next to the editable files |
<iter_strategy> | snapshots or branches | snapshots | is the working dir a clean git repo? |
<gate> / <budget> | time (min) or epochs, and the limit | — | existing time/epoch settings in config |
<swing_budget> | forced wild swings before adaptive mode | 3 (3–5) | wider = more initial breadth |
<stagnation_limit> | max consecutive exploits before a forced pivot | 3 | — |
FILE EDIT GUARD: before touching any file at any point — setup or loop — confirm it is in
<editable_files>, because everything else is read-only ground truth (the eval harness defines
<metric>). No exceptions.
Initialise the sandbox
Create the layout and write the ledger headers:
<sandbox_root>/
├── loop.run.yaml ← resolved bindings (written now)
├── results.tsv ← experiment ledger, header only (written now)
├── approaches.md ← registry of every distinct approach (header only, written now)
└── iter1/ ← created at loop start
results.tsv header (tab-separated; move_type ∈ {swing, merge, exploit}):
iter <metric> status move_type analysis_summary description
approaches.md header: # Approach Registry plus a one-line note that the merge step consults it to
find complementary approaches to combine.
The loop (LOOP FOREVER — until interrupted)
Iteration 1 is always the unmodified baseline (it does not count as a swing): skip move-selection and
change-planning, but still run the mandatory analysis — it is the first empirical anchor iteration 2
builds on. Everything in <editable_files> is fair game (architecture, optimizer, hyperparameters, data
pipeline, loss, init, eval); on swings especially, full rewrites are encouraged. The only constraints are
that the code runs and finishes within <budget>. Epoch efficiency is part of the objective — a
change that reaches the same score in fewer effective steps is a real win. Simplicity criterion: all
else equal, simpler is better — a 0.