quant-experiment-runtime
Quant research experiment executor: discover an offline source database under the workdir's code-repo, build a panel, run a Research Artifact's entry point to compute research-object values, and evaluate IC/ICIR/RANKIC/coverage metrics. Runtime = Experiment Executor; it runs a Research Artifact via a Python-native Entry Point and is agnostic to research-object type and expression form. Self-contained: panel building and IC metrics are implemented inside this skill. The dataset is identified at runtime by discover_data.py (never hard-coded). Use when: a research proposal is ready and you need to actually run the proposed factor/method on real data and get quantitative metrics. Do NOT use for: designing which experiments to run (use experiment-pipeline / paper-planning), debugging a single failed experiment (use experiment-craft), or searching papers (use local-paper-navigator).
CamusGIT · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add CamusGIT/EvoQuant --skill quant-experiment-runtime技能说明
name: quant-experiment-runtime description: "Quant research experiment executor: discover an offline source database under the workdir's code-repo, build a panel, run a Research Artifact's entry point to compute research-object values, and evaluate IC/ICIR/RANKIC/coverage metrics. Runtime = Experiment Executor; it runs a Research Artifact via a Python-native Entry Point and is agnostic to research-object type and expression form. Self-contained: panel building and IC metrics are implemented inside this skill. The dataset is identified at runtime by discover_data.py (never hard-coded). Use when: a research proposal is ready and you need to actually run the proposed factor/method on real data and get quantitative metrics. Do NOT use for: designing which experiments to run (use experiment-pipeline / paper-planning), debugging a single failed experiment (use experiment-craft), or searching papers (use local-paper-navigator)." allowed-tools: "write_file edit_file read_file think_tool execute" metadata: author: quant-research-team version: '1.0.0' tags: [core, experimentation, quant, runtime, executor, data, metrics]
Quant Research Experiment Runtime
An Experiment Executor for quant auto-research: take a Research Artifact
(LLM-generated, exposing a callable entry point), run it against a real
offline source database to compute research-object values, and evaluate
quantitative metrics. It does not design experiments (that is
experiment-pipeline) — it executes one.
Mental model: Runtime = Experiment Executor
Workflow (experiment-pipeline) ── owns when/whether to run
│
▼
Experiment Runtime ── owns how to run one experiment
│
▼
Research Artifact ── a runnable research product (py file / package / future workspace|docker|notebook)
│
▼
Entry Point ── Python-native callable, e.g. "path/to/code.py::run" or "pkg.mod:run"
│
▼
Results ── Runtime does NOT interpret; Metric does
│
▼
Metric (registry, extensible) ── evaluates; does NOT realign
│
▼
ExperimentResult (+ artifacts/) ── Reflection / downstream Workflow depend only on this
The Runtime only knows "I run a Research Artifact via its Entry Point." It is
agnostic to: research-object type (factor / generation method / portfolio),
expression form (DSL / python / generator), and the internal structure of
results. Those belong to the Research Artifact / Workflow / Metric.
This skill is self-contained: panel building (scripts/_panel.py) and IC
metrics (scripts/_metrics.py) are implemented inside the skill and need only
pandas / pyarrow / numpy. There is no dependency on any external
factor-research project.
When to use
- You need to actually run the proposed object on real data.
- Experiment needs concrete IC-style metrics.
- You need to evaluate a batch of candidates.
When NOT to use
- Designing which experiments to run / stage budgets →
experiment-pipeline. - Debugging a single failed experiment →
experiment-craft. - Searching/reading papers →
local-paper-navigator.
The convention that constrains LLM-generated code
LLM-generated research code is constrained by this convention, not by Python types:
A Research Artifact exposes one Entry Point — a callable with signature
entry(context, config) -> results. The function name is not fixed (run/experiment/evaluate/mainall fine);compute_refnames it.resultsdefault contract for the IC metric isdict[split, pd.Series]where each Series is factor exposure already aligned to that split's panel index — alignment is the research code's job, the Metric only evaluates.
See references/research-code-convention.md for the full contract and
assets/research-artifact-example/ for a runnable copy-pasteable example.
Database identification contract
The dataset is not hard-coded. The agent identifies it at runtime:
- Run
discover_data.py --code-repo code-repo→ a JSON catalog of every data package under the code-repo (each entry hasname,root,artifacts_root, file list, coverage). - Inspect the catalog, pick a dataset, and pass its
roottobuild_panel.py --data-root <root>. --data-root/--panelare required, with no default — the dataset name must come from the discover step, never hard-coded in a Candidate or script.
Paths follow the convention: script paths use the /skills/ virtual mount
(/skills/<skill>/scripts/..., resolved by the EvoQuant sandbox to the
installed skill directory — same convention used by paper-graph); data/output paths point into the EvoQuant workdir
and may be given relative to the workdir (with the workdir as cwd) or as
absolute paths. In docs, a leading / denotes the workdir root (e.g.
/code-repo/, /experiments/); in shell commands these are plain relative
paths (code-repo, experiments/...).
How to run (minimal demo)
Run with the EvoQuant workdir as cwd (the code-repo lives at code-repo
under it). Script paths use the /skills/ virtual mount, matching the convention (python /skills/<skill>/scripts/<x>.py); data/output
paths are relative to the workdir (cwd). Experiment outputs go under the
current cycle's project directory experiments/<project>/ (see
experiment-pipeline's "Project directory" convention — one per research cycle).
# 1. discover usable datasets under the code-repo (autonomous, no hard-coded names)
python /skills/quant-experiment-runtime/scripts/discover_data.py --code-repo code-repo --out catalog.json
# 2. from catalog.json pick datasets[i].root, then build the panel offline
python /skills/quant-experiment-runtime/scripts/build_panel.py \
--data-root <selected-dataset-root> \
--out experiments/<project>/panel_1d.parquet
# 3. run an experiment (single or batch Candidate JS
阅读完整来源 ↗