Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Finance

quant-experiment-runtime

Quant research experiment executor: discover an offline source database under the workdir's code-repo, build a panel, run a Research Artifact's entry point to compute research-object values, and evaluate IC/ICIR/RANKIC/coverage metrics. Runtime = Experiment Executor; it runs a Research Artifact via a Python-native Entry Point and is agnostic to research-object type and expression form. Self-contained: panel building and IC metrics are implemented inside this skill. The dataset is identified at runtime by discover_data.py (never hard-coded). Use when: a research proposal is ready and you need to actually run the proposed factor/method on real data and get quantitative metrics. Do NOT use for: designing which experiments to run (use experiment-pipeline / paper-planning), debugging a single failed experiment (use experiment-craft), or searching papers (use local-paper-navigator).

CamusGIT · AI review on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add CamusGIT/EvoQuant --skill quant-experiment-runtime

About this skill


name: quant-experiment-runtime description: "Quant research experiment executor: discover an offline source database under the workdir's code-repo, build a panel, run a Research Artifact's entry point to compute research-object values, and evaluate IC/ICIR/RANKIC/coverage metrics. Runtime = Experiment Executor; it runs a Research Artifact via a Python-native Entry Point and is agnostic to research-object type and expression form. Self-contained: panel building and IC metrics are implemented inside this skill. The dataset is identified at runtime by discover_data.py (never hard-coded). Use when: a research proposal is ready and you need to actually run the proposed factor/method on real data and get quantitative metrics. Do NOT use for: designing which experiments to run (use experiment-pipeline / paper-planning), debugging a single failed experiment (use experiment-craft), or searching papers (use local-paper-navigator)." allowed-tools: "write_file edit_file read_file think_tool execute" metadata: author: quant-research-team version: '1.0.0' tags: [core, experimentation, quant, runtime, executor, data, metrics]

Quant Research Experiment Runtime

An Experiment Executor for quant auto-research: take a Research Artifact (LLM-generated, exposing a callable entry point), run it against a real offline source database to compute research-object values, and evaluate quantitative metrics. It does not design experiments (that is experiment-pipeline) — it executes one.

Mental model: Runtime = Experiment Executor

Workflow (experiment-pipeline)   ── owns when/whether to run
        │
        ▼
Experiment Runtime               ── owns how to run one experiment
        │
        ▼
Research Artifact                ── a runnable research product (py file / package / future workspace|docker|notebook)
        │
        ▼
Entry Point                      ── Python-native callable, e.g. "path/to/code.py::run" or "pkg.mod:run"
        │
        ▼
Results                          ── Runtime does NOT interpret; Metric does
        │
        ▼
Metric (registry, extensible)    ── evaluates; does NOT realign
        │
        ▼
ExperimentResult (+ artifacts/) ── Reflection / downstream Workflow depend only on this

The Runtime only knows "I run a Research Artifact via its Entry Point." It is agnostic to: research-object type (factor / generation method / portfolio), expression form (DSL / python / generator), and the internal structure of results. Those belong to the Research Artifact / Workflow / Metric.

This skill is self-contained: panel building (scripts/_panel.py) and IC metrics (scripts/_metrics.py) are implemented inside the skill and need only pandas / pyarrow / numpy. There is no dependency on any external factor-research project.

When to use

  • You need to actually run the proposed object on real data.
  • Experiment needs concrete IC-style metrics.
  • You need to evaluate a batch of candidates.

When NOT to use

  • Designing which experiments to run / stage budgets → experiment-pipeline.
  • Debugging a single failed experiment → experiment-craft.
  • Searching/reading papers → local-paper-navigator.

The convention that constrains LLM-generated code

LLM-generated research code is constrained by this convention, not by Python types:

A Research Artifact exposes one Entry Point — a callable with signature entry(context, config) -> results. The function name is not fixed (run/experiment/evaluate/main all fine); compute_ref names it. results default contract for the IC metric is dict[split, pd.Series] where each Series is factor exposure already aligned to that split's panel index — alignment is the research code's job, the Metric only evaluates.

See references/research-code-convention.md for the full contract and assets/research-artifact-example/ for a runnable copy-pasteable example.

Database identification contract

The dataset is not hard-coded. The agent identifies it at runtime:

  1. Run discover_data.py --code-repo code-repo → a JSON catalog of every data package under the code-repo (each entry has name, root, artifacts_root, file list, coverage).
  2. Inspect the catalog, pick a dataset, and pass its root to build_panel.py --data-root <root>.
  3. --data-root / --panel are required, with no default — the dataset name must come from the discover step, never hard-coded in a Candidate or script.

Paths follow the convention: script paths use the /skills/ virtual mount (/skills/<skill>/scripts/..., resolved by the EvoQuant sandbox to the installed skill directory — same convention used by paper-graph); data/output paths point into the EvoQuant workdir and may be given relative to the workdir (with the workdir as cwd) or as absolute paths. In docs, a leading / denotes the workdir root (e.g. /code-repo/, /experiments/); in shell commands these are plain relative paths (code-repo, experiments/...).

How to run (minimal demo)

Run with the EvoQuant workdir as cwd (the code-repo lives at code-repo under it). Script paths use the /skills/ virtual mount, matching the convention (python /skills/<skill>/scripts/<x>.py); data/output paths are relative to the workdir (cwd). Experiment outputs go under the current cycle's project directory experiments/<project>/ (see experiment-pipeline's "Project directory" convention — one per research cycle).

# 1. discover usable datasets under the code-repo (autonomous, no hard-coded names)
python /skills/quant-experiment-runtime/scripts/discover_data.py --code-repo code-repo --out catalog.json

# 2. from catalog.json pick datasets[i].root, then build the panel offline
python /skills/quant-experiment-runtime/scripts/build_panel.py \
  --data-root <selected-dataset-root> \
  --out experiments/<project>/panel_1d.parquet

# 3. run an experiment (single or batch Candidate JS
Read the complete source ↗