evaluating-gameplay-balance
Evaluates and improves gameplay balance from telemetry in any engine. Use when comparing monotonous vs exploratory play, diagnosing death/spawn/scoring/input issues, setting difficulty, timer, or pacing targets with human-limited simulated players, or proposing structural balance fixes instead of numeric tuning.
abagames · Static checks on file
Installation
Read the source and dependencies before running a command. Copying does not install or execute anything.
npx skills add abagames/agentic-gamedev-skills --skill evaluating-gameplay-balanceAbout this skill
name: evaluating-gameplay-balance description: "Evaluates and improves gameplay balance from telemetry in any engine. Use when comparing monotonous vs exploratory play, diagnosing death/spawn/scoring/input issues, setting difficulty, timer, or pacing targets with human-limited simulated players, or proposing structural balance fixes instead of numeric tuning."
Use this skill to analyze whether a game rewards skillful play and to propose structural improvements.
Engine-neutral contract:
- Produce comparable runs for monotonous policies and exploratory policies.
- Define the public input schema and visible-state features available to policies.
- Keep seeds, tick rate, max duration, policy definitions, search budget, and aggregation logic comparable across runs.
- Record score, elapsed time, end state, and telemetry for death, spawn, scoring, and input behavior.
- Compute
exploratory_ratio = exploratory.best.score / monotonous.max_score. - Treat the ratio as a quality detector, not an optimization target.
- Declare an execution profile for every non-baseline policy as well as its visible-state features:
oracle(hidden information allowed; upper bound),precise(visible state, near-perfect timing and attention), orhuman-limited(visible state plus the human limits inreferences/simulation-harness.md). Compute the ratio fromprecisepolicies (search-based runners included, unless labeledoracle); it detects exploits, not difficulty. - Set difficulty, timers, pacing, and progression targets only from
human-limitedruns, and label every conclusion with the profile it rests on. Without a human-limited run, do not state a difficulty or pacing verdict — report it as not yet measured. - A claim that a strategy dominates, or that a trade-off is balanced, names the profiles it was tested on. If the ranking flips between profiles, that alone is not a balance defect — report it as skill-dependent, but still apply the experience guardrails to each profile's best strategy (a human-limited optimum of idling or waiting is a defect).
Non-deterministic harnesses:
The contract above assumes a seeded forward model. Some rigs cannot provide one — a real browser driven over a wall-clock window, an engine whose RNG you do not control, a policy comparison run against live rendering. Seeding is still the first choice; when it is genuinely unavailable, the following replaces it, and skipping it produces confident conclusions from noise:
- Report a min–max band over n ≥ 3 runs per policy, never a point estimate. A single run of a long-tailed distribution proves nothing about either policy.
- To claim a band has moved, require n ≥ 5 and no overlap with the recorded prior band. A sample that overlaps the previous band is not a finding, however plausible the mechanism that would explain it.
- Record the accepted band and its known tails in the project's spec, so later runs compare against history instead of rediscovering the same spread. If a policy is known to land near a bad-looking value in some fraction of runs (e.g. "~20 % of idle runs reach 0.18 of skilled score, by design, because the field can collapse on its own"), that tail is documented and settled — not re-litigated every time it appears.
- Retraction is a valid and expected output. A conclusion published from a small sample that a larger sample does not support should be withdrawn in place: keep the mechanism if it was real, and state that its effect size was indistinguishable from noise. Observed case: a band shift published at n=4, with a coherent causal story attached, was retracted at n=6 as the same spread the project had already recorded twice.
Experience guardrails:
- Reject changes that degrade play experience even if metrics improve.
- Score only in-game causal events; do not award points for raw input facts.
- Game-over should be tied to hazards or world-state collapse.
- Do not add hidden behavior that only helps or hurts test agents.
- Avoid numeric-only tuning, branch-only fixes, and added randomness as the primary answer.
- Treat hands-on play reports as calibration data for the human-limited profile. When a report conflicts with simulated results, first check the harness for defects, then adjust the profile toward the report, re-run, and record the adjustment; do not dismiss the report with bot data alone.
- Do not tune a parameter with a policy whose behavior is defined by that parameter (e.g. "wait while the timer is above 80 %"); the measurement moves with the tuning.
Workflow:
- Locate an existing simulation harness. If none exists, design one using the harness contract before judging balance.
- Choose the comparison protocol:
- If seeding is available, run comparable monotonous and exploratory policies with the same deterministic seeds.
- If seeding is genuinely unavailable, use the non-deterministic band protocol above and keep run count, wall-clock window, policy definitions, and sampling cadence comparable.
- Validate that the report includes run configuration plus death, spawn, scoring, and input telemetry.
- If telemetry is incomplete, report the gap and request instrumentation/rerun instead of judging balance from score alone.
- Sanity-check non-trivial policies before analyzing the game (see Simulated-player sanity checks in
references/simulation-harness.md). A failed check is a harness defect: fix the policy, re-run, and report the defect instead of drawing conclusions from it. - Analyze death, spawn, scoring, and input patterns.
- Identify root causes in rules or generation logic.
- Propose at least three candidate fixes with expected impact, risk, and complexity.
- Re-test with the same policies, budgets, aggregation, and comparison protocol after implementation. Preserve the same seeds for deterministic runs; preserve the same run count and sampling setup for non-deterministic runs.
Project checker triage, when the report includes ratio.diagnostic:
- Bef