Skip to content
OpenAgentSkill
Browsing is open · Catalog captured 2026-11-21 · Automatic updates and account services are paused.
Development

theory-diagnostics

Diagnostics for formal-theory results — theorems, propositions, proofs, derivations, closed-form claims. Re-derives from the paper's own primitives, constructs counterexamples, stress-tests the assumptions, and checks that each proof delivers exactly what it claims. Invoke from `code-and-data` whenever a load-bearing result rests on a proof or derivation rather than an estimate. A kill here is a construction, never an assertion.

isitcredible · Static checks on file

Installation

Read the source and dependencies before running a command. Copying does not install or execute anything.

npx skills add isitcredible/research_skills --skill theory-diagnostics

About this skill


name: theory-diagnostics description: >- Diagnostics for formal-theory results — theorems, propositions, proofs, derivations, closed-form claims. Re-derives from the paper's own primitives, constructs counterexamples, stress-tests the assumptions, and checks that each proof delivers exactly what it claims. Invoke from code-and-data whenever a load-bearing result rests on a proof or derivation rather than an estimate. A kill here is a construction, never an assertion.

Theory diagnostics

The empirical modules break a result by running the regression that kills it. A formal result has no regression: its kill is a construction — a concrete counterexample, a numerical simulation of the paper's own model, or a re-derivation from the paper's own primitives that lands somewhere other than the claim. This module is the battery's "run it" ethos applied to a proof. The object changes; the discipline does not: you may not call a theorem false by argument alone. Build the counterexample, run the simulation, or carry the derivation through — then report what it shows.

Trigger: any target result whose truth rests on a theorem, proposition, lemma, proof, derivation, or closed-form claim — including the model section of an otherwise-empirical (hybrid) paper, where the estimation inherits its meaning from a model that must itself hold. NOT_APPLICABLE when every target result is an estimate and no formal claim is load-bearing; say so in a one-line stub naming the estimates and move on. Do not manufacture a proof audit on a paper that has no theory.

The one failure this module exists to prevent

"Re-derive it yourself and see if it breaks" is a strawman generator the moment your premise is wrong. Give a symbol a meaning the paper never assigned it, or attack a step the paper never took, and you can derive a rigorous-looking contradiction from your own fabrication — and because the fabricated premise is non-reproducible, a second pass "confirms" it by construction. This is not hypothetical: it is the exact way a confident, false "the method is mathematically invalid" kill gets shipped. The structure below is built around that hazard. Do not collapse it into a single pass.

Operate in three separated roles — never one context

Run these as distinct jobs (separate subagents where the harness supports them; otherwise separate, dedicated passes each starting from a clean read, and the module report says which way the roles ran). The attacker must not rule on its own findings, and the judge must not run a fresh derivation from memory. Segregation is the mechanism, not decoration.

1. Attacker — try to break each load-bearing result

Spend the majority of effort on the core: the theorems, propositions, proofs, and definitions are the paper. A pass whose findings all sit in the periphery while the core was never seriously attacked is a coverage failure, not a verdict — but a core that was seriously attacked and held is a survived result, not a missing issue. The construction ledger separates the two: attacked-and-held is evidence; never-attacked is a hole. For each load-bearing result:

  • Re-derive from the paper's own stated premises and definitions. Check every skipped step and every "it follows trivially." When the text says an expression "is," "corresponds to," or "is the [named object] of" something — build that object independently and compare term by term; a constant-factor mismatch (the factor-of-2 that nobody re-derives) is a finding. For algebraic steps a symbolic check (SymPy/Mathematica simplify–differentiate– compare, saved as an artifact) is stronger evidence than an eyeballed derivation — use it where the hunt is a skipped step or a factor.
  • Treat the skip-words as the attack surface. "It is easy to see," "clearly," "obviously," "it follows trivially," "the other case is similar," "by symmetry" — each marks a step the author declined to write, and each is a named point to open. The characteristic sleights parked behind them: several manipulations fused into one line; a sign asserted without the derivative or inequality that establishes it; one effect declared to "dominate" another with no bound; an inequality direction reversed across a multiplication or reciprocal without remark; Jensen applied in the wrong direction; a discrete object (an integer threshold, a count) differentiated as if continuous; an edge case silently dropped (x = 0, a singular matrix, an empty configuration); cases claimed "analogous" with no explicit substitution carrying the proved case onto them; existence asserted with neither a construction nor a named theorem whose hypotheses are checked here.
  • Construct a counterexample. For each theorem or proposition, actively try to build an object that satisfies every stated hypothesis and violates the conclusion — a specific function, sequence, or parameter value. A concrete counterexample is the single most powerful refutation available; a theorem that survives a serious construction attempt is far stronger evidence than one you read and nodded along to.
  • Proof of a weaker statement. Does the proof establish exactly the claim, or something weaker — the same quantifiers, domain, strict-vs-weak inequality, existence-vs-uniqueness strength, mode of convergence, and notion of solution/equilibrium/optimality? A correct proof of a different, weaker theorem is a finding even when every line is right.
  • Hunt the characteristic kill-shapes. The errors a fluent proof hides best are plausible, internally consistent, and shaped alike: a property true in expectation claimed ex post (one realization's report moving another's adjustment, while the mean stays put); a cancellation or identity at a point mistaken for a global property (a cross-partial vanishing at the target read as balance everywhere — check whether the mixed derivative
Read the complete source ↗