claims
Third review stage: work up from the surviving estimates to the narrowest story each can carry, test that story, and build the claim ledger — one row per load-bearing claim recording how central it is, whether it holds, what survives and what is not established, and the strongest defence. The ledger records what is true; it rates nothing.
isitcredible · Static checks on file
Installation
Read the source and dependencies before running a command. Copying does not install or execute anything.
npx skills add isitcredible/research_skills --skill claimsAbout this skill
name: claims description: >- Third review stage: work up from the surviving estimates to the narrowest story each can carry, test that story, and build the claim ledger — one row per load-bearing claim recording how central it is, whether it holds, what survives and what is not established, and the strongest defence. The ledger records what is true; it rates nothing.
Claims
A coefficient is not a finding. Work up from each surviving estimate to the story it can genuinely carry, then test the story — a result can pass every robustness check and still support only a story that cannot be true.
Start from the artifacts
Read analysis/code-and-data/results.csv and review/code-and-data-REPORT.md — the
files, never a remembered coefficient. Screen every row:
validitynotclean→ evidence that the procedure is uninformative here, never a test outcome: an uninformative result says nothing against the claim and rescues nothing for it.unvalidatedrows cannot carry a verdict yet.row_role = rebuttal→ answers the paper's own selected number, not the test the design requires; if it holds, it cannot settle the claim. If it fails (the number does not reproduce, or loses support under corrected inference), it counts against. Check the labels, not just the field: a re-run of the selected estimate is a rebuttal however it is tagged.- A margin whose only direct tests are invalid or unrun is a ledger row ("not testable under this design"), never a silent pass.
- Scope assumptions in the earlier reports carry into every row that relies on them; find them all, and if one was overturned, re-derive or flag what depended on it.
The work
- Decompose what drives the estimate — functional form, sample and scope, between
vs within variation, influential observations, timing, placebos, construction
seams — in
analysis/claims/driver-matrix.csv(template../assets/driver-matrix.csv). Pure theory: use the attacker–defender–judge record fromtheory-diagnostics; hybrids need both the estimate and the theory outcome. A conclusion resting on a WHIMSICAL premise (assumption-ledger.csv) supports only the ledger's minimal repair. - State the supportable story — who, what exposure, what mechanism, when, how big
(outcome units plus the three dimensions: magnitude, significance, variance share),
against what counterfactual, over what scope. Do not import the abstract's causal
nouns where the estimate supports only a contrast. An average effect does not say
who carries it: one carried by a subpopulation does not support a population-wide
story (
../references/methods/gelman_hullman_kennedy_2024_causal_quartets.md). A hypothesis never stated precisely enough to fail was never tested. - Test it (
../references/claim-test-matrix.md):- printed-table contradictions and version-history leads from
paper; - each mechanism link — existence, direction, timing, magnitude — and the
load-bearing cases checked in the data; a mechanism read off effect
heterogeneity needs its two exclusion assumptions named and an outcome that
is not a non-affine transform of the theory's construct, and absent
heterogeneity rules out no channel
(
../references/methods/fu_slough_2026_hte_mechanisms.md); - design validity: identifying variation, isolation, measurement, power;
- the specialist literature — reading the works is the check
(
../references/literature-protocol.md); another paper drawing on the same sources is context, not corroboration; - claims that reach the conclusion by a bank shot, or label a mechanical consequence as a finding;
- the piranha check: a large general claim about a heavily studied outcome
competes with every other large claim
(
../references/methods/tosh_etal_2025_piranha.md); - the selection filter: published estimates are selected on significance, and
a headline just above the threshold is where selection operates
(
andrews_kasy_2019_publication_bias.md,elliott_kudrin_wuthrich_2022_phacking.md,simonsohn_simmons_nelson_2020_spec_curve.mdin../references/methods/). Label each failure verification or reanalysis.
- printed-table contradictions and version-history leads from
- Try to overturn your own verdict, in whichever direction it runs: for each headline claim, run one defensible challenge that could reverse the row — a test the claim could fail where the row says it holds, or the authors' strongest specification where the row says it fails — and record the result.
The claim ledger
Go through the title, abstract, introduction and conclusion sentence by sentence. One
row per load-bearing claim, in review/claims-REPORT.md and
analysis/claims/claim-ledger.csv (template ../assets/claim-ledger.csv):
claim_id | claim (verbatim) | page | centrality | holds | kernel | not_established | strongest_defence | defence_tested | status | evidence
One sentence becomes several rows when its claim holds at one inference level and has no valid test at another; each row quotes the sentence.
The ledger records what the evidence establishes. It carries no severity: how much a finding matters to a reader is decided when the digest is written.
centrality— where the claim sits in the paper, read from the print:headline— the title, the abstract, or a finding the introduction or conclusion states as the paper's contribution;supporting— a result a headline rests on: an estimate, test, robustness check, replication or mechanism claim in the results;peripheral— an implication, policy prescription, interpretation or framing the paper does not test.
holds— one of:holds— the claim stands as printed; any gap is disclosure or a nuance that changes no reader's conclusion;narrower— the claim's point stands, but only in a narrower scope, size, precision, population or sample than print