跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
搜索与研究

referee-sim

Use before circulating any outward-facing scientific document — simulate the strongest standard objection from every audience it claims and verify the document answers each where that reader would look.

BootLoops-ai · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add BootLoops-ai/skills --skill referee-sim

技能说明


name: referee-sim description: Use before circulating any outward-facing scientific document — simulate the strongest standard objection from every audience it claims and verify the document answers each where that reader would look.

referee-sim — the frame linter

Fact-checking and frame-checking are different audits. A verification pass confirms every sentence is true; this pass checks what a reader concludes. A document whose every claim is sourced can still mislead by assembly: the caveats live in the body while impressions form in the abstract; the comparison is honest but the baseline is one nobody uses; the theorem is real but the reader leaves believing it covers the case it excludes. That exact failure can survive multiple honesty passes and surface only when an outside expert finally reads the document — the expensive way to catch it, because by then the impression has already formed in the one reader whose report matters.

This is an audit of the IMPLICATURE — what each kind of reader walks away believing — run before anyone outside sees the document. Run your sentence-level style and honesty linter separately; the two passes catch disjoint failures, and neither substitutes for the other.

Procedure

1. Enumerate the audiences, explicitly and in writing

List every community that could plausibly write a report on this document. Sources for the list, in order:

  • The introduction's "this matters to X, Y, and Z" sentence. That sentence is a contract; every community it names gets an audit row.
  • Every community whose methods the document borrows. Using their machinery invokes their standards, whether or not the document ever addresses them.
  • Every community whose results the document takes as input or uses as a comparison.
  • The incumbent: whoever currently does the thing the document claims to do better, faster, or differently. This audience exists even when the introduction never mentions them — especially then.
  • The practitioner who would act on the result, whose question is never "is it interesting" but "what breaks if I rely on this."
  • The venue's general reader, who determines which claims must survive without the surrounding expertise.

Two rules. First, any community borrowed from for authority — a method, a dataset, a benchmark, a motivating application — gets a row without exception; the missing-audience failure below is almost always one of these. Second, a claim of interdisciplinarity RAISES the bar. Each additional audience brings its own standard objection, and outsiders read less charitably than insiders: they apply their home field's first-order standard and have no reason to extend the benefit of the doubt.

2. Generate each audience's strongest STANDARD objection — on two axes

For each audience row, generate two separate objections:

  • Correctness: "is this right, and does it hold in the cases my community cares about?"
  • Novelty: "is this new, and what exactly is the advance over what we already do?"

These are different referee reports, answered in different places — correctness in scope statements, controls, and comparisons; novelty in the introduction's positioning and its engagement with prior work. A document that answers only one axis dies on the other, and conflating them produces a characteristic hybrid: a document that proves everything and never says what is new, or one that claims a first while the incumbent community's state of the art goes unexamined.

"Standard" means the first thing that community's referee asks, not an exotic one. Calibration by community type:

  • Engineering/applications: "does this survive the generic case (generic noise, generic inputs, adversarial settings), or only the structured case you studied?"
  • Experimentalists: "what does the apparatus actually measure, and is it the same object you computed? Which parts of the prediction survive the differences?"
  • Numericists: "is anything converged? In which units or scheme — and does the flattering choice hide the drift? What is the null hypothesis your signal must reject?"
  • Formal theorists: "is the named object well-defined here (does the symmetry, charge, or protection actually exist in this setting)? What is conjecture vs theorem, and does the abstract distinguish them?"
  • The incumbent method's community: "we already do this better — what is your edge, precisely, and have you checked our state of the art?"

3. Steelman every objection

The objection must be one a well-informed, unsympathetic expert would sign — steelmanned, never strawmanned. Three tests:

  • The sting test. If the document as written already answers the objection cleanly, you have probably written a weak one; sharpen until acting on it would require an edit. Occasionally the document really has pre-answered the strongest standard objection — treat that verdict as suspect and earn it, because a clean sweep is also exactly what a strawman pass produces.
  • The knowledge test. Write the objection in the referee's own voice and name what they know that the document ignores: the prior result, the standard control, the benchmark their community demands. If you cannot name what the objector knows, you have not simulated them; you have simulated yourself in their seat.
  • The lead test. A real referee leads with the standard objection. If your simulated objection is clever but nonstandard, generate the standard one first; the exotic one may follow as a second row, never as a substitute.

4. Verify each answer sits where the objector would look

Hostile readers read in a fixed order: title, abstract, figures, conclusions — then, only if still engaged, the one section their objection lives in. "Derivable by assembling facts scattered across the document" is a FAIL: the test is whether the objector meets the answer on their actual reading path, at or before the point where the objection forms. A disqualifying caveat that first appears mid

阅读完整来源 ↗