跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
搜索与研究

paper-review

Review academic papers for correctness, quality, and novelty using OpenJudge's multi-stage pipeline. Supports PDF files and LaTeX source packages (.tar.gz/.zip). Covers 10 disciplines: cs, medicine, physics, chemistry, biology, economics, psychology, environmental_science, mathematics, social_sciences. Use when the user asks to review, evaluate, critique, or assess a research paper, check references, or verify a BibTeX file.

agentscope-ai · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add agentscope-ai/OpenJudge --skill paper-review

技能说明


name: paper-review description: > Review academic papers for correctness, quality, and novelty using OpenJudge's multi-stage pipeline. Supports PDF files and LaTeX source packages (.tar.gz/.zip). Covers 10 disciplines: cs, medicine, physics, chemistry, biology, economics, psychology, environmental_science, mathematics, social_sciences. Use when the user asks to review, evaluate, critique, or assess a research paper, check references, or verify a BibTeX file.

Paper Review Skill

Multi-stage academic paper review using the OpenJudge PaperReviewPipeline:

  1. Safety check — jailbreak detection + format validation
  2. Correctness — objective errors (math, logic, data inconsistencies)
  3. Review — quality, novelty, significance (score 1–6)
  4. Criticality — severity of correctness issues
  5. BibTeX verification — cross-checks references against CrossRef/arXiv/DBLP

Prerequisites

# Install OpenJudge
pip install py-openjudge

# Extra dependency for paper_review
pip install litellm
pip install pypdfium2  # only if using vision mode (use_vision_for_pdf=True)

Gather from user before running

InfoRequired?Notes
Paper file pathYesPDF or .tar.gz/.zip TeX package
API keyYesEnv var preferred: OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.
Model nameNogpt-5.2, anthropic/claude-opus-4-6, dashscope/qwen-vl-plus. See Model selection below
DisciplineNoIf not given, uses general CS/ML-oriented prompts
VenueNoe.g. "NeurIPS 2025", "The Lancet"
InstructionsNoFree-form reviewer guidance, e.g. "Focus on experimental design"
LanguageNo"en" (default) or "zh" for Simplified Chinese output
BibTeX fileNoRequired only for reference verification
CrossRef emailNoImproves API rate limits for BibTeX verification

Quick start

File type is auto-detected: .pdf → PDF review, .tar.gz/.zip → TeX review, .bib → BibTeX verification.

# Basic PDF review
python -m cookbooks.paper_review paper.pdf

# With discipline and venue
python -m cookbooks.paper_review paper.pdf \
  --discipline cs --venue "NeurIPS 2025"

# Chinese output
python -m cookbooks.paper_review paper.pdf --language zh

# Custom reviewer instructions
python -m cookbooks.paper_review paper.pdf \
  --instructions "Focus on experimental design and reproducibility"

# PDF + BibTeX verification
python -m cookbooks.paper_review paper.pdf \
  --bib references.bib --email your@email.com

# Vision mode (for models that prefer images over text extraction)
python -m cookbooks.paper_review paper.pdf \
  --vision --vision_max_pages 30 --format_vision_max_pages 10

# TeX source package
python -m cookbooks.paper_review paper_source.tar.gz \
  --discipline biology --email your@email.com

# TeX source package with Chinese output and custom instructions
python -m cookbooks.paper_review paper_source.tar.gz \
  --language zh --instructions "This is a short paper, be concise"

# Verify a standalone BibTeX file
python -m cookbooks.paper_review --bib_only references.bib --email your@email.com

All options

FlagDefaultDescription
input (positional)—Path to PDF, TeX package, or .bib file
--bib_only—Path to .bib file for standalone verification (no review)
--modelgpt-4oModel name
--api_keyenv varAPI key
--base_url—Custom API endpoint — must end at /v1, not /v1/chat/completions (litellm appends the path automatically)
--discipline—Academic discipline
--venue—Target conference/journal
--instructions—Free-form reviewer guidance
--languageenOutput language: en or zh
--bib—Path to .bib file (for PDF review + reference verification)
--email—CrossRef mailto for BibTeX check
--paper_namefilename stemPaper title in report
--outputautoOutput .md report path
--no_safetyoffSkip safety checks
--no_correctnessoffSkip correctness check
--no_criticalityoffSkip criticality verification
--no_biboffSkip BibTeX verification
--visiononUse vision mode (requires pypdfium2); enabled by default
--vision_max_pages30Max pages in vision mode (0 = all)
--format_vision_max_pages10Max pages for format check (0 = use --vision_max_pages)
--timeout7500API timeout in seconds

Interpreting results

Review score (1–6):

  • 1–2: Reject (major flaws or well-known results)
  • 3: Borderline reject
  • 4: Borderline accept
  • 5–6: Accept / Strong accept

Correctness score (1–3):

  • 1: No objective errors
  • 2: Minor errors (notation, arithmetic in non-critical parts)
  • 3: Major errors (wrong proofs, core algorithm flaws)

BibTeX verification:

  • verified: found in CrossRef/arXiv/DBLP
  • suspect: title/author mismatch or not found — manual check recommended

Model selection

This pipeline uses litellm for model calls. Provider prefixes are handled automatically by the pipeline — see the table below.

IMPORTANT: The model MUST support multimodal (vision) input. PDF review uses vision mode (--vision) to render pages as images, which requires a vision-capable model. Text-only models will fail or produce empty reviews.

The --model value uses a provider/model-name convention so the pipeline knows which API endpoint to call. The table below shows the exact string to pass:

Provider--model valueEnv varNotes
OpenAIgpt-5.2, gpt-5-mini, …OPENAI_API_KEYNo prefix needed; gpt-5.2 is the current flagship vision model; check OpenAI models for the latest
Anthropicanthropic/claude-opus-4-6, `anthropic/claude-son
阅读完整来源 ↗