Paper Review

Paper Review

A systematic approach to self-reviewing academic papers before submission. Covers a 5-aspect review checklist, reverse-outlining for structural clarity, figure/table quality checks, and rebuttal preparation.

When to Use This Skill

• User wants to review or check a paper draft before submission
• User asks for feedback on paper quality or completeness
• User wants to prepare for potential reviewer criticism
• User mentions "review paper", "check my draft", "self-review"

If the user has already received reviewer comments and needs to write a rebuttal, use the paper-rebuttal skill instead.

Prerequisites

Before starting review, confirm the paper-writing handoff checklist is satisfied: all sections drafted, claims anchored to evidence, limitation section present, figures finalized, and no unresolved \todo{} markers. If any item is incomplete, finish writing before reviewing.

---

The Perfectionist Approach

Strive for perfection: review your own paper, consider every question a reviewer might ask, and address them one by one.

The best defense against negative reviews is a thorough self-review:

1. Adversarial review: Read your own paper as a critical reviewer would
2. Seek advisor feedback: Ask your advisor to review — the more feedback, the better
3. Address everything: For every potential weakness you find, either fix it or prepare a defense

Counterintuitive Review Protocol

Run this protocol before final polishing:

1. Reject-first simulation: Force yourself to write a one-paragraph reject summary before writing any positive comments.
2. Delete one unsupported strong claim: If a strong claim lacks direct evidence, remove it instead of defending it.
3. Score trust, not only score gains: Papers with slightly lower gains but higher fairness and reproducibility often receive better review outcomes.
4. Promote one explicit limitation: Move one meaningful limitation from hidden notes into the paper; transparency can increase confidence.
5. Attack your novelty claim: Ask "Could a strong PhD derive this in one afternoon?" If yes, narrow and sharpen the novelty statement.

See references/counterintuitive-review.md

---

5-Aspect Self-Review Checklist

Aspect 1: Contribution Sufficiency

The paper does not provide readers with new knowledge.

Ask these questions to evaluate whether the contribution is sufficient:

• Are the failure cases common? If the failure cases are frequent and obvious, reviewers may question whether the method is ready for publication.
• Is the proposed technique well-explored? If the technique is already widely studied, what new insight or improvement do we bring?
• Is the improvement foreseeable / well-known? If the improvement was predictable from combining known ideas, the novelty may be questioned.
• Is the technique too straightforward? A straightforward application of existing techniques may lack sufficient contribution.

Red flag: If "yes" to any of these, strengthen the contribution narrative or add more technical depth.

Aspect 2: Writing Clarity

Missing technical details, not reproducible; a method module lacks motivation.

• Missing technical details? Would a reader be able to reproduce the method from the paper alone?
• Missing module motivation? Does every module in the Method section explain why it exists, not just what it does?
• Paragraph structure: Does each paragraph have a clear topic? Does the first sentence state the point?
• Flow: Is the logical flow between paragraphs and sections smooth?
• Terminology: Are terms used consistently throughout?

Red flag: If reproducibility is in doubt, add implementation details or supplementary material.

Aspect 3: Experimental Results Quality

Only slightly better than previous methods; or better than previous methods but still not good enough.

• Marginal improvement? If the improvement over SOTA is very small, is it statistically significant?
• Absolute quality insufficient? Even if better than baselines, is the output quality good enough for the application?
• Visual quality: Do qualitative results look convincing? Are improvements visible?

Red flag: If improvements are marginal, emphasize other advantages (speed, generalizability, simplicity) or add more challenging test cases.

Aspect 4: Experimental Testing Completeness

Missing ablation studies; missing important baselines; missing important evaluation metrics; data too simple.

• Missing ablation studies? Is every core contribution ablated?
• Missing important baselines? Are recent SOTA methods included?
• Missing evaluation metrics? Are all standard metrics for this task reported?
• Datasets too simple? Do the benchmarks truly test the method's capabilities?
• No failure case analysis? Honest failure analysis increases credibility.

Red flag: Missing ablations or baselines is one of the most common reasons for rejection.

Aspect 5: Method Design Issues

Experimental setting is impractical; method has technical flaws; method is not robust; new method's costs outweigh its benefits.

• Impractical experimental setting? Are assumptions realistic for the intended use case?
• Technical flaws? Does the method have theoretical or conceptual weaknesses?
• Not robust? Does the method require per-scene hyperparameter tuning?
• Benefit < Limitation? Does the new module introduce limitations that outweigh its benefits?

Red flag: If the method requires significant tuning per scenario, add robustness experiments or acknowledge and address the limitation.

---