Review Phase

/review-phase -- Run Review Cycle for a Phase

Run the review cycle for a completed phase artifact. This enhanced protocol uses 4-bot review (replacing the previous 3-bot) for most phases, 5-bot for final documentation, and 1-bot for selection and observed results. All review tiers include plot-validator for phases that produce figures.

Arguments: $ARGUMENTS

The argument is optionally a phase identifier: 1, 2, 3, 4a, 4b, 4c, or 5. If omitted, read STATE.md to determine the current phase.

Step 1: Determine Phase and Review Tier

1. Read STATE.md to confirm the analysis state.
2. If a phase argument was given, use it. Otherwise use the current phase from STATE.md.
3. Read analysis_config.yaml for cost controls (max_review_iterations, review_warn_threshold).
4. Determine the review tier for this phase:

If phase is 2, report "Phase 2 uses self-review. No external review required." and return.

Step 2: Locate the Artifact Under Review

Find the latest artifact for this phase:

Also locate upstream artifacts that reviewers need for context (see the dependency table in CLAUDE.md).

Read the experiment log for this phase if it exists.

Identify all figures in the figures/ directory for this phase -- these will be passed to the plot-validator.

Step 3: Run the Review

Update STATE.md: status: reviewing, timestamp.

Initialize iteration counter: iteration = 0.

2-bot Review (Phases 0, 1)

Loop until PASS, ESCALATE, or max iterations:

1. Increment iteration counter. Check cost controls (same thresholds as 4-bot).

2. Spawn logic-reviewer in parallel with plot-validator (if figures exist):

   Logic reviewer instructions:

   • Read: methodology spec (review focus for this phase), the artifact under review, upstream artifacts, experiment log
   • Evaluate: reasoning validity, DAG consistency, EP arithmetic, logical completeness
   • Classify every issue as (A) must resolve, (B) should address, (C) suggestion
   • Include structured fix instructions (exact or requires_reasoning) for every A/B item
   • Write output to: review/logic/ with session-named filename

   Plot-validator: same instructions as 4-bot below.

3. Spawn arbiter via SendMessage:

   • Read: the artifact, logic review, plot-validation report (if exists)
   • Write output to: review/arbiter/
   • Decision: PASS, ITERATE, or ESCALATE

4-5. Handle decision same as 4-bot below.

4-bot Review (Phases 4a, 4b)

Loop until PASS, ESCALATE, or max iterations:

1. Increment iteration counter.

2. Check cost controls:

   • If iteration > review_warn_threshold: log WARNING -- "Review iteration {iteration} for phase {phase}. Consider whether issues are fundamental enough to escalate."
   • If iteration > 5: log STRONG WARNING.
   • If iteration >= max_review_iterations: force ESCALATE to human. Update STATE.md: status: blocked. Report and stop.

3. Spawn physics-reviewer, critical-reviewer, and constructive-reviewer in parallel via SendMessage. If this phase has figures, also spawn plot-validator in parallel with the three reviewers:

   Physics reviewer instructions:

   • Read: src/methodology/03-phases.md (review focus for this phase), the artifact under review, upstream artifacts, experiment log
   • Evaluate physics correctness: signal model assumptions, background treatment, kinematic reasoning, systematic uncertainty coverage, statistical methodology
   • Classify every issue as (A) must resolve, (B) should address, (C) suggestion
   • Write output to: review/physics/{REVIEW}.md with session-named filename

   Critical reviewer instructions:

   • Read: methodology spec (review focus for this phase), the artifact under review, upstream artifacts, experiment log
   • Find flaws: incomplete estimates, missing systematics, unjustified assumptions, biases, physics errors, code bugs
   • Classify every issue as (A) must resolve, (B) should address, (C) suggestion
   • Write output to: review/critical/ with session-named filename

   Constructive reviewer instructions:

   • Read: same inputs as critical reviewer
   • Strengthen the analysis: clarity, additional validation, presentation improvements
   • Focus on B and C issues but escalate to A if genuine errors found
   • Write output to: review/constructive/ with session-named filename

   Plot-validator instructions (if figures exist):

   • First iteration: Read all figures in this phase's figures/ directory
   • Subsequent iterations: Read only figures affected by the latest fixes (listed in arbiter's fix instructions) plus any newly created figures. Unmodified figures retain prior validation status.
   • Read: conventions/ plotting standards (axis labels, font sizes, color schemes, legend placement, ratio panels, domain style requirements)
   • Validate each figure against the conventions
   • Check: axis labels and units, legend completeness, ratio panel presence where required, color accessibility, resolution and format, statistical uncertainty display, verification compliance (no signal region data shown before approval)
   • Classify issues as (A) must fix, (B) should fix, (C) cosmetic suggestion
   • Write output to: review/plot-validation/{REVIEW}.md

4. After all reviewers complete, spawn arbiter via SendMessage:

   • Read: the artifact, all review files (latest from review/physics/, review/critical/, review/constructive/, and review/plot-validation/ if it exists)
   • For each issue: if multiple reviewers agree, accept; if they disagree, assess independently; if all missed something, raise it
   • Incorporate plot-validator findings: Category A plot issues are treated as Category A overall
   • Write output to: review/arbiter/ with session-named filename
   • End with a clear decision: PASS, ITERATE (list Category A items including plot issues), or ESCALATE (document why)

5. Read the arbiter decision from the latest file in review/arbiter/.

6. Handle the decision:

   • PASS: Check for regression triggers in the review output. If none found, update STATE.md (status: passed), record in Phase History table (including iteration count), and return the result.

   • ITERATE: Apply fixes using the arbiter's structured instructions:

     1. Exact fixes (type: exact): Apply directly via Edit tool — no subagent needed.
     2. Reasoning fixes (type: requires_reasoning): Spawn fix agent with the section path, instruction, and previous artifact. Fix agent addresses only these items.
     3. Re-verify based on remaining issue severity:
        • A-present: Continue the arbiter via SendMessage with the updated artifact and fix diff. Arbiter re-checks only the fixed items. On 3rd+ iteration, spawn a fresh arbiter instead.
        • B-only remaining: Executor self-verifies against a checklist of the B items. If all addressed → PASS. If any fail → full re-review.
     4. Loop until PASS or iteration limit.

   • ESCALATE: Update STATE.md (status: blocked, record escalation reason). Report to user: "Phase {phase} review escalated. Reason: {reason}. Human intervention required." Stop.