Run Judges

Run Judges Skill

Purpose

Execute specialized judge agents in parallel to evaluate implementation plan quality (16 judges, 4 batches), code quality (11 judges, 3 batches), PRD quality (5 judges, 2 batches), or Feature quality (3 judges, 1 batch). All batches respect the Task tool's 4-concurrent-agent limit. Aggregates results into $CLOSEDLOOP_WORKDIR/plan-judges.json (plan), $CLOSEDLOOP_WORKDIR/code-judges.json (code), $CLOSEDLOOP_WORKDIR/prd-judges.json (prd), or $CLOSEDLOOP_WORKDIR/feature-judges.json (feature) with validated output format.

Parameters

--workdir: Path to the working directory containing judge artifacts (optional)

• Resolved in order: --workdir argument → $CLOSEDLOOP_WORKDIR environment variable → .closedloop-ai/judges (default, relative to current working directory)
• The directory is created automatically if it does not exist
• All output files (plan-judges.json, code-judges.json, prd-judges.json, judge-input.json, perf.jsonl, etc.) are written to this resolved directory

--artifact-type: Artifact category to evaluate (plan | code | prd | feature), default: plan

• plan (default): Evaluate implementation plan with 16 judges, 4 batches, output to plan-judges.json
• code: Evaluate implemented code with 11 judges, 3 batches, output to code-judges.json
• prd: Evaluate PRD document with 5 judges across 2 sequential batches (3 + 2, max 4 concurrent per batch), output to prd-judges.json
• feature: Evaluate Feature artifact with 3 judges, 1 batch, output to feature-judges.json

Judge Input Contract (judge-input.json)

The judge input contract is maintained in:

skills/run-judges/references/judge-input-contract.md (resolve to an absolute path at runtime via Glob)

This keeps orchestration flow readable while preserving a single source of truth for contract fields and semantics.

run-judges is the producer chokepoint for judge-input.json. After mode-specific context preparation and before launching any judge agent, invoke the deterministic mapper:

uv run "${CLAUDE_PLUGIN_ROOT}/skills/run-judges/scripts/judge_input_mapping.py" \
  --workdir "$CLOSEDLOOP_WORKDIR" \
  --artifact-type "$ARTIFACT_TYPE" \
  --schema "${CLAUDE_PLUGIN_ROOT}/schemas/judge-input.schema.json"

The mapper builds from the runtime workdir contract: primary artifacts under <runDir>, supporting context under <runDir>/.closedloop-ai/context, and attachments under <runDir>/.closedloop-ai/work/attachments. It validates the generated envelope against schemas/judge-input.schema.json before judge launch. If mapping fails, emit a clear warning and use the documented one-run legacy fallback paths (prd.md, plan.md, or existing compatibility artifacts) only for that run.

Task Context

You are orchestrating quality evaluation for a ClosedLoop artifact (implementation plan, code, or PRD). Your responsibilities:

For plan artifacts (default):

1. Launch context-manager-for-judges agent to prepare compressed plan context
2. Build judge-input.json with plan task/context mapping
3. Launch all 16 judge agents in parallel batches
4. Aggregate their CaseScore outputs into a valid EvaluationReport
5. Write the report to $CLOSEDLOOP_WORKDIR/plan-judges.json
6. Validate output structure and completeness

For code artifacts (--artifact-type code):

1. Launch context-manager-for-judges agent to prepare compressed context
2. Build judge-input.json with code task/context mapping
3. Launch 11 judge agents in parallel batches
4. Aggregate their CaseScore outputs into a valid EvaluationReport
5. Write the report to $CLOSEDLOOP_WORKDIR/code-judges.json
6. Validate output structure and completeness

For PRD artifacts (--artifact-type prd):

1. Check $CLOSEDLOOP_WORKDIR/prd.md exists (graceful exit if missing)
2. Build and schema-validate judge-input.json by invoking scripts/judge_input_mapping.py
3. Launch the 5 PRD judges in 2 sequential batches (3 + 2, max 4 concurrent per batch)
4. Aggregate all 5 CaseScores into a valid EvaluationReport
5. Write the report to $CLOSEDLOOP_WORKDIR/prd-judges.json
6. Validate output structure and completeness

For Feature artifacts (--artifact-type feature):

1. Check $CLOSEDLOOP_WORKDIR/feature.md exists, or $CLOSEDLOOP_WORKDIR/prd.md exists for legacy Feature inputs (graceful exit code 0 if both are missing)
2. Build and schema-validate judge-input.json by invoking scripts/judge_input_mapping.py
3. Launch 3 judges in 1 batch (feature-completeness-judge + prd-testability-judge + prd-dependency-judge)
4. Aggregate 3 CaseScores into a valid EvaluationReport
5. Write the report to $CLOSEDLOOP_WORKDIR/feature-judges.json
6. Validate output structure and completeness

Feature mode judge selection rationale:

• prd-auditor is excluded because it assumes US-###/AC-#.# numbering and multi-story traceability, which Feature artifacts do not follow
• prd-scope-judge is excluded because it assumes In/Out-of-Scope sections that are not present in Feature artifacts

Feature mode preamble: Feature mode uses the dedicated feature_preamble.md so judges receive a Feature-shaped contract (evaluation_type=feature, lightweight structure, no PRD-only sections). Do NOT substitute prd_preamble.md — it would frame the input as a full PRD and contradict the envelope's evaluation_type.

Success criteria:

• All judges executed (or error CaseScores generated for failures)
• Valid JSON written to appropriate output file
• Validation script passes with zero errors

Threshold Overrides

The run-judges skill supports per-artifact-type threshold customization via JSON configuration files. This allows you to adjust evaluation strictness for different artifact types (e.g., applying a lower threshold for test-judge when evaluating code vs plan).

Configuration Schema

Threshold overrides are defined in a JSON file with the following structure:

{
  "overrides": {
    "artifact_type:judge_name": <threshold_float>
  }
}

Where:

• Key format: "artifact_type:judge_name" (e.g., "code:test-judge", "plan:technical-accuracy-judge")
• Value: Threshold as a float in range [0.0, 1.0]

Example configuration:

{
  "overrides": {
    "code:test-judge": 0.75,
    "plan:technical-accuracy-judge": 0.85
  }
}

Loading Precedence

The skill checks the following locations in order, using the first valid configuration found:

1. Run-specific overrides (highest precedence):

   • Path: $CLOSEDLOOP_WORKDIR/.closedloop-ai/settings/threshold-overrides.json
   • Use case: Override thresholds for a specific ClosedLoop run

2. Repo-level defaults (fallback):

   • Path: <project-root>/.closedloop-ai/settings/threshold-overrides.json
   • Use case: Set project-wide threshold defaults

3. Hardcoded defaults (graceful degradation):

   • If no configuration file exists at any location, use built-in defaults
   • No error is raised for missing configuration files

Default Overrides

The following default overrides apply when evaluating code artifacts:

All other judges use the same threshold (typically 0.8) across artifact types.

Validation and Error Handling

When loading threshold overrides, the skill applies the following validation rules:

Schema Validation:

• Configuration must contain an "overrides" key
• Each key must match the pattern artifact_type:judge_name
• Each value must be a float in range [0.0, 1.0]
• Keys must reference valid artifact types (plan, code, prd) and judge names

Error Behavior:

• Malformed JSON: Log warning and continue with hardcoded defaults

  Warning: Invalid threshold-overrides.json, skipping overrides: {error}

• Invalid schema: Log warning and continue with hardcoded defaults
• File not found: Silently use defaults (no warning logged)

Error recovery ensures the skill always completes judge execution, even if threshold configuration is incorrect.

Integration with Judge Execution

When executing judges:

1. Before launching judge batches: Load threshold overrides from the precedence chain
2. Merge with defaults: Loaded overrides take precedence over hardcoded defaults
3. Apply per-judge: Each judge receives its artifact-type-specific threshold via the evaluation context
4. CaseScore validation: Thresholds are used to determine final_status (pass/fail) based on metric scores

When artifact type is code:

• Load threshold overrides before executing judge batches
• Apply code-specific thresholds to each judge's evaluation criteria
• Merge loaded overrides with defaults (loaded values take precedence)

Performance Instrumentation (Mandatory)

You MUST emit a pipeline_step event to $CLOSEDLOOP_WORKDIR/perf.jsonl at the end of each phase below. This keeps perf telemetry in the canonical schema and adds nested metadata for judge/sub-agent work.

Context: CLOSEDLOOP_WORKDIR, CLOSEDLOOP_RUN_ID, and CLOSEDLOOP_ITERATION are set by the run-loop. CLOSEDLOOP_PARENT_STEP and CLOSEDLOOP_PARENT_STEP_NAME are set as env vars on the claude invocation by run-loop; they are inherited by all Bash tool calls — no sourcing needed. Use sub_step as numeric phase order and optional sub_step_name to capture the judge/sub-agent name when applicable (for batch-level phases where many judges run, use the batch label).

Sub-step numbering:

Start of phase (run Bash once at the beginning of each phase): Set the two sub-step variables at the top for the current phase, then run the block. It writes start time to a temp file so the end-of-phase Bash can compute duration. CLOSEDLOOP_PARENT_STEP and CLOSEDLOOP_PARENT_STEP_NAME are already in the environment (set by run-loop on the claude invocation).

# Set these two values for the current phase:
SUB_STEP_NUM=0
SUB_STEP_LABEL="context_manager"   # context_manager | batch_1 … | aggregate | validate

mkdir -p "$CLOSEDLOOP_WORKDIR/.closedloop-ai"
{
  echo "SUB_STEP=${SUB_STEP_NUM}"
  echo "SUB_STEP_NAME=${SUB_STEP_LABEL}"
  echo "PARENT_STEP=${CLOSEDLOOP_PARENT_STEP:-0}"
  echo "PARENT_STEP_NAME=${CLOSEDLOOP_PARENT_STEP_NAME:-unknown}"
  echo "STARTED_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
  echo "START_EPOCH=$(date +%s)"
} > "$CLOSEDLOOP_WORKDIR/.closedloop-ai/perf-substep-start.env"

End of phase (run Bash once at the end of each phase, after the phase work is done): Read start time, compute duration, append one line to perf.jsonl, then remove the temp file.

source "$CLOSEDLOOP_WORKDIR/.closedloop-ai/perf-substep-start.env"
END_EPOCH=$(date +%s)
ENDED_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)
DURATION=$((END_EPOCH - START_EPOCH))
jq -n -c \
  --arg event "pipeline_step" \
  --arg run_id "${CLOSEDLOOP_RUN_ID:-unknown}" \
  --argjson iteration "${CLOSEDLOOP_ITERATION:-0}" \
  --argjson step "$PARENT_STEP" \
  --arg step_name "$PARENT_STEP_NAME" \
  --argjson sub_step "$SUB_STEP" \
  --arg sub_step_name "$SUB_STEP_NAME" \
  --arg started_at "$STARTED_AT" \
  --arg ended_at "$ENDED_AT" \
  --argjson duration_s "$DURATION" \
  --argjson exit_code 0 \
  --argjson skipped false \
  '{event:$event,run_id:$run_id,iteration:$iteration,step:$step,step_name:$step_name,sub_step:$sub_step,sub_step_name:$sub_step_name,started_at:$started_at,ended_at:$ended_at,duration_s:$duration_s,exit_code:$exit_code,skipped:$skipped}' >> "$CLOSEDLOOP_WORKDIR/perf.jsonl"
rm -f "$CLOSEDLOOP_WORKDIR/.closedloop-ai/perf-substep-start.env"

Order of operations per phase: Run the "start of phase" Bash first (set SUB_STEP_NUM and SUB_STEP_LABEL at the top, then run the block), then perform the phase work, then run the "end of phase" Bash.

Execution Workflow

Working Directory Resolution

Before any other step, resolve the working directory and export it as CLOSEDLOOP_WORKDIR:

# Resolve working directory (precedence: --workdir arg > env var > default)
if [ -n "$ARG_WORKDIR" ]; then
  WORKDIR="$ARG_WORKDIR"
elif [ -n "$CLOSEDLOOP_WORKDIR" ]; then
  WORKDIR="$CLOSEDLOOP_WORKDIR"
else
  WORKDIR="$(pwd)/.closedloop-ai/judges"
fi

mkdir -p "$WORKDIR"
export CLOSEDLOOP_WORKDIR="$WORKDIR"

Where $ARG_WORKDIR is the value passed via --workdir in the invocation prompt. All subsequent references to $CLOSEDLOOP_WORKDIR use this resolved value.

Agents Snapshot (Pre-Step)

Before any judge execution, ensure a snapshot of judge agent definitions exists in $CLOSEDLOOP_WORKDIR/agents-snapshot/. This preserves the exact agent versions used for each evaluation run.

Action: Run the snapshot script via Bash:

bash "${CLAUDE_PLUGIN_ROOT}/skills/run-judges/scripts/ensure_agents_snapshot.sh" "$CLOSEDLOOP_WORKDIR"

The script is idempotent — it skips if manifest.json already exists.

Error handling: If the script fails or is not found, log a warning and continue — snapshot failure must not block judge execution.

Agent Registry Validation (Pre-Flight Check)

Before any judge execution, validate the agent registry to ensure all judge agents required for the current artifact type are resolvable. This prevents launching batches only to discover agents are missing mid-run.

Action: Run validate_agent_registry.py via Bash:

uv run "${CLAUDE_PLUGIN_ROOT}/tools/python/validate_agent_registry.py" \
  --artifact-type "$ARTIFACT_TYPE" \
  --workdir "$CLOSEDLOOP_WORKDIR"

Exit behavior:

• Exit code 0 — all required agents are registered; proceed with judge execution
• Exit code non-zero — one or more required agents are missing or unresolvable; abort immediately and do NOT proceed to judge batches

On failure:

• Log the validation error output in full
• Exit the skill with a non-zero status code
• Do NOT generate partial error CaseScores for this failure mode (the workflow should not proceed at all)

Step 0: Mandatory Contract Pre-Read

Before any prerequisite checks or judge launches:

1. Resolve the contract file path using Glob with:
   • **/skills/run-judges/references/judge-input-contract.md
2. Read the resolved judge-input-contract.md file in full.
3. Apply the contract requirements when constructing $CLOSEDLOOP_WORKDIR/judge-input.json.
4. If the file is missing, ambiguous (multiple matches), or unreadable, fail fast with a clear error (do not proceed with judge execution).

Prerequisites Check

Performance: At the start of this phase run the "start of phase" Bash with SUB_STEP_NUM=0 and SUB_STEP_LABEL=context_manager for both plan and code modes. For prd and feature modes, emit sub_step=0 with SUB_STEP_LABEL=context_prep and skipped=true immediately (no context manager runs). At the end of the phase run the "end of phase" Bash.

Before starting, verify required inputs exist:

For plan artifacts (default):

# Validate input files exist
if [ ! -f "$CLOSEDLOOP_WORKDIR/prd.md" ]; then
  echo "WARNING: $CLOSEDLOOP_WORKDIR/prd.md not found. Skipping judges."
  exit 0  # Graceful skip - do not fail workflow
fi

if [ ! -f "$CLOSEDLOOP_WORKDIR/plan.json" ]; then
  echo "WARNING: $CLOSEDLOOP_WORKDIR/plan.json not found. Skipping judges."
  exit 0
fi

Investigation log resolution (plan mode):

After validating prd.md and plan.json, resolve supporting context for plan judges:

1. Use existing file first

   • If $CLOSEDLOOP_WORKDIR/investigation-log.md exists, use it as-is.

2. Check @code:pre-explorer availability before invoking

   • Perform an explicit capability probe for @code:pre-explorer in the active Claude/plugin environment.
   • Treat "unknown agent", "agent not found", or plugin resolution errors as pre-explorer unavailable.
   • Recommended probe pattern:
     • Attempt a minimal Task() call targeting @code:pre-explorer.
     • If the platform rejects the agent type before execution, classify as unavailable and continue to internal fallback.

3. If available, invoke pre-explorer

   • Launch @code:pre-explorer with WORKDIR=$CLOSEDLOOP_WORKDIR to generate missing pre-exploration artifacts.
   • Re-check for $CLOSEDLOOP_WORKDIR/investigation-log.md after completion.

4. If unavailable or invocation failed, run internal fallback

   • Generate investigation-log.md with a lightweight local-only investigation.
   • Keep it fast and deterministic (no external web research).
   • Internal fallback should:
     • Read prd.md and extract top entities/actions as search seeds.
     • Run targeted Glob/Grep against the local repository for likely implementation files.
     • Record top relevant files and short rationale under Files Discovered / Key Findings.
     • Add requirement-to-code evidence links under Requirements Mapping.
   • Use the canonical sections:
     • ## Search Strategy
     • ## Files Discovered
     • ## Key Findings
     • ## Requirements Mapping
     • ## Uncertainties

5. Never block plan context preparation on investigation context

   • If log generation still fails, emit a warning and continue.

6. Prepare plan-context.json via context-manager-for-judges

   • Launch @judges:context-manager-for-judges with artifact_type=plan.
   • Verify $CLOSEDLOOP_WORKDIR/plan-context.json exists.
   • If missing after invocation, log warning and activate compatibility mode for this run:
     • Compatibility mode allows one emergency fallback to raw plan.json + prd.md.
     • Use compatibility mode only when context generation fails.

7. Plan-mode source-of-truth policy

   • Normal mode: plan-context.json is primary and required.
   • Compatibility mode: plan.json + prd.md may be used for this run only.

8. Build plan-mode judge-input.json

   • Invoke scripts/judge_input_mapping.py with --artifact-type plan.
   • The mapper sets evaluation_type, task, primary_artifact, supporting_artifacts, source_of_truth, fallback_mode, and metadata from the runtime workdir contract.
   • In compatibility mode, allow the mapper to produce a schema-valid fallback envelope for existing plan compatibility artifacts and include prd.md as supporting evidence when available.
   • If the mapper exits non-zero, log the error and use the one-run legacy fallback only if prd.md plus plan.md or the existing compatibility artifact is readable.

For code artifacts (--artifact-type code):

# Resolve investigation context for code judges (best effort)
if [ ! -f "$CLOSEDLOOP_WORKDIR/investigation-log.md" ]; then
  echo "INFO: investigation-log.md missing. Attempting best-effort generation via @code:pre-explorer..."
  # Launch @code:pre-explorer with WORKDIR=$CLOSEDLOOP_WORKDIR
  # If unavailable/fails, continue with warning (non-blocking for code judges)
fi

# Launch context-manager-for-judges agent to prepare compressed context
# This agent reads code artifacts (git diff, changed-files.json, etc.)
# and produces .closedloop-ai/context/code-context.json with token-budgeted compression

# investigation-log.md is optional secondary context for code judging
if [ ! -f "$CLOSEDLOOP_WORKDIR/investigation-log.md" ]; then
  echo "WARNING: investigation-log.md unavailable. Continuing code judges with canonical code context only."
fi

# Verify canonical code context exists after context manager completes. The root
# code-context.json path is fallback-only for old runs.
if [ ! -f "$CLOSEDLOOP_WORKDIR/.closedloop-ai/context/code-context.json" ] && [ ! -f "$CLOSEDLOOP_WORKDIR/code-context.json" ]; then
  echo "ERROR: Context preparation failed - .closedloop-ai/context/code-context.json not found"
  # Abort with error CaseScore for all judges
  # Generate error report with final_status=3, justification="Context preparation failed"
  exit 1
fi

# Build and validate code-mode judge-input.json with scripts/judge_input_mapping.py.
# The mapper prefers .closedloop-ai/context/code-context.json as primary and
# preserves root code-context.json as a one-run legacy fallback when needed.

For PRD artifacts (--artifact-type prd):

PRD mode does NOT use context-manager-for-judges. Context preparation is lightweight: verify the PRD document exists, then build judge-input.json directly from it.

# PRD mode context prep: check prd.md exists
if [ ! -f "$CLOSEDLOOP_WORKDIR/prd.md" ]; then
  echo "WARNING: $CLOSEDLOOP_WORKDIR/prd.md not found. Skipping PRD judges."
  exit 0  # Graceful exit — do not fail parent workflow
fi

# Build and validate prd-mode judge-input.json with scripts/judge_input_mapping.py.
# The mapper sets primary_artifact to primary_prd and includes mapped context,
# prompt, repo metadata, prior summaries, and attachments in source_of_truth order.

PRD context prep notes:

• Missing prd.md results in a WARNING and graceful exit (code 0), not an error
• No context manager is launched; judge-input.json is built by scripts/judge_input_mapping.py and validated against schemas/judge-input.schema.json
• Performance: emit sub_step=0 (context_prep, skipped=true) perf event immediately, then proceed to sub_step=1 (batch_1) and sub_step=2 (batch_2)

For Feature artifacts (--artifact-type feature):

Feature mode does NOT use context-manager-for-judges. Context preparation is lightweight: verify feature.md exists, or prd.md exists for legacy Feature inputs, then build judge-input.json from the mapper.

# Feature mode context prep: check feature.md or legacy prd.md exists
if [ ! -f "$CLOSEDLOOP_WORKDIR/feature.md" ] && [ ! -f "$CLOSEDLOOP_WORKDIR/prd.md" ]; then
  echo "WARNING: neither $CLOSEDLOOP_WORKDIR/feature.md nor legacy $CLOSEDLOOP_WORKDIR/prd.md found. Skipping Feature judges."
  exit 0  # Graceful exit — do not fail parent workflow
fi

# Build and validate feature-mode judge-input.json with scripts/judge_input_mapping.py.
# The mapper prefers feature.md and marks fallback_mode.active=true when it must
# use the legacy prd.md Feature path.

Feature context prep notes:

• Missing both feature.md and legacy prd.md results in a WARNING and graceful exit (code 0), not an error
• No context manager is launched; judge-input.json is built by scripts/judge_input_mapping.py with evaluation_type="feature"
• Performance: emit sub_step=0 (context_prep, skipped=true) perf event immediately, then proceed to sub_step=1 (batch_1), sub_step=2 (aggregate), sub_step=3 (validate)
• Preamble: use feature_preamble.md for all 3 feature judges

If required files are missing:

• Plan mode: Exit gracefully with code 0 (do not fail parent workflow)
• Code mode: Exit with error if context preparation fails
• PRD mode: Exit gracefully with code 0 if prd.md is not found
• Feature mode: Exit gracefully with code 0 if prd.md is not found

Artifact Type Configuration

The run-judges skill supports three artifact types with different judge configurations:

Plan Artifacts (Default)

• Judges: 16 total
• Batches: 4 sequential batches (max 4 concurrent per batch)
• Output: plan-judges.json
• Report ID: {RUN_ID}-plan-judges
• Validation: --category plan (16 judges expected)

Code Artifacts (--artifact-type code)

• Judges: 11 total (excludes goal-alignment-judge, verbosity-judge)
• Batches: 3 sequential batches (max 4 concurrent per batch)
• Output: code-judges.json
• Report ID: {RUN_ID}-code-judges
• Validation: --category code (11 judges expected)

Code Judge Batches:

Batch 1: Core Principles (4 judges)

• judges:dry-judge
• judges:ssot-judge
• judges:kiss-judge
• judges:code-organization-judge

Batch 2: Best Practices + SOLID Principles (4 judges)

• judges:custom-best-practices-judge
• judges:readability-judge
• judges:solid-isp-dip-judge
• judges:solid-liskov-substitution-judge

Batch 3: Technical Quality + Testing (3 judges)

• judges:solid-open-closed-judge
• judges:technical-accuracy-judge
• judges:test-judge

PRD Artifacts (--artifact-type prd)

• Judges: 5 total
• Batches: 2 sequential batches (max 4 concurrent per batch)
• Output: prd-judges.json
• Report ID: {RUN_ID}-prd-judges
• Validation: --category prd (5 judges expected)
• Canonical input: $CLOSEDLOOP_WORKDIR/judge-input.json produced by scripts/judge_input_mapping.py, with primary_prd normally pointing to $CLOSEDLOOP_WORKDIR/prd.md

Feature Artifacts (--artifact-type feature)

• Judges: 3 total (feature-completeness-judge, prd-testability-judge, prd-dependency-judge)
• Batches: 1 batch (max 4 concurrent per batch)
• Output: feature-judges.json
• Report ID: {RUN_ID}-feature-judges
• Validation: --category feature (3 judges expected)
• Canonical input: $CLOSEDLOOP_WORKDIR/judge-input.json produced by scripts/judge_input_mapping.py, with primary_feature normally pointing to feature.md and legacy fallback to prd.md
• Preamble: use feature_preamble.md (Feature-shaped contract; do NOT substitute prd_preamble.md)

Feature Mode Execution:

Batch 1: Feature Quality (sub_step=1)

• judges:feature-completeness-judge — evaluates Feature request completeness and clarity
• judges:prd-testability-judge — evaluates requirement testability
• judges:prd-dependency-judge — evaluates dependency clarity and completeness

PRD Mode Execution:

Batch 1: Structure & Completeness (sub_step=1)

• judges:feature-completeness-judge — evaluates Feature request completeness and clarity
• judges:prd-auditor — structural completeness audit of the PRD
• judges:prd-scope-judge — evaluates scope definition and boundary clarity

Batch 2: Quality Gates (sub_step=2)

• judges:prd-dependency-judge — evaluates dependency clarity and completeness
• judges:prd-testability-judge — evaluates requirement testability

Step 1: Launch Judge Agents in Parallel

Performance: For each batch/phase, run "start of phase" Bash before launching the batch and "end of phase" Bash after the batch completes. Plan: batch_1=sub_step 1, batch_2=sub_step 2, batch_3=sub_step 3, batch_4=sub_step 4. Code: batch_1=sub_step 1, batch_2=sub_step 2, batch_3=sub_step 3. PRD: batch_1=sub_step 1, batch_2=sub_step 2. Feature: batch_1=sub_step 1.

Constraint: The Task tool supports maximum 4 concurrent agents per batch.

Action: Launch judges in sequential batches based on artifact type.

<judge_batches>

Plan Artifact Judge Batches (16 judges, 4 batches)

Batch 1: Core Principles (DRY/SSOT/KISS + Organization)

Batch 2: Best Practices + Response Quality

Batch 3: SOLID Principles

Batch 4: Plan Grounding + Testing

PRD Artifact Judge Batches (5 judges, 2 batches)

Batch 1: Structure & Completeness (sub_step=1)

Batch 2: Quality Gates (sub_step=2)

Feature Artifact Judge Batches (3 judges, 1 batch)

Batch 1: Feature Quality (sub_step=1)

Excluded judges (feature mode):

• judges:prd-auditor — excluded because it assumes US-###/AC-#.# numbering and multi-story traceability that Feature artifacts do not follow
• judges:prd-scope-judge — excluded because it assumes In/Out-of-Scope sections that are not present in Feature artifacts

</judge_batches>

<prompt_template>

Preamble Injection

Before invoking each judge, prepend the common and artifact-specific preambles:

1. Locate preamble files:

   • skills/artifact-type-tailored-context/preambles/common_input_preamble.md
   • skills/artifact-type-tailored-context/preambles/{artifact_type}_preamble.md
   • Use Glob tool to find: **/artifact-type-tailored-context/preambles/*.md
   • Validate both files exist (fail with error CaseScore if either is missing)

2. Read preamble content:

   • Read common_input_preamble.md
   • Read {artifact_type}_preamble.md
   • Validate combined preamble size is reasonable for judge context (target: < 8000 characters)

3. Concatenate:

   • common_input_preamble + "\n\n---\n\n" + artifact_preamble + "\n\n---\n\n" + judge_prompt
   • common_input_preamble.md is the only runtime source of judge input-loading contract text; judge-specific agent files should not duplicate that contract.

4. Pass to judge: Use concatenated prompt as judge's full prompt

If either preamble file is missing:

• Generate error CaseScore with final_status=3, justification="Preamble file not found: {path}"
• Continue with other judges

NOTE — Feature Mode: When --artifact-type feature is used, resolve {artifact_type}_preamble.md as feature_preamble.md (not prd_preamble.md). The Feature preamble frames the input as a Feature artifact (evaluation_type=feature, lightweight structure, no PRD-only sections such as US-###/AC-#.# numbering or In/Out-of-Scope) and aligns with the envelope built by feature mode. Substituting prd_preamble.md would inject contradictory contract instructions and may cause judges to error or evaluate against PRD-only expectations.

Prompt Templates

For plan artifacts:

WORKDIR=$CLOSEDLOOP_WORKDIR. Read $CLOSEDLOOP_WORKDIR/judge-input.json first.
Evaluate according to `task` and `source_of_truth` ordering.
Treat the envelope's `primary_artifact` as authoritative.
If `fallback_mode.active=true`, use fallback artifacts specified in the envelope.

For code artifacts:

WORKDIR=$CLOSEDLOOP_WORKDIR. Read $CLOSEDLOOP_WORKDIR/judge-input.json first.
Evaluate according to `task` and `source_of_truth` ordering.
Treat the envelope's `primary_artifact` as authoritative.
Apply your {judge_name} criteria to assess code quality.

For PRD artifacts:

WORKDIR=$CLOSEDLOOP_WORKDIR. Read $CLOSEDLOOP_WORKDIR/judge-input.json first.
Evaluate according to `task` and `source_of_truth` ordering.
Treat the envelope's `primary_artifact` as the authoritative PRD document and load supporting descriptors as source-of-truth evidence.
Apply your {judge_name} criteria to assess PRD quality.

For Feature artifacts:

WORKDIR=$CLOSEDLOOP_WORKDIR. Read $CLOSEDLOOP_WORKDIR/judge-input.json first.
Evaluate according to `task` and `source_of_truth` ordering.
Treat the envelope's `primary_artifact` as the authoritative Feature document and load supporting descriptors as source-of-truth evidence.
Apply your {judge_name} criteria to assess Feature quality.

</prompt_template>

Expected Output Format

<expected_output> Each judge returns a CaseScore JSON object:

{
  "type": "case_score",
  "case_id": "dry-judge",
  "final_status": 1,
  "metrics": [
    {
      "metric_name": "dry_score",
      "threshold": 0.8,
      "score": 0.85,
      "justification": "Plan follows DRY principles..."
    }
  ]
}

Status Code Semantics:

</expected_output>