Source Completeness

Source-to-Spec Completeness Validation

Why This Exists

The analysis pipeline (Layers 1-3) reads source code and produces behavioral specs. But it can miss things:

• Registered commands or tools defined in source but not in the tool catalog
• Environment variables read by the code but not documented
• CLI flags defined in the argument parser but not cataloged
• Subcommands registered dynamically or conditionally
• Config keys accessed in settings but not listed in the config spec
• Events dispatched but not inventoried
• Error categories defined but not documented

The fidelity validation (Layer 7) only checks raw→clean. This skill checks source→raw — it goes back to the actual source code and verifies that every discoverable user-facing surface was captured in the specs.

When to Run

Run AFTER Layer 3 deep documentation is complete, ideally as part of Gate 1 or as a pre-Gate-1 completeness check. If gaps are found, dispatch deep-dive agents to cover the missing areas before proceeding.

Also run AFTER Layer 7 fidelity validation to catch anything the fidelity check couldn't find (because it was never in the raw specs).

The Checks

Search patterns depend on target shape. For a single-file bundle, grep the bundle directly. For a source tree, grep recursively across the source root. Patterns also depend on the target language's idioms for registering each surface — the examples below give common patterns for several languages; adjust to the actual target.

Check 1: Tool / Command Registration Completeness

Applies to targets that expose a registered command, tool, or handler surface (CLIs with subcommands, plugin hosts, RPC servers with method registration).

Source extraction: grep for the registration pattern used by the target. Examples:

# Named-object pattern (common in JS/TS/Go handler registration)
grep -roE 'name:\s*"[A-Z][a-zA-Z]+"' <source-root> | sort -u

# Decorator-based registration (Python)
grep -roE '@(command|tool|handler|register)\(' <source-root>

# Macro-based registration (Rust)
grep -roE '#\[(command|tool|handler)' <source-root>

# Assigned-constant pattern (JS/TS)
grep -roE 'var [a-zA-Z0-9_]+="[A-Z][a-zA-Z]+"' <source-root> | grep -v 'Exception\|Error\|Element'

Spec extraction: every registered name in the tool/command catalog section of the spec.

Diff: every registered name in source must appear in the spec. Missing names are P0-CRITICAL gaps.

Check 2: Environment Variable Completeness

Source extraction: patterns by language:

# Node/TypeScript
grep -roE 'process\.env\.[A-Z_][A-Z0-9_]*' <source-root> | sed 's/.*process\.env\.//' | sort -u
# Python
grep -roE 'os\.environ(\.get)?\(?"[A-Z_][A-Z0-9_]*"' <source-root> | grep -oE '[A-Z_][A-Z0-9_]+' | sort -u
# Go
grep -roE 'os\.Getenv\("[A-Z_][A-Z0-9_]*"\)' <source-root> | grep -oE '[A-Z_][A-Z0-9_]+' | sort -u
# Rust
grep -roE 'env::var\("[A-Z_][A-Z0-9_]*"\)' <source-root> | grep -oE '[A-Z_][A-Z0-9_]+' | sort -u

Spec extraction: all env var names from the environment contract spec.

Diff: categorize missing env vars:

• User-facing (DATABASE_*, AWS_*, GCP_*, LOG_*, and similar) → P0: must document
• Internal-only (test vars, build vars) → P2: note but don't require
• Standard (HOME, PATH, SHELL, TERM) → ignore

Check 3: CLI Flag Completeness

Source extraction:

# Quoted flag strings — works across most CLI frameworks
grep -roE '"--[a-z][a-z0-9-]+"' <source-root> | sort -u

# Framework-specific definitions:
#   Rust/clap:    .arg("--flag-name")
#   Python/argparse: add_argument("--flag-name")
#   Go/flag:      flag.StringVar(&v, "--flag-name", ...)
grep -roE '\.arg\("--[a-z-]+"|add_argument\("--[a-z-]+"|flag\.\w+Var\([^,]+,\s*"--[a-z-]+"' <source-root>

Spec extraction: every --flag name in the CLI contract spec.

Diff: every flag must appear. Hidden flags should be documented as hidden.

Check 4: Subcommand Completeness

Applies to CLIs or REPLs with a subcommand surface.

Source extraction: target-dependent. For subcommand CLIs, grep for the subcommand registration or dispatch table (clap subcommands, Cobra, argparse subparsers). For REPL-style interfaces, grep for the command-prefix pattern (e.g., /help, :quit).

Spec extraction: every subcommand from the CLI interface spec.

Diff: every registered subcommand must appear.