Reverse Spec From Code

reverse-spec-from-code

Turn spec-less code into OpenSpec capability specs so kb_search has high-signal, consistently-formatted behavioral documents to index. Tuned via a blind generate→judge loop against 6 real specs: requirement coverage 97%, scenario coverage 91% (see docs/research/reverse-spec-from-code.md for the tuning record

• a model-loss test across opus / deepseek-flash / haiku).

Scratch MUST live OUTSIDE openspec/ (kb indexes openspec/). Use the gitignored repo-root dir .reverse-spec-scratch/ — otherwise every draft pollutes kb_search with duplicate spec chunks. Promotion MOVES the file into openspec/specs/ (the only kb-indexed copy).

When to use

• A package/directory under packages/ has behavior but no openspec/specs/<cap>/spec.md.
• You want to enrich kb_search (it indexes openspec/ markdown) with behavioral specs.
• An existing spec is stale and you want a code-current reconciliation (--refresh).

Skip for a single trivial file, or when the capability already has an accurate spec.

Core principle (the lever that matters)

A capability's contract is not confined to one file. The single biggest quality driver is making each generator FOLLOW the behavioral contract across file boundaries: every emitted message/event, registry write, spawned/killed process, config read, or DOM attribute is a contract with another component and must be spec'd too. In tuning this moved requirement coverage from 40% to 95% on the cross-cutting capability. The generator prompt (prompts/generator.md) enforces this in STEP 1 — do not weaken it.

Fitness, honestly

"Match an existing spec" is a PROXY, not the goal. Real specs drift from code. The goal is a spec that accurately describes current code and is searchable. Target: high requirement coverage + zero code-ungrounded hallucination. Code-current divergence from a stale spec is a win, not a miss.

Procedure

1. Resolve target + scope. User names a directory/package (e.g. packages/server) and optionally a single capability. Confirm the target path exists.

2. Discover capability boundaries. Spawn ONE discovery subagent (prompts/discovery.md, model @compact is fine) that clusters the target's files into capabilities using the directory AGENTS.md tree (kb agents <dir>, kb_search --doc-type agents) + grep. It returns a manifest: [{ capability, purpose_hint, files[] }]. For a single-capability target you may skip this and build the manifest by hand.

3. Skip already-specced capabilities. For each manifest entry, if openspec/specs/<capability>/spec.md exists and --refresh was NOT requested, drop it (report as skipped). With --refresh, keep it and reconcile.

4. Generate in parallel (blind). Fan out ONE generator subagent per remaining capability IN A SINGLE MESSAGE (prompts/generator.md). Each reads code only — never an existing spec — and writes .reverse-spec-scratch/<capability>/spec.md. Pass capability, purpose_hint, start files, and the output path. Model: a fast/cheap model (@fast/@compact) is viable AS LONG AS the format gate (step 6.5) and the @research auditor run — see "Model choice" below.

5. Audit in parallel (code-grounding). Fan out ONE auditor subagent per generated spec IN A SINGLE MESSAGE (prompts/auditor.md, model @research). Each verifies the generated spec against the ACTUAL code and returns strict JSON: hallucinated_requirements[] (in spec, not in code), missing_behaviors[] (in code, not in spec), format_ok, verdict (pass|revise). No real spec is needed — the code is the oracle.

6. Revise if needed. For any spec with verdict: revise, re-spawn its generator with the auditor's findings appended (remove the listed hallucinations, add the listed missing behaviors). One revise pass is usually enough; re-audit only if the first audit was severe.

6.5. Format gate (openspec validate) — HARD, deterministic. openspec validate only reads specs under openspec/specs/, so validate each scratch spec via a throwaway id, then delete it:

for c in <cap1> <cap2> ...; do
  d="openspec/specs/_rsfc-val-$c"; mkdir -p "$d"
  cp ".reverse-spec-scratch/$c/spec.md" "$d/spec.md"
  openspec validate "_rsfc-val-$c" --type spec 2>&1 | grep -qi "is valid" \
    && echo "$c: VALID" || echo "$c: INVALID"
  rm -rf "$d"
done

Any spec that is INVALID is treated exactly like verdict: revise with reason "format: openspec validate failed" — re-spawn its generator emphasizing the FORMAT rule (no tables, no bold **Scenario:**, no numbered requirements), then re-run this gate. A spec that fails validate is NEVER promoted. Cheap generator models fail here most often — this gate is what makes them safe.

7. Present + promote on confirm. Show the user: per-capability spec path, requirement count, and audit + validate summary (skipped / passed / revised / valid). Only specs that BOTH audit-pass AND validate-pass are promotable. Use ask_user (confirm or multiselect) to choose which to promote. On confirm, MOVE .reverse-spec-scratch/<cap>/spec.md → openspec/specs/<cap>/spec.md (create the dir; move, don't copy, so no duplicate stays under an indexed root). NEVER write openspec/specs/ without explicit confirm.

8. Verify KB indexing. After promotion, run kb_search "<a phrase from a new spec>" to confirm the spec is discoverable. Report the result.