AI Prototyping

AI Prototyping

AI app builders, AI design tools and AI coding assistants turn an idea into a clickable or working prototype in hours. That changes the economics of discovery and adds three failure modes: prototyping what should have been specified, mistaking a polished demo for validation, and shipping prototype code by accident.

The agent runs the whole loop: decide whether to prototype, pick the fidelity rung, brief the generation tool, test with real users against pre-set thresholds, and hand off with everything a prototype cannot carry. Two stdlib tools gate the steps teams skip most: untestable hypotheses and incomplete handoffs.

When to use

• A team wants to "just build it with AI" and someone must decide if that is the right move
• Choosing between a clickable mock, a working prototype on synthetic data, or a spec
• Writing the brief for an AI prototyping tool so the output is testable
• Planning a prototype test: tasks, participants, success and kill criteria
• Handing a validated prototype to engineering without it becoming production code by default
• Setting governance for prototype tools: data, security, IP, accessibility

When NOT to use: the dominant risk is viability (pricing, margin, legal) - model it; the solution is already known (parity feature, regulatory requirement) - write the spec; there is no evidence the problem exists - interview users first; you need a rate with a confidence interval - run a powered quantitative test.

Clarify First

Before planning, confirm these inputs. If any is unknown or vague, ASK - do not assume:

• Hypotheses and their uncertainty type - desirability, usability, feasibility or viability (drives the rung, and whether a prototype can answer it at all)
• Pre-set success threshold per hypothesis - the bar that validates or kills it (without it, any result reads as success)
• Data constraints - is real customer data approved for prototype tools, or synthetic only? (caps fidelity at F3 until approved)
• Stage - planning, mid-test, or handing off (selects which tool runs)

Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.

Quick Start

# 1. Plan: fidelity, method, sample, success + kill criteria per hypothesis
python3 project-management/discovery/ai-prototyping/scripts/prototype_plan.py \
  --input hypotheses.json                      # exit 2 = a hypothesis is untestable

# 2. Brief the tool with assets/prototype_brief_template.md, test, record findings

# 3. Gate the handoff (JSON or the markdown template)
python3 project-management/discovery/ai-prototyping/scripts/prototype_handoff_checker.py \
  --input handoff.md --strict                  # exit 2 = handoff incomplete

Tools Overview

Both support --format markdown (default) or --format json (wrapped as {"schema", "generated_at", "data"} per SHARED_OUTPUT_SCHEMA.md) and --output <file>.

Exit code contract [PROVEN]

Workflow

Step 1 - Prototype or spec? [RECOMMENDED]

Prototype to learn; write the spec to decide. When risk is spread across types, do both in parallel.

Step 2 - Pick the fidelity rung

Start at the lowest rung that can falsify the hypothesis; climb only when it passed or cannot test the risk by construction. Generate 2-3 variants (one per candidate solution), not one polished version. Traps per rung and worked decisions: references/prototype-decision-guide.md.

Step 3 - Brief the generation tool

Write the brief before generating, one per variant, from assets/prototype_brief_template.md: (1) role and disposability, (2) user and context, (3) the job in the user's words, (4) explicit screen and state list including error states the tasks hit, (5) synthetic data only, (6) constraints incl. an accessibility baseline, (7) out of scope. When output is structurally wrong, edit the brief and regenerate - do not hand-patch.

Step 4 - Test with users, not at them [PROVEN]

• Pre-register success and kill thresholds before session 1; recruit the target segment, not colleagues.
• Goal-based tasks ("deal with the expense that breaks policy"), never UI instructions ("click the red badge").
• The user drives; the facilitator is silent; stuck counts as failure.
• Measure behavior per task (unassisted success, time, errors). For desirability, end with a commitment ask and count commitments, not compliments.
• Report against the bar: "5 of 6 completed task 2 unassisted against a bar of 5 of 6".

Participant numbers from the planner are planning defaults, not statistical guarantees. Demo-ware bias checklist: references/testing-handoff-governance.md.

Step 5 - Write down what the prototype cannot carry

The spec/PRD still owns: problem and users with evidence; success metrics (baseline, target, window); constraints; non-goals; risks and open hypotheses; data and privacy decisions.