Rethink Surveys

Rethink Surveys

A survey is a conversation with a busy person. Design it like one.

This skill packages a framework for designing, critiquing, and operationalizing surveys that capture real intent — not satisficed checkbox approval. Built from the experience of designing the Proxymate muShanghai survey (which surfaced bugs in ~6 hours of testing that a category-checkbox flow would have hidden for weeks).

Tool affordances (read first)

The bundled rethink-survey-mcp-server exposes the skill's deterministic operations as MCP tools, its canonical content as MCP resources, and three framing prompts as slash-prompts. When available, prefer them over reasoning from memory.

• Use MCP tools when the user wants a specific deterministic result: critique_survey for lint, get_template for a use-case starter, design_survey_session for the staged wizard, score_response for a single response, cluster_responses for a batch.
• Reason from skill content when the user wants to understand: debate a principle, discuss trade-offs, explain why hypotheticals fail, or do anything not exposed as a tool (e.g. /turn-into-app codegen).
• Fetch resources for grounding instead of paraphrasing: rethink://principles, rethink://principles/{id}, rethink://structure/4-part, rethink://question-library/{part}, rethink://use-case/{event|founder|gig}, rethink://scoring/rubrics, rethink://modality/decision-tree, rethink://anti-patterns.
• Slash-prompts the user may invoke: /jarrett-review (Caroline Jarrett-style review), /design-coaching (Socratic design questions), /mom-test-check (Rob Fitzpatrick founder-discovery lint).

If the MCP server isn't connected, all of the below still works from this skill's reference files. For details on tool inputs/outputs, fallback rules, and prompt framing, read references/mcp-integration.md.

Three operating modes (commands)

1. /design-survey — interactive design session: research goal → audience → hypotheses → questions → modality → output schema. Walks the user through 5–8 decisions, each with explicit trade-offs. Reuses templates where appropriate.
2. /critique-survey — paste-in review: identify leading questions, double-barreled items, satisficing bait, missing behavioral anchors, demographic-for-demographic noise. Returns a Jarrett-style critique with concrete rewrites.
3. /turn-into-app — implementation handoff: takes a finalized question set and produces app scaffold (TanStack Start / Next.js / pure HTML, plus Supabase or simple JSON storage). User picks the stack; this command emits the code.

If the user's intent is unclear, ask: "Are you (a) starting fresh, (b) reviewing something you already wrote, or (c) ready to ship — meaning: turn an existing question set into an app?"

The seven design principles (the spine)

Every survey-design decision routes back to one of these. Memorize.

1. Ask about real past behavior, not hypotheticals. "Last time X happened, what did you do?" beats "What would you do if X happened?" by an order of magnitude in signal quality. Hypotheticals invite imagined-self answers.
2. Don't pre-signal the right answer. "Most people find X useful — would you?" is broken. Replace with neutral framing.
3. Start broad and open-ended before narrowing. Open-ended question first → respondent's authentic frame; structured questions after → that frame doesn't get lost in a checkbox.
4. Separate screening / diagnosis / segmentation. Screening = "are you the right respondent?" Diagnosis = "what's actually happening?" Segmentation = "what cohort do you fit into?" Mixing them leads to confusing branching and worse data.
5. No double-barreled questions. "Was the food fast and cheap?" should be two questions. If you can't decompose without losing the meaning, reword.
6. Behavioral anchors over Likert scales when anchors exist. "I'd lose ~30 minutes" is universal; "Somewhat annoying" is whatever the respondent thinks today.
7. Honest length claim on the landing page. If you say 60 seconds, deliver 60 seconds. Lying about length is the single fastest way to nuke completion rate. Jarrett's rule: never lie about length.

For the why behind each principle (Tourangeau's 4-stage cognitive model, satisficing theory, etc.) read references/design-principles.md.

The 4-part hybrid structure

This is the structural skeleton every well-designed survey shares. Use it as a checklist:

For each question, tag it with which part it serves. Questions that don't clearly belong to one part are usually candidates for cutting.

For battle-tested question patterns by part (with EN+ZH parallel text and trade-off notes), read references/question-library.md.

Modality decision tree

Match modality to context:

• Pure form (tap-to-select) — physical QR scan, on-venue, low-attention. Target ≤60s.
• Form with optional voice on key questions — moderate attention; the diagnostic question benefits from authentic phrasing. Target ≤90s.
• AI-conversational interviewer — pre-event email warm-list, deep research mode, willing respondents. AI probes with follow-ups based on prior answers. Target 3–5 minutes, ~10 substantive answers.
• Voice-note only — accessibility-first / language-flexible / low-literacy contexts. Single open prompt, transcribed server-side, post-processed for cluster signals.

Default rule: start with form + optional voice on the discovery question. Add AI-interviewer only if the research goal requires probing depth that a flat form can't deliver.

For the full UX/UI guide (with React/HTML scaffolds and storage schema recommendations), read references/multimodal-ux.md.

What to ask AI conversationally vs. show as form fields

AI is for the parts where you don't know what answers to expect. Form is for the parts where you do.

Scoring & tagging framework

After collection, every response should map to:

• pain_unit — the specific worry/problem captured in the discovery question, ideally with a cluster ID assigned by post-hoc LLM enrichment
• severity (1–5 with behavioral anchors)
• willingness_to_pay (bucket or numeric range, currency-localized for bilingual audiences)
• workaround_strength — derived from whether the respondent has an existing fallback (text length + sentiment)
• segment — derived from screening/segmentation answers
• interview_score — composite for ranking interview-call candidates: severity × specificity × wtp_weight × consent_research_call × novelty_bonus

For the full intent-scoring math (composite formula, tag taxonomy, cluster assignment heuristics), read references/scoring-framework.md.