Product Research

Product Research

The operational layer of continuous product discovery: choosing a method that actually answers the question asked, recruiting the right people without poisoning the sample, running interviews that surface behaviour rather than opinion, and converting a pile of session notes into insights with an honest confidence attached.

When to use this skill

• A team is about to build something and the evidence behind it is three sales anecdotes and a strongly held opinion
• Choosing a method — someone has asked for "a survey" or "some user interviews" before anyone has written down the question
• Designing a screener for a study where recruiting the wrong participants would be worse than not running it
• Writing an interview guide that has to be run consistently by several people across a dozen sessions
• Synthesising evidence into insights after a round of discovery, with a defensible confidence level on each claim
• Standing up a continuous discovery cadence — a repeatable weekly rhythm rather than one-off project research

Inputs the skill expects

• The decision the research feeds, and who makes it
• The question in interrogative form — what you do not know, not what you want confirmed
• Decision reversibility — can this be undone in a sprint, or is it a one-way door
• Timeline and budget for the study
• Access to participants: existing customers, prospects, panel, or none
• Existing evidence already on hand (tickets, session recordings, sales calls)

Clarify First

Before generating, confirm these inputs. If any is unknown or vague, ASK — do not assume:

• The decision this research informs, and its reversibility — a one-way door justifies weeks of evidence; a reversible change is often better answered by shipping an experiment
• The question in interrogative form — "do users want X" and "how do users currently accomplish X" call for completely different methods
• Participant access — whether you can reach real users determines whether the plan is feasible at all, and it is the constraint teams discover last
• Timeline — a two-day answer and a three-week answer are different studies, not the same study rushed

Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.

Workflows

Workflow 1 — Pick the method before anyone books a session

1. Write the question in interrogative form. If it starts with "should we," it is a decision, not a research question — rewrite it as what you would need to know to decide.
2. Classify the question: generative (what is going on), evaluative (does this work), or descriptive (how many, how often).
3. Rate decision reversibility and state the timeline and participant access.
4. Run the recommender. It returns a primary method, a cheaper fallback, the minimum sample, and the methods it explicitly ruled out with reasons.
5. If the recommendation is "ship an experiment instead," take that seriously. For reversible decisions, an experiment usually beats a study on both speed and evidence quality.

python3 research-ops/product-research/scripts/method_recommender.py \
  --input research-ops/product-research/assets/sample_research_question.json \
  --format text

Workflow 2 — Validate the screener before recruiting opens

1. Draft the screener: qualifying criteria, disqualifying criteria, and the items that test each.
2. Run the validator. It checks for transparent qualifying answers, missing disqualification logic, professional-respondent exposure, quota coverage, and criteria that no item actually tests.
3. Fix every fail. A screener defect costs you the whole study — you find out only during the sessions, by which point the incentives are spent.

python3 research-ops/product-research/scripts/screener_validator.py \
  --input research-ops/product-research/assets/sample_screener.json \
  --format text

Workflow 3 — Score insight confidence during synthesis

1. Draft each candidate insight as a claim, and attach the evidence items that support it — each with its source type, participant, and whether it is observed behaviour or reported opinion.
2. Run the scorer. It weights observed evidence above reported evidence, rewards source and participant diversity, and penalises claims resting on a single session or a single channel.
3. Ship only the insights scoring moderate or above as decision inputs. Everything below that is a hypothesis and must be labelled as one.

python3 research-ops/product-research/scripts/insight_confidence_scorer.py \
  --input research-ops/product-research/assets/sample_evidence.json \
  --format text

Decision frameworks

Method by question type

The pattern worth internalising: quantitative methods tell you what and how many; qualitative methods tell you why and how. Reaching for interviews to answer a "how many" question, or for a survey to answer a "why" question, is the most common and most expensive method error in product research.

Reversibility gate

[PROVEN] Match evidence spend to reversibility, not to how interesting the question is. The most common research-ops failure is not too little research — it is expensive research on reversible decisions while one-way doors get decided on intuition.