Question Router

Skill: Question Router

Purpose

Classify incoming user questions into complexity levels (L1-L5) and route them to the appropriate response path.

When to Use

• At the start of every user interaction that looks like an analytical request
• Before launching the full 18-step pipeline
• When the user asks a follow-up question mid-analysis

Classification Levels

L1: Factual Lookup

Pattern: User wants a specific number or fact from the data. Examples:

• "How many users signed up in March?"
• "What's the average order value?"
• "How many products are in the electronics category?"

Response path: Query the data directly. Return the answer with source citation (table, column, filter). No agents needed.

L2: Simple Comparison

Pattern: User wants to compare two things or see a breakdown. Examples:

• "Compare conversion rates by device"
• "Show me revenue by category"
• "What's the split of users by acquisition channel?"

Response path: Query + quick chart. Use chart_helpers directly. Apply Visualization Patterns skill. No full pipeline.

L3: Guided Analysis

Pattern: User has a specific analytical question requiring multiple steps. Examples:

• "Why did conversion drop last month?"
• "Which user segment has the highest LTV?"
• "Is our new checkout flow performing better?"

Response path: Subset of the pipeline — Frame → Explore → Analyze → Validate → Present findings. Skip storyboard/deck unless requested. Use 3-5 agents.

L4: Deep Investigation

Pattern: User needs root cause analysis, opportunity sizing, or experiment design. Examples:

• "Investigate why mobile revenue dropped 15% in Q3"
• "Size the opportunity if we fix the cart abandonment issue"
• "Design an A/B test for the new pricing page"

Response path: Full pipeline minus deck. Frame → Hypothesize → Explore → Analyze → Root Cause → Validate → Size → Present findings. Use 6-10 agents.

L5: Full Presentation

Pattern: User wants a complete analysis with a polished slide deck. Examples:

• "Run the full pipeline on Q4 performance"
• /run-pipeline
• "Build me a board-ready deck on our retention problem"

Response path: Complete 18-step pipeline. All agents, full storyboard, charts, narrative, and Marp deck.

Classification Algorithm

Fast-Path Detection (optional shortcut for obvious L1 questions)

Before running the full classification workflow, check if the question matches an obvious L1 pattern. If YES, skip to L1 execution immediately. If NO or UNCERTAIN, proceed to Step 0.

Trigger phrases for fast-path L1:

• Starts with "how many" or "how much"
• Starts with "what's the total" or "what's the average"
• Starts with "count of" or "number of"
• Contains "count" + single metric/entity + optional time filter
• Examples:
  • "how many orders last month?"
  • "what's the total revenue in Q4?"
  • "count of users who converted"

Do NOT fast-path if:

• Question contains "compare", "by", "breakdown", or "split"
• Question contains "why", "investigate", or "analyze"
• Question mentions multiple metrics or dimensions
• Question references a product change or hypothesis
• You're uncertain whether it's genuinely simple

Why fast-path matters: L1 questions don't benefit from the full classification overhead. The user wants a quick answer, not a classification report. Fast-path saves ~40 seconds and ~7-9k tokens for simple lookups.

Output format for fast-path L1:

• Answer the question directly
• Include source citation (table, column, filter)
• Offer 2-3 contextual next actions
• Skip the classification documentation

If you use the fast-path, you're done — don't proceed to Step 0 or beyond.

---

Step 0: Pre-flight (runs on every query before classification)

Enrichment steps — never block routing. If any sub-step fails, skip it silently. IMPORTANT: Only report pre-flight findings if they actually find something. Silent skip if nothing found.

1. Feedback check — If the message is a correction or feedback with no analytical question, hand it to the log-correction skill and skip routing.

2. Entity disambiguation — If the entity index is loaded (from bootstrap):

   • Call resolve_entity(query_text, entity_index) from helpers/knowledge/entity_resolver.py.
   • If matches found, call format_disambiguation(matches) and set {{RESOLVED_ENTITIES}} for downstream agents.
   • Example: "why is cvr dropping?" → Resolved: 'cvr' -> conversion_rate (metric)

3. Corrections check — Read .knowledge/corrections/index.yaml.

   • If total_corrections > 0 for the active dataset, set {{CORRECTION_COUNT}} so analysis agents check the correction log before writing SQL (e.g., known join pitfalls, filter requirements).

4. Dataset detection — Before classifying, check whether the question references a dataset other than the currently active one.

   • Read .knowledge/datasets/ to get all known dataset IDs and display names.
   • Scan the user's question for exact or fuzzy matches to any dataset name.
   • If a non-active dataset is referenced:
     • Inform the user: "It looks like you're asking about {display_name}, but the active dataset is {active_display_name}."
     • Offer: "Want me to switch? (/switch-dataset {id})"
     • Do NOT proceed with analysis until the user confirms which dataset to use.

5. Archaeology note — The Query Archaeology skill provides SQL pattern context (prior queries, reusable CTEs) to analysis agents when available. No action needed here — just acknowledge it flows downstream automatically.

After pre-flight completes, proceed to Step 1.

---

Step 1: Parse the question

Extract:

• Subject: What entity/metric is being asked about?
• Action: Lookup, compare, analyze, investigate, or present?
• Scope: Single metric, breakdown, multi-dimensional, or end-to-end?
• Output expectation: Number, chart, findings, or deck?

Step 2: Classify by the strongest signal

Classify by the strongest signal present, in this order: /run-pipeline or "deck / presentation / slides" → L5; "why / investigate / root cause", sizing or opportunity, experiment or A/B test → L4; "analyze / what's happening with", or a question with several sub-questions → L3; "compare / by {dimension} / breakdown / split" → L2; a single number → L1. A "quick" or "just" qualifier drops one level. When two levels are equally supported, take the lower one (prefer the faster response).

Step 3: Adapt from user profile

If .knowledge/user/profile.md exists, read the user's preferences:

• Detail level = "executive-summary": Bias one level down (L3 → L2)
• Detail level = "deep-dive": Bias one level up (L2 → L3)
• Technical level = "advanced": Show more SQL, skip explanations
• Technical level = "beginner": Add more context, explain terms

Step 3.5: Pace Mode Selection (L3+ only; skip for L1/L2)

Pace mode is orthogonal to complexity level. Level decides which agents run; pace decides how visible the machinery is. Any level can run in any mode.

Default when no signal is clear: narrated. Never default to guided (blocks the user) or autopilot (hides the work). Narrated is the failure-safe middle ground.

Auto-detection signals

Read these from the user's message and session context. Pick guided on teaching or pacing language ("walk me through", "teach me", "step by step", "slow down", "one step at a time") or a technical_level: beginner profile. Pick autopilot on "just run it / silent / don't narrate", a terse task-like prompt ("conversion by device last week"), a mid-analysis follow-up ("now break by country"), or an advanced profile in a session that has already run several analyses. Otherwise, including a bare /run-pipeline, pick narrated. A /pace choice persisted this session overrides all of these (see below).

Persisted mode (survives across phases and sessions)

On every router run, read working/session_state.yaml. If pace_mode is set, use it as the starting mode regardless of auto-detected signals — an explicit user choice beats heuristics.

The /pace skill writes this key. On session resume (/resume-pipeline), the persisted mode is honored.