Agentsop Query Routing

Query-Type Routing · Enhancement Overlay

Third-person operating model for a coder agent that owns a multi-handler answering surface. Audience is the LLM agent writing/reviewing the routing code — not the end user.

One sentence: A retriever is shaped by the query type it was built for; a summary index, a vector index, and a text-to-SQL engine are not interchangeable — so classify the query and route first, then retrieve.

This is an ENHANCE overlay. It distills the cross-framework routing pattern from three source skills. For the per-framework API, cross-link the base skill: [[llamaindex]] (RouterQueryEngine), [[agentsop-dify]] (Question Classifier node), [[agentsop-langgraph]] (conditional edges).

---

1. 何时激活 (Activation Rules)

Activate when any of the following holds:

1. The system has multiple indices / tools / engines behind one entry point, and a query must be dispatched to exactly one (or a few) of them.
2. Inbound queries differ in kind — at least two of: lookup ("what does the contract say about termination"), summarize ("give me the gist of doc Y"), compute/aggregate ("how many tickets closed last week"), compare ("diff the 2024 vs 2025 policy").
3. A VectorStoreIndex (or any single retriever) is being stretched to answer query types it was not built for, and quality is uneven across the mix.
4. The user names a routing primitive: RouterQueryEngine, SelectorPromptTemplate, LLMSingleSelector, Dify Question Classifier, LangGraph add_conditional_edges, "intent classifier", "text-to-SQL or RAG".
5. PR review touches a function that reads a query and returns which handler to call.

Do not activate when:

• One index/tool serves every query correctly — routing adds an LLM hop and a failure mode for no benefit (see §6 anti-pattern A1).
• The split is by who is asking (tenant / ACL), not what is asked — that is filtering, route to [[agentsop-multi-tenant-rag]].
• The downstream choice is fixed (always retrieve then summarize) — that is a static edge / linear pipeline, not a router.

---

2. 核心心智模型 (Core Mental Model)

Three principles. Violating any of them produces a router that misroutes silently or routes when it should not.

Principle 1 — One retriever cannot serve all query types

The index taxonomy is not cosmetic. From [[llamaindex]]: a SummaryIndex is a "small, fan-out synthesis" primitive — it reads every node to digest a doc; a VectorStoreIndex is top-k semantic lookup — it reads the few most similar chunks; a text-to-SQL engine answers aggregate/compute queries that no chunk contains the answer to. Ask a vector index to "summarize the whole document" and it returns 4 arbitrary chunks; ask a summary index "what is the late-fee clause" and it fans out over the whole corpus wastefully. The query type names the correct primitive. Routing is the act of recovering that name at runtime.

Operational corollary: index.as_query_engine() over a single VectorStoreIndex answering a heterogeneous query mix is the symptom this skill exists to fix. The fix is per-type handlers + a router on top.

Principle 2 — Route first, retrieve second

Routing is a classification step that runs before any retrieval. It reads only the query (and optionally light context) and emits a destination, not an answer. This ordering is what bounds latency and cost: you pay for the router once, then exactly one downstream handler, instead of fanning out to all of them and merging. LlamaIndex's RouterQueryEngine, Dify's Question Classifier node feeding IF/ELSE branches, and LangGraph's conditional edge over state are the same shape — a selector function (query) -> handler_id evaluated up front. The three frameworks differ only in how the selector is implemented (§7).

Principle 3 — A router is only as good as its destinations' descriptions

Every router — LLM, embedding, or keyword — picks among destinations described in words or examples. In LlamaIndex the signal is the QueryEngineTool.description; in Dify it is the class label + instruction; in LangGraph it is whatever the routing function reads off state plus the node names. From [[llamaindex]] Dilemma 3: "invest in QueryEngineTool.description — it's the only signal the router/agent sees." A misroute is, four times out of five, a bad description, not a bad model. Fix the description before swapping the router type.

The fourth case is genuinely ambiguous queries — for those, Principle of Fallback (§3 Stage 4) applies: route to a safe default, never guess silently.

---

3. SOP 工作流 (Agentic Protocol)

Five stages. Each gates the next. Stop and reconsider at the first "no".

Stage 0 — Confirm routing is warranted

Gate questions:

• Are there genuinely ≥2 handlers with different query-type fit? If no → single index, no router (anti-pattern A1).
• Does real traffic actually span query kinds? Sample 50 real queries and label them. If >90% are one kind → build that handler well; skip the router.
• Is the choice query-dependent (not fixed)? If fixed → static edge.

If all three are yes, continue.

Stage 1 — Enumerate the query types and their handlers

Build the table before writing the router. One row per query kind:

This table is the spec for both the handlers and the router. Mapping mirrors [[llamaindex]] Stage 4 ("Compose for query heterogeneity").

Stage 2 — Build one handler per type, in isolation

Implement and test each engine independently against its own query kind before wiring the router. A misroute is undebuggable if the handlers themselves are wrong. Author each handler's description / label here (Principle 3) — the destination metadata is part of the handler, not the router.

Stage 3 — Write the router (pick the cheapest sufficient family)

Three router families, cheapest to most capable:

Default ladder: start keyword if the types are lexically separable; else LLM selector; reach for embedding when you want a middle point (no LLM hop, better than keyword). Always emit a confidence / score, never just a label.

Stage 4 — Add the fallback default (non-negotiable)

Every router must define behavior for the unrouteable query:

• LLM selector returns low confidence / no match → route to a documented default handler (usually the broad vector index) or an explicit "I can't answer that" path. Never let an ambiguous query silently hit a random branch.
• Set a confidence threshold: below it → fallback, not best-guess.
• Log every routing decision {query_hash, chosen_handler, score, fell_back} so misroutes are observable, not anecdotal.

A router with no fallback is the single most common production failure here (anti-pattern A2).

---

4. 操作模型 (Operation Models)

Format: Trigger / Action / Output / Evidence.

OP-01 EnumerateQueryTypes

• Trigger: A single endpoint fronts heterogeneous queries; no type table exists.
• Action: Sample ≥50 real queries, label each by kind (summarize / lookup / compute / compare / out-of-scope), map each kind to its correct handler + primitive (Stage 1 table).
• Output: A query-type → handler spec table that drives both handler build and router design.
• Evidence: [[llamaindex]] Stage 0 ("What is the query distribution?") + Stage 4 heterogeneity table.

OP-02 BuildPerTypeHandler

• Trigger: Type table exists; handlers not yet built.
• Action: Implement one engine per type (SummaryIndex for digest, VectorStoreIndex for lookup, NL2SQL for compute, SubQuestionQueryEngine for compare). Test each against its own kind in isolation. Author its description/label.
• Output: N independently-verified handlers, each with a destination description (Principle 3).
• Evidence: [[llamaindex]] OP-06 RouteByQueryType; OP-07 DecomposeMultiHop.

OP-03 KeywordRouter

• Trigger: Destinations are lexically distinct; latency/cost-critical path.
• Action: regex/substring rules over the query → handler id; explicit default for no-match. No model call.
• Output: Sub-millisecond, deterministic, free routing for the separable cases.
• Evidence: [[agentsop-dify]] IF/ELSE node over query; [[agentsop-langgraph]] conditional edge as a pure Python predicate (add_conditional_edges).

OP-04 EmbeddingRouter

• Trigger: Destinations semantically (not lexically) distinct; no LLM budget per call.
• Action: Embed the query, cosine-match against pre-embedded destination descriptions / few examples per route; pick argmax; threshold for fallback.
• Output: Cheap, low-latency semantic routing without an LLM hop.
• Evidence: [[llamaindex]] EmbeddingSingleSelector family; router docs.

OP-05 LLMSelectorRouter

• Trigger: Disambiguation needs reasoning; or a query maps to multiple handlers.
• Action: Use an LLM selector (single or multi) reading the query + each handler's description; return chosen id(s) + reasoning. In LlamaIndex: RouterQueryEngine(selector=LLMSingleSelector.from_defaults(), query_engine_tools=[...]).
• Output: Highest-accuracy routing including multi-route fan-out; costs one LLM round-trip.
• Evidence: [[llamaindex]] OP-06 + Dilemma 3; SelectorPromptTemplate.

OP-06 ConfidenceThresholdFallback

• Trigger: Any router that can be uncertain (LLM/embedding) on ambiguous queries.
• Action: Read the selector's score; below threshold → route to documented default (broad vector index) or explicit refuse path. Never best-guess silently.
• Output: Unrouteable queries land somewhere safe and auditable.
• Evidence: [[agentsop-langgraph]] conditional edge can return a "__default__" branch; [[agentsop-dify]] Question Classifier has a built-in "other/else" class.

OP-07 RouteDecisionLogging

• Trigger: Any production router.
• Action: Log {ts, query_hash, chosen_handler, selector_score, fell_back} on every decision; surface a misroute dashboard.
• Output: Misroutes become observable metrics, not user-reported anecdotes.
• Evidence: [[agentsop-dify]] 7-class trace incl. routing; [[agentsop-langgraph]] LangSmith trace of the conditional edge.

OP-08 DescriptionFirstDebug

• Trigger: Router misroutes a known query class.
• Action: Before swapping router type, rewrite the destination description / add 1-2 disambiguating examples; re-evaluate on a labeled routing eval set.
• Output: Most misroutes fixed at the description layer, no model change.
• Evidence: [[llamaindex]] Dilemma 3 ("the only signal the router sees").

---

5. 困境决策案例 (Dilemma Cases)

Dilemma 1 — Router misroutes: improve the classifier or add a fallback?

困境: A 3-way router (summary / lookup / SQL) misroutes ~12% of queries — some lookup queries land on the SQL engine and error out. The team is split: fine-tune / swap to a bigger LLM selector, vs. add a catch-all fallback.

约束:

• Bigger LLM selector adds latency + cost to every query to fix 12%.
• A pure fallback masks the misroute but still sends those queries somewhere generic, lowering answer quality for the 12%.
• The misroutes cluster: most are lookup queries phrased like questions the SQL engine "thinks" it can answer.

决策步骤:

1. Diagnose before fixing — pull the routing log (OP-07); cluster the 12%. The clustering reveals it's a description problem, not a model problem (the SQL engine's description over-claims).
2. Fix descriptions first (OP-08) — tighten the SQL handler's description to "aggregate/count/sum over the orders table only"; add 2 negative examples. Re-run the routing eval set. Misroute drops to ~4%.
3. Then add fallback (OP-06) for the residual ambiguous tail — low-confidence → broad vector index, not SQL.
4. Only if still bad consider the bigger selector — but now on 4%, the cost/benefit usually says no.

结果: Description fix + fallback, no model change. The two moves are complementary, not either/or: improve the classifier signal (descriptions) and add a fallback for the irreducible ambiguity. Mirrors [[llamaindex]] Dilemma 3's "fix the supervisor before switching paradigms" logic and [[agentsop-langgraph]]'s "hitting the limit means the logic is wrong" stance.

可提取的操作: OP-08, OP-06, OP-07.