Lemonade Router Builder

Lemonade Router Config Generator

Generate a collection.router policy JSON from a plain-English description of how requests should be routed. The skill produces and validates the JSON only - it does not call the live server, register the policy, or run requests through it. The JSON is accepted by the strict server-side parser on the first try and stays editable in the desktop app's Hybrid Router editor.

Prerequisites

• Lemonade Server v11.5.0+ running locally (lemonade server start). Required only to register and test the generated policy - the skill itself (JSON generation + offline validation) works without a live server.
• No GPU or ROCm dependency for authoring. The router policy is a JSON document; no hardware is needed to generate or validate it.
• Python (any 3.x) in PATH - used by the bundled offline validator (scripts/validate.py). No extra packages required.

The router picks one candidate model per request. Two authoring modes exist, and choosing the right one is the first decision:

routing.router is mutually exclusive with routing.rules and routing.classifiers - never emit both.

Step 1 - Extract from the user's words

• Candidates: the models that may answer requests. Verbatim model names (e.g. Gemma-3-4b-it-GGUF). If the user names none, ask - never invent model names. lemonade list or GET /api/v1/models shows what's available. A name the user did give may still not exist on the target host - the offline validator can't check that (Step 8b closes the gap).
• Default / fallback: which candidate gets everything that matches nothing. If unstated, use the model the user framed as "local", "small", or "safe"; otherwise the first candidate mentioned.
• Signals: every condition mentioned (keywords, patterns, length, images, tools, topics, PII, safety) and which model each one routes to.
• Classifier models: models named for detection rather than answering (BERT-style encoders, embedding models, an LLM used as judge).

Step 2 - Scaffold

Always exactly this envelope (the parser rejects unknown or missing keys):

{
  "version": "1",
  "model_name": "user.MyHybridRouter",
  "recipe": "collection.router",
  "components": [],
  "routing": { }
}

• version is the literal string "1".
• model_name must start with user.; slug from the user's description if they gave a name (user.<Name> using only [A-Za-z0-9._-]). If they didn't name it, derive one from context instead of a fixed literal - e.g. user.<slug-of-default-candidate>-Router - so two different policies don't collide by default. /pull is idempotent per model_name: registering a second policy under the same name silently overwrites the first. If this conversation already produced an unnamed router, don't reuse the same derived name for the next one - ask, or pick a visibly different name.

Step 3 - Candidates and default

"candidates": ["<answering models>"],
"default_model": "<one of candidates>"

default_model MUST be listed in candidates. Candidates should be chat-capable LLMs - not embedding, classification, or image models.

Step 4 - Mode A: LLM-as-router

"router": {
  "type": "llm",
  "model": "<small chat LLM>",
  "prompt": "You route user requests to the best model. <one sentence per candidate: when to pick it, using the exact model name>."
}

• model defaults to the most capable candidate, not the cheapest one. The router judges every single request that flows through the policy, so a weak judge silently misrouting everything is a worse default than the extra cost of a stronger one. State the choice in the summary you give the user - router.model: <chosen> (most capable candidate available; pick a smaller dedicated judge model yourself for lower per-request cost, at the risk of the failure mode below). If the user already named a separate model for this role, use that instead of a candidate.

• If default_used stays true across varied test prompts even after fixing the prompt (see the bullet below and Step 9), the fix is a more capable router.model, not a further prompt edit - this is a judge-model-capability limit, not something prompt wording alone can solve.

• Write intent only - never specify a reply format and never use imperative "Pick X" phrasing. The engine unconditionally appends its own contract after your prompt: it lists the candidate names and demands a strict JSON reply {"model": "<name>", "rationale": "<one sentence>"}, then falls back to default_model on any deviation. A prompt that says "reply with ONLY the model name", "Pick Model-A", "respond with the model name", or similar is wrong about the wire format and causes weaker judge models to reply with a bare string that fails to parse - silently falling back to default_model on every request with no visible error.

  Bad (do not write): "Pick Qwen3.5-9B-GGUF for sensitive queries, pick Qwen3.5-9B-NoThinking for everything else." Good: "Route to Qwen3.5-9B-GGUF when the request appears sensitive or contains personal information. Route to Qwen3.5-9B-NoThinking for all other requests."

  Only describe when each candidate is appropriate. Never say "pick", "output", "reply with", or "respond with".

• NEVER emit rules or classifiers in this mode. The routing object in Mode A must contain exactly: candidates, default_model, and router. Adding rules or classifiers alongside router is a schema violation that the server parser rejects. If you catch yourself writing both, stop and remove rules/classifiers entirely.

Step 5 - Mode B: classifiers

Only declare classifiers the rules actually reference. Three types:

{ "id": "clf-1", "type": "classifier", "model": "<classification model>",
  "labels": ["PII", "Jailbreak"], "default_label": "PII", "on_error": "match_false" }

{ "id": "clf-2", "type": "semantic_similarity", "model": "<embedding model>",
  "reference_phrases": { "shopping": ["I want to shop for pants", "add to cart"] },
  "default_label": "shopping", "on_error": "match_false" }

{ "id": "clf-3", "type": "llm", "model": "<chat LLM>",
  "prompt": "Classify the request into only labels SAFE, RISKY",
  "labels": ["SAFE", "RISKY"], "default_label": "SAFE", "on_error": "match_false" }

Hard constraints (parser-enforced - see reference.md for the full matrix):

• classifier type: model should be a text-classification model (an onnxruntime encoder like Bert-Phishing-ONNX); labels must match the model's actual output labels - unverifiable offline, and a mismatch silently scores 0.0 forever (see reference.md's classifier notes for why, Step 8b for how to catch it). A chat LLM here is legal (LLM-as-classifier via chat) but prefer type: "llm" for that - it is explicit and prompted.
• semantic_similarity: reference_phrases is {concept: [phrases...]}, at least one concept, each with at least one phrase. Concept names ARE the labels - a labels key is rejected for this type. Model must be an embedding model. Give 3–5 varied phrases per concept when inventing them.
• llm: prompt AND non-empty labels are both required. Write intent only - never tell the model how to format its reply. The engine appends its own {"model": "<chosen_label>", "rationale": "..."} contract after your prompt (the same contract as routing.router). An authored line like "Reply with exactly one label: SAFE or RISKY" causes weaker models to output bare SAFE, which the parser rejects - the score comes back empty and the rule silently never fires. Describe what makes a request belong to each label; leave the reply format to the engine. If it still never fires after that, see Step 4's judge-capability note above - the same fix applies here (Step 9 shows how to catch it).
• default_label, when present, must be one of the labels/concepts.
• Defaults when unspecified: id = clf-1, clf-2, …; on_error = "match_false" (fail-open: a broken classifier doesn't match, so requests fall through - use "match_true" only when the user wants fail-closed safety); default_label = the first label.