Mecatl Model Router Config

mecatl model router config builder

Prerequisites

• The operator has a mecatl deployment and an API key for at least one provider (OpenRouter, Anthropic, or OpenAI). The config targets ONE provider — provider is fixed per session, so every alias must resolve to a model id on the same provider.
• Web search is available for live benchmark/pricing lookup.

Workflow

Step 1 — Elicit preferences (BEFORE any search)

Ask one concise question at a time in this order; wait for the answer before the next question. Use ordinary conversational text as the default, with the recommended option stated plainly. Do not render banners, checkmark summaries, or a widget/protocol syntax.

Interpret natural-language answers rather than requiring exact option labels. If an answer is ambiguous, ask a brief follow-up. The operator may revise an earlier answer conversationally at any point (for example, “Actually, use Anthropic”); confirm the changed answer and revisit any dependent choice if necessary.

If the current client explicitly provides a native question UI, it may present the same question and choices there. Never assume it exists or expose its internal protocol in chat.

Q1 — Provider

Ask: “Which provider should this config use? I recommend OpenRouter because one key can reach multiple vendors. OpenRouter, Anthropic direct, OpenAI direct, or another provider?”

Q2 — Priority axis

Ask: “What matters most: balanced cost and capability (recommended), cost-tiered (prefer cheaper models when they can do the job, saving money but possibly trading away capability), or capability-first (prefer the strongest fit, with potentially higher cost)?”

Q3 — Open vs proprietary

Ask: “Are hosted proprietary models acceptable (recommended), or do you require MIT/Apache open-weight models?”

Q4 — Multimodal

Ask: “How often do you need image input: not at all (recommended), rarely, or commonly?”

Q5 — Target ceiling (optional)

Ask: “Is there a target coding ceiling or model you want to match? You can name one, or say there is no specific target.”

Q6 — Existing config (auto-discovered, not asked blank)

Do NOT ask. Silently run cat ~/.config/mecatl/settings.yaml.

• Found with models: block — show it fenced, then ask: “I found an existing models: config. Should I use it as a base (recommended) or start fresh?”
• Not found or no models: block — skip silently, proceed to Step 2.

Record all answers. These determine which models are even candidates.

Step 2 — Search the latest benchmarks & pricing

For each tier the operator needs (heavy/coder/quick, + optional image/specialty), search for:

• SWE-bench Verified (the primary coding benchmark — prefer independent Vals.ai scores over vendor self-reports; note both).
• SWE-bench Pro (harder real-world repos) and Terminal-Bench (agentic multi-step shell — closest proxy to a harness loop) where available.
• LiveCodeBench / HumanEval (pure code-gen) as secondary signals.
• Pricing (input + output per 1M tokens) on the operator's chosen provider — verify the exact OpenRouter/Anthropic/OpenAI model id.
• Modality (text-only vs multimodal) — critical if vision is in scope.
• License (MIT/Apache open vs proprietary) if the operator cares.
• Context window (1M+ is common in 2026; note anything below).

Cross-check at least two sources per model (vendor model card + independent leaderboard like Vals.ai / llm-stats / LLMReference). Flag vendor-reported vs independently-verified scores — they diverge by 2–4 pts regularly.

Present a shortlist table (model | key benchmarks | price | modality | license) for each tier, then recommend one per tier with rationale tied to the operator's stated preferences.

Step 3 — Map to the config schema

Read references/config-format.md for the exact schema and rules. Then build the config:

1. Aliases — the spine. Define heavy/coder/quick (minimum) pointing at concrete provider model ids. Add image (or another specialty alias) only if the operator needs multimodal. Every alias must be on the chosen provider.
2. default — the session model. Usually the heavy alias (the strongest reasoning model), unless the operator wants a cheaper default.
3. Slots — route the four housekeeping calls (compaction/ask-reviewer/ guardrail/router) to the quick alias (they're one-turn, tool-less calls that need instruction-following + JSON discipline, not deep coding). Route plan to heavy (the opusplan pattern — plan-mode turns swap to a strong reasoning model).
4. Router categories — 3 categories minimum (large/medium/small mapped to heavy/coder/quick). Add a 4th ONLY for a genuinely distinct model class (e.g. image for multimodal) — do NOT add a 5th; more categories degrade classifier accuracy and widen the steering surface. Write thorough description: fields — the classifier reads them literally to route.
5. default-category — usually medium (the bulk of delegation work).

Step 4 — Deliver the config

Emit the complete models: YAML block, ready to drop into ~/.config/mecatl/settings.yaml. Include inline # → <concrete-id> comments on each category's model: line so the operator can see the resolution at a glance.

After the config, include:

• A changes-from-previous summary (if revising) — one line per changed alias/ slot/category, with the rationale.
• A verification note — tell the operator to check the mecated log for model slot ACTIVE / subagent model router ACTIVE lines on startup (a missing line = that binding failed to resolve and degraded to the session model).
• An honest gaps note — call out anything the chosen fleet can't do (e.g. "nothing here reaches Opus 4.8's 88.6% SWE-bench Verified; that ceiling requires Anthropic"). Don't oversell.