Overmind

Overmind MCP

Overmind models production work as Capability > behaviour > task execution. A capability is the product AI surface, a behaviour is a scanned contract, and a task execution is a carved, scored unit of a trace.

MCP prompts are the native guided workflows. Invoke the matching prompt when the client lets the agent invoke prompts. Otherwise, or when local repository work is required, follow the matching reference in this directory. In Claude Code only the user can run a prompt, as /mcp__overmind__<prompt-name>.

Arguments

When invoked with an argument, such as /overmind setup, open the matching reference and follow it:

• onboard — references/onboard.md
• setup — references/setup.md
• ensure-tracing — references/telemetry.md
• dataset — references/datasets.md
• finetune — references/finetuning.md
• optimise — references/optimizer.md
• backtest — references/backtest.md

Native prompts

Route guided work to these exact prompt names:

• investigate-capability — health, failures, traces, task executions, and instrumentation gaps.
• instrument-repository — translate an instrumentation plan into a human-applied code change and verify supplied spans.
• upload-dataset-file — upload local data through the CLI, then land the dataset through MCP or REST.
• export-dataset — download a dataset version through the local CLI; MCP carries guidance, not file bytes.
• download-checkpoint — download an archived fine-tuned deployment checkpoint through the local CLI; MCP carries guidance, not checkpoint bytes.
• connect-traces — connect a tracing provider, review capability boundaries and verify imported traces.
• prepare-evaluation — check evaluation dataset, evaluators, eval set, bindings, and credits.
• evaluate-change — run an evaluation and compare it with a supplied baseline.
• finetune-capability — check, estimate, launch, and verify fine-tuning.
• optimize-capability — schedule and inspect prompt/code optimization.
• compare-models — schedule and inspect model comparison.
• ship-model — verify deployment, activate a model, and hand off repository rollout.

Initial Console onboarding and local capability discovery remain local workflows: use references/onboard.md for a new project and references/setup.md for repository scanning and sync. Both use references/onboarding-progress.md for the opening roadmap, numbered progress updates, and data disclosures.

Do not reimplement these workflows as a single generic call. The prompt supplies the workflow; the skill supplies only missing local actions, human approval boundaries, and fallback sequencing.

The plugin is an optional distribution package for this skill and the existing MCP connection. Essential guidance is supplied by MCP initialization, tool descriptions and resources. Installing the plugin adds no separate product UI or additional platform permissions.

Product surface skills

The plugin and overmind init include these focused workflows. Select the one that matches the user's task; do not load all of them for a single operation. Each works directly with the configured MCP connection and can be used on its own.

Connection and safety

• overmind sync stores the project-scoped API key locally and configures each initialized MCP client with X-Api-Key. Plugin connections use OAuth; account API keys also work. Never ask the user to paste a key into chat.
• The public server supports read and write permissions for the curated surface and returns structured errors as values. Start with list_projects. Account connections require project_id for every project operation and resource URI query. Follow returned resource links; selection is per request. Project keys remain limited to their configured project.
• There are no public product tools for deletion, cancellation, or removal. Deployment recovery is limited to retry_deployment for failed or deleted deployments; do not invent other lifecycle tools.
• Treat dataset landing, evaluator writes, job starts, deployment changes, and active-model changes as mutations. Confirm user intent where the workflow requires approval; the server does not add a confirmation dialog.

Core principles

Follow these for ALL Overmind work:

1. Local setup, then MCP. Capability discovery is local: scan the repo, write overmind.toml, run overmind sync — see references/setup.md. After that, all platform work goes through the Overmind MCP server. Do not curl REST endpoints, do not invent base URLs, and do not hardcode hosts. The server is already configured (plugin, or overmind init) and authenticated through OAuth or an account/project API key. Call the named tools; inspect each tool's schema for arguments. If tools are missing, tell the user to run overmind init for the IDE and overmind sync to install its project credential. Do not paste a URL or ask them to paste the raw key into chat.
2. Reference file per use case. Check the relevant reference below before implementing. This file holds conventions that apply everywhere; the workflow lives in the reference.
3. Discover, then pass the id the schema asks for. There are no list_capabilities, list_traces, get_trace, get_capability, list_eval_sets, list_evaluators, list_eval_runs, list_finetune_jobs, list_deployed_models, or job_status tools. Use list_datasets, query_*, inspect_*, get_job, and resource reads. Never paste raw UUIDs to the user when a name/slug exists. Dataset names are not unique: list_datasets then pass that UUID to inspect_dataset / query_dataset (those two reject names). Fine-tune, eval, and optimizer tools also accept a unique dataset name. Capability tools accept name, slug, or id. Stamp the capability resource id into the SDK. Pass a READY deployed-model UUID to set_active_model (omit to clear), then poll the returned model_activation job until verification and routing complete. See references/capabilities.md.
4. Behaviours have no resource. There is no overmind://behaviours/...; read them from query_task_executions.
5. Contracts gate every dataset workflow. Intent is train, eval, or pending — never ft or surface. Fine-tuning needs train; eval runs and optimizer experiments need eval. pending is refused. There is no reingest tool and no dual-intent dataset. Set intent at upload (overmind dataset upload FILE --json --intent train|eval), at create (create_dataset_from_traces / _failures), or later with message_dataset_agent ("set intent to train") if no version has been used. create_dataset_from_traces with split lands one selection as a train dataset and an eval dataset with disjoint rows; so does --split PERCENT on overmind dataset upload. A used cell freezes intent: upload a second dataset with the other --intent instead of retagging. Read the contracts section below.
6. Errors are values; mutations run immediately. Every tool returns {"error": "..."} instead of raising — follow fields when present. There is no confirmation gate, so verify arguments (and ask the user when destructive) before create/delete/cancel. There are no delete or cancel tools.
7. Ticketed instrumentation. Call get_instrumentation_plan with no capability for project-wide work, or with a capability for scoped work, and treat each placement as an edit ticket. If the result has human_action or no placements, report its instruction and stop this attempt. Copy every ticket field verbatim, including key, behaviour_id, version_id, version_analyzed_sha, contract_fingerprint, capability, capability_id, placement_mode, allowed_keys, grain, target, required_scope, required_spans, and required_identity. File + qualname is enough to locate the function. Keep a primary scope outermost when a specialized span targets the same function. When coding subagents are available and permitted, group tickets by every file they touch, including required_spans[].target.file, so one worker owns each overlapping group.
8. Explicit run approval. After applying the ticketed code changes, report the changed files and local checks, generate a unique verification correlation, then ask the user to choose a real run or bounded smoke run. Present each choice's exact command or input, capability, environment, provider/model, expected side effects, correlation value, and approved attempt count; mark unknown fields as needing user input. Do not execute either mode before explicit approval. A real-run retry needs fresh approval unless an exact input and bounded attempt count were approved.
9. Server-side verification. Stamp the approved correlation as conversation.id, run only the approved input, and flush. Poll query_traces(session=<correlation>, all_spans=false, limit=2) within a fixed bound and require page.total == 1. Pass that row's trace_id to verify_instrumentation(trace_id=...); the server grades the ingested spans (no DB writes). Report application outcome separately from instrumentation status.

Use-case references

• Local setup (overmind chassis → scan repo → capability cards / trajectory maps / eval matrix → overmind.toml → overmind sync): references/setup.md
• Resolving / updating agents, prompts, and eval spec: references/capabilities.md
• Tasks (behaviour registry, task executions, eval coverage): references/behaviours.md
• Telemetry (add tracing, inspect traces / sessions / health, connectors): references/telemetry.md
• Landing datasets (from traces, failures, a file or rows), handing a version to a consumer, and pulling a version to disk: references/datasets.md
• Authoring evaluators, grouping them into eval sets, running and comparing eval runs: references/evals.md
• Fine-tuning a model (prerequisites, recommended-model sweep, deploy, swap PR): references/finetuning.md
• Optimizer experiments (/overmind optimise — skill writes diffs/commands; SDK runs locally; server scores): references/optimizer.md
• Model backtest (skill rewrites provider + model onto OpenRouter via overmind.backtest.rewrite_repo; MCP posts outputs; server scores): references/backtest.md

Conventions (read before any workflow)

• List first. list_datasets for datasets. Everything else: inspect_capability_health, query_traces, query_task_executions, query_failures, or a resource read. Pass the UUID list_datasets returned into dataset inspect/query. Capability name/slug/id, eval-set name, and unique dataset names work on the tools whose schemas accept them. Read overmind://capabilities/{capability} for capability id and active_model.
• Async jobs. Poll returned job references with get_job(kind, id). Dataset work uses kind=dataset_run; other supported kinds include eval_run, finetune_job, deployment, model_activation, and optimizer_experiment.
• Chat-UI-only helpers (propose_plan, suggest_navigation) are not exposed on MCP.

Curated MCP tools

Use only these implemented names and inspect their schemas at call time.

Observability:

inspect_capability_health, query_failures, query_traces, query_task_executions, get_job.

Datasets:

list_datasets, inspect_dataset, query_dataset, create_dataset_from_traces, create_dataset_from_llm_calls, message_dataset_agent, run_dataset.

Evaluations:

check_evaluation_readiness, upsert_evaluator, create_eval_set, run_evaluation, compare_evaluations, annotate_evaluation_sample.

Fine-tuning and serving:

get_model_catalog, check_finetune_readiness, estimate_finetune, start_finetune, retry_deployment, set_active_model, set_benchmark_model, run_inference, get_model_swap_prompt.

Call get_model_catalog before choosing a fine-tuning model. It is dataset-independent and reports the active backend, tier, context limits, batch bounds, training methods, tool-calling support, and disabled rows.

Optimization:

check_optimizer_readiness, start_optimizer, inspect_optimizer_result.

Connectors:

inspect_connectors, configure_connector, sync_connector.

Instrumentation:

get_instrumentation_plan, verify_instrumentation.

The server does not expose delete, cancel, or generic API tools. Use retry_deployment only for its documented failed/deleted deployment recovery case. Use the returned structured fields and resource links rather than guessing older endpoint-shaped names.

Dataset contracts — read first, they gate every workflow

A dataset is a landed source and a linear chain of cells; every cell that ran is a version (1.0 is the source, then 1.1, 1.2, …), the dataset has an intent (train or eval, proposed at landing) and a capability, and every version carries two measured contracts (list_datasets shows the active version's):

• train ("Train") — a messages column whose every row is a chat transcript with an assistant turn (tools optional).
• eval ("Eval") — an input on every row plus an expected_output column with at least one reference.
• pending — the intent is not decided yet; refused by every run.
• There is no ft intent. A leftover stored ft is train.

Capability prompt/schema mismatches, incomplete quality reviews and train/eval overlap are advisory warnings, not technical-format errors. Explain the remaining work and offer the workshop for repairs; users can continue without a quality approval step. Inspect cell warnings and readiness.quality_reason. A passing format contract is not a claim that the answers are supported by the inputs.

What each workflow accepts:

• Eval runs (run_evaluation) and optimizer experiments (start_optimizer) use the active version of an eval dataset.
• Fine-tuning (start_finetune) uses a train version, plus a separate eval dataset for in-training judge evals.

A use freezes the version and everything before it, and starts a new major (2.0). A contract is measured, never declared. The dataset's own agent shapes the chain; if a consumer rejects a dataset for its contract, inspect_dataset names the reason. Use message_dataset_agent to request changes, then poll get_job(kind=dataset_run) and inspect again. Rows are never cleaned locally: land them raw, shape them on the server. Local loops pull one version by cell id (datasets.md).