Tool Defs Analysis

Context

Every string in a tool/resource/prompt definition is part of an LLM-facing API contract. The model reads the description, every parameter .describe(), the output schema, the recovery hints — and decides what to call and how. Definition language drifts: an internal mapping leaks into a parameter doc during a fix, a self-referential output description survives a refactor, a default that suited the developer at scaffold time stays after the typical call shape changes.

This skill is the review-time pass for that drift. Read each definition the way a mid-tier model with no project context would — can it pick the tool, fill the fields, and recover from errors using only the rendered schema?

field-test already audits descriptions for implementation leaks, meta-coaching, and consumer-aware phrasing during its catalog step — that's a fast shallow pass alongside live tool calls. This skill is the deeper review: 16 categories, every field, every recovery hint, every default value, with file:line citations — plus a cross-surface pass for the drift no single file shows.

Read-only. This skill produces a report; the maintainer applies fixes. While running it, do not run git, do not stage or commit, do not update the changelog, do not run devcheck, do not invoke wrapup or release workflows. Fixes flow through the normal authoring path (edit the definition, then re-run this skill if you want to verify).

When to Use

• After a polish session or refactor that touched definitions
• Before a release, alongside polish-docs-meta and security-pass
• When the user says "review my tool definitions", "audit descriptions", "are my tool descriptions any good"
• After scaffolding a new server but before it ships

Skip during initial authoring — add-tool and design-mcp-server cover that. Skip diff-only review — read each file in full so drift across the whole definition surfaces.

Inputs

Gather before starting. Ask if unclear:

1. Scope — whole server, specific definitions, or a single directory?
2. Severity floor — all findings (default), or skip nits?
3. Known concerns — anything the user already wants emphasized?

Steps

1. Build the inventory

find src/mcp-server/tools/definitions     -type f -name "*tool.ts"     2>/dev/null | sort
find src/mcp-server/resources/definitions -type f -name "*resource.ts" 2>/dev/null | sort
find src/mcp-server/prompts/definitions   -type f -name "*.prompt.ts"  2>/dev/null | sort

The *tool.ts / *resource.ts patterns also catch *.app-tool.ts / *.app-resource.ts. If the server's definitions live elsewhere (examples/, a packages workspace, …), audit those paths too. Also locate the server-level instructions string if the server sets one (the createApp option — grep -rn "instructions" src/ --include="*.ts"); it's audited in the cross-surface pass.

Use TaskCreate — one task per file. Mark each complete after its findings are captured.

2. Walk the 16 categories per file

Read each definition file in full. Apply every category — most files trip more than one. Capture each hit with file:line, the offending excerpt, and a one-line fix.

1. Voice & tense

Look in: tool / resource / prompt description.

Check: imperative present-tense. "Search for trials" beats "Searches for trials" or "This tool will search trials".

Smell: "Allows you to…", "This tool…", "Provides functionality to…", "Searches for…", "Fetches…", "Will return…".

(Parameter .describe() text describes the value, not the tool — it doesn't need imperative voice.)

2. Internal leaks

Look in: every description and .describe().

Check: internal API routes, endpoint paths, API call counts, internal parameter mappings, sibling service names, version notes, TODOs.

Smell: "/api/v2/by-state", "Adds a second API call", "API requires two_year_period", "(deprecated; use bar_v2)", "TODO: support batch mode", "Used internally by FooService".

3. Audience leaks

Look in: every description and .describe().

Check: reader-naming or meta-coaching directed at the LLM rather than describing the tool.

Smell: "suitable for LLM consumption", "Treat the returned ID as the canonical Y", "Agents should…", "Callers should…", "When you call this tool…", any reference to "LLM", "agent", "Claude", "the model".

Field-test catches this in its leak audit; this skill is the more thorough pass.

4. Defaults

Look in: every .default(...) call in input schemas.

Check: the default matches the typical caller's case. A default that suited the developer at scaffold time often skews real calls — limit: 1 makes default-args searches useless, verbose: true floods context, dryRun: false on a destructive op invites an irreversible accident.

Smell: dev-convenience values that survived the schema's first draft, dangerous defaults on destructive operations, defaults that contradict the description's framing of typical use.

5. Recovery hints

Look in: errors: [{ recovery: '…' }] arrays, data.recovery.hint at throw sites in handler bodies.

Check: the hint directs the agent to its next action, not the developer to debugging. "Call pubmed_search with a narrower query" beats "Verify the configuration is correct" or "Internal error".

Smell: "Check the logs", "See documentation", "Contact admin", "Try again later" (with no condition), generic non-actionable text, hints that name internal classes or files.

6. Field descriptions

Look in: every field in input and output schemas; resource URI template variables.

Check: every field carries a .describe(), and it tells the agent what the value is — not just the field name restated, not silent on dynamic shapes. Enum variants — especially operation discriminators — are explained.

Smell:

• An input field with no .describe() at all
• name: z.string().describe('Name') — tautology
• operation: z.enum([...]) whose variants are never explained
• metadata: z.record(z.string(), z.unknown()).describe('Metadata') — opaque dynamic shape with no hint about keys/values
• Optional fields with no note on when they're absent
• Paging fields (total, hasMore, nextCursor) with semantics unstated — or a limit param that doesn't say whether it caps the page or the whole result
• A URI template variable ({cid}) never described anywhere

7. Cross-references

Look in: tool descriptions, prompt content, recovery hints.

Check: when one tool/resource is mentioned, when to reach for it is explained — and the references cover the relevant siblings, not a partial sample.

Smell: "Use foo_search to find IDs" (no when); a prompt naming 3 of 7 landscape-relevant tools; a tool description listing one sibling but not the others that fit the same workflow.

8. Sparsity

Look in: output schemas (especially fields wrapping external API data), format() rendering.

Check: optional upstream fields are acknowledged as such — not implied to always be present. format() doesn't print fabricated values for missing fields.

Smell:

• pmid: z.string().describe('PubMed ID') when only ~60% of records have one (should be .optional() and noted)
• format() printing **PMID:** undefined
• A required field in output for an upstream value the API doesn't always return

9. Examples

Look in: parameter .describe() text containing "e.g.,", "(e.g. ...)", .example(...) calls.

Check: examples are domain-realistic — real-shaped IDs, real query strings, real values from the upstream domain. One example is usually enough.

Smell: .describe('Item ID (e.g., "abc123")') when real IDs have structure (NCT12345678); toy values ("foo", "bar"); padding multiple toy examples instead of one realistic one.

10. Structure

Look in: tool / resource / prompt description.

Check: single cohesive paragraph, written as one string literal in source. No bullet lists, no blank-line-separated sections, no markdown headers inside the description; no +-joined fragments in the file — a description assembled one sentence per line reads as a list of disconnected claims and grows a line at a time until it is several times its siblings' length.

Smell: blank lines (\n\n) inside a description string, - bullet lines, ## Header lines, "Operations:\n- foo: …" duplicating an enum's .describe() text, '…' + continuation lines under description:.

11. Mutator observability

Look in: mutator tools — any tool that writes, updates, deletes, appends, or patches (i.e., definitions without annotations.readOnlyHint: true).

Check: output carries a state-change discriminator (created, updated, mutated, unchanged) or before/after observable state the agent can use to confirm intent-effect match. The server reports what it observed; the agent decides whether it matches what it meant.

Smell: mutator output is { path, ok } or { success: true } — no pre/post state, no discriminator. Server-side defensive throws on synthetic deltas (file shrunk, count decreased) the server can't authoritatively classify as bugs.

12. Unit-bearing numeric names

Look in: every z.number() field in output schemas.

Check: the field name carries a unit when not pinned by context — sizeInBytes, durationInMs, priceInCents, latencyInMs. The .describe() drops in summarization or gets truncated; the field name persists into the JSON the agent reads. Scores, ratios, and percentages carry their range the same way — in the name or as the first thing in the describe (0–1).

Smell: size, duration, price, latency — bare names that force the agent to guess units; score/confidence with no stated range (0.87 and 87 both pass the schema). Exempt: index, position, page, offset, limit, totalCount, itemCount (dimensionless).