Agent Prompt Audit

Agent Prompt Audit

An Agent's behavior comes from every piece of text that reaches it over its lifecycle, not just its system prompt. The audit's job is to find which text causes the behavior the user sees, on the backend and model that actually ran it, and to propose the smallest change that fixes it. Judge each instruction by what it does to behavior, not by its length: sometimes the fix is adding a missing reason or exit, and a clean surface is a valid result.

Deliver a report of findings — each with its evidence, confidence, and a concrete proposed change — and apply changes only when asked.

Core ideas

Evidence over reading. What Agents actually did beats what the text seems to say. Start from real runs and user corrections ("you stopped", "why didn't you report", "don't ask me that"), trace each symptom to the line that caused it or the missing line that would have prevented it, and use git blame to learn what incident a rule was written for and whether it still happens. A finding without evidence or documented model behavior is a flag, not a fix.

Context is kept; constraints must earn their place. Facts only the author knows — environment, contracts, ownership, quality bar, the reason behind a rule — are what prompts are for. Behavioral constraints are what go stale. Keep exact scripts where one sequence is safe (destructive commands, auth, merge gates, key custody), prohibitions against failures that still reproduce, and the scope bounds that make autonomy safe.

Say intent and reason, not pressure or method. Caps, MUST/NEVER, and emphasis without a reason make current models rigid; step scripts for judgment work and strategy coaching usually do worse than the model's own plan; fixed formats, word caps, and "don't narrate" rules produce silence or starved answers. Fossils — named-model workarounds, incident numbers as authority, "now/no longer" phrasing, one session's stumble made permanent — should become the current rule they stand for.

Every stop needs an exit. Agents run across turns, wake on callbacks, and hand work to each other, so the costliest defects are lifecycle gaps: a "stop/wait" with no statement of what the turn produces instead, asking permission for reversible in-scope steps, continuing without bounds after repeated failure, waiting with no durable waiter or expiry meaning, briefs missing the goal or report target, callbacks that say "done" without the result, and Task/Watch messages that restate rules on every fire.

One home per rule, at the layer whose timing fits. Always-loaded and recurring text has the most leverage and deserves the most scrutiny. Duplicates that disagree force the Agent to guess; keep the mechanism in one place and a principle or pointer elsewhere. Agreeing fallbacks are fine. Long procedures belong in on-demand Skills, not always-loaded rules.

Shared text runs on every backend. GPT/Codex tend to follow a bare prohibition or stop literally, so they need scope and exit conditions; strong Claude models tend to over-reach, so they need scope bounds and a definition of done; tool names and native mechanics dangle on other backends. Take model-specific behavior from the vendor's current docs, and lower confidence when you cannot reach them.

A removal is a hypothesis. For contested changes, compare behavior before and after with a scratch run on the target that produced the failure, and read the transcript rather than asking the model whether it needs the rule.

Where the surface lives

Verify against the current machine; these are starting points. Each backend also reads its own native configuration — config directories moved by environment variables (CLAUDE_CONFIG_DIR, CODEX_HOME, OpenCode's config path), and native subagent definitions such as .claude/agents/, .codex/agents/, or OpenCode agents — so resolve what the target backend actually loads rather than assuming default paths.

Only Skill descriptions on the first catalog page are loaded every turn; later pages, disable-model-invocation Skills, Skill bodies, and references load on demand. Project Skills in that catalog resolve from the target Session's working directory, not yours, so read them from there.