Copy Audit

copy-audit

Running as a subagent

This skill runs with context: fork, in a fresh subagent. Nothing from the conversation reaches it, so everything it needs is stated here or in its arguments. It returns one thing: the triaged report of step 4. It does not fix anything: step 5 is the maintainer's choice, made in the main conversation from that report.

• Range: the argument, if one names a revision or a..b. Otherwise origin/master..HEAD, the unpushed work. Step 2's --since is the range's base.
• Evidence dir: localresearch/copy_audit/<YYYY-MM-DD>/, which is gitignored. If the argument names a directory that already holds the step-1 files, that directory is the evidence dir: reuse its files and regenerate only what is missing.
• Use absolute paths in every shell command: a cd that fails leaves the shell elsewhere for every later call.
• Run reviewers as subagents at model: opus, because this is truth-checking, not comprehension. Give each one slice and one evidence source, as §3 says.
• Return the step-4 report verbatim as your final message: ranked, every finding with severity, the true statement and its evidence, the discarded findings with why, the known limits, and the patterns that went unrendered.

/user-review grades comprehension, and a false sentence can score perfectly there. That loop's own fixes were the biggest single source of untrue statements, because simplifying a hedged sentence is how a hypothesis becomes an assertion.

This audit walks a git range and checks each claim against the evidence its type demands. The claim-type checklist it applies, with examples of a dropped qualifier, is .claude/rules/claims.md ("What each claim rests on").

Non-expert phrasing is the design goal and is never a finding. A reviewer reporting that copy is informal, imprecise or jargon-free has misunderstood the task. Only FALSE, UNSUPPORTED, or TRUE-ONLY-FOR-SOME-DEVICES counts.

Copy this checklist and track progress:

Copy audit:
- [ ] 1. Prepare the evidence (corpus sweep, renders, system probe)
- [ ] 2. Build the claim inventory
- [ ] 3. Fan out reviewers, one slice and one evidence source each
- [ ] 4. Re-check every finding yourself; discard what doesn't survive
- [ ] 5. Fix, grouped by topic, then verify

1. Prepare the evidence, once

Every reviewer reads files, and none re-runs a tool. That is the whole cost control: the audit is affordable because the expensive work happens once.

python3 tools/corpus_audit.py > <out>/corpus_audit.txt
python3 tools/preview_output.py --full --examples 3 --width 80 > <out>/renders/findings_all.txt
python3 tools/render_forced_conditions.py --out-dir <out>/renders

--examples 3 is load-bearing: the same message rendered for three different devices is what exposes a sentence true only for the one it was written from.

render_forced_conditions.py covers what preview_output.py structurally cannot: messages whose trigger value never occurs in the corpus, so no real device can show them. Those are exactly the messages nobody has ever read.

Then render the conditions that need flags rather than XML values: a SOUNDWIRE_* file against an HDA one, a simplified-schema file, --all-profiles, -v, each --disable name, and a dolby_to_pipewire.py --no-activate --dry-run. Probe the live system too, because several claims are about other people's software: easyeffects --version, pw-cli ls Node | grep alsa_output, pw-link -l, command -v lv2info.

2. Build the claim inventory

python3 tools/extract_claims.py --since <rev> --out-dir <out>

It writes claims.md and the per-reviewer slices. Only CHANGED rows are targets. Unchanged strings stay in the file so a claim can be read against the run it prints in.

It reports two totals, so compare the distinct count, which counts sentences rather than sites. A refactor that collapses a string written at two sites legitimately shrinks the row count, while the run prints the same words.

3. Fan out, partitioned by evidence source

Give each reviewer one slice and one evidence source. Partitioning by evidence rather than by file keeps each context small. The reviewer checking corpus figures never loads the wrapper's source, and the one checking the wrapper never loads the docs.

Each returns a fixed schema, worst first, and no restated copy:

ID | SEVERITY | <=10-word quote | what is actually true | evidence file:line | confidence

CRITICAL false and it changes what the user does · HIGH false but low consequence, or an unvalidated hypothesis stated as fact · MEDIUM true only for some triggering devices · LOW stale figure, no user consequence.

Tell every reviewer: a finding must name the true statement. "This seems wrong" without a replacement is not a finding, and will be discarded.

Also tell them the settled decisions from /user-review, so they don't relitigate copy that survived eleven rounds. Those reopen only on proof that one is false, not awkward.