Apr Dogfood

aprender Dogfood — Coverage-First Pre-Release Protocol

Version: 3.0 (supersedes the 19-gate v2.0 skill — Gates 0–18, 924 lines at origin/main; every gate body below is spliced from it verbatim) Preserves: every falsifier ID from v2.0 (FALSIFY-QA-*, F-SILENT-*, F-META-*, F-COV-*, F-CHAOS-*, F-DIFF-*, F-WORKTREE-HEAD-001, F-EXPORT-ROUNDTRIP-001, F-VALIDATE-QUALITY-001, F-RUN-EXIT-SANITY-001, F-7B-INFERENCE-001, F-APR-INFERENCE-PARITY-001) Defers to (v3.1, aprender#2640 — this line used to read "Absorbs: dogfood.sh release-gate protocol", and "absorbs" is the wrong verb: it invited a second copy and the runner promptly grew one): scripts/dogfood.sh is the ONE fleet release-gate protocol and .claude/skills/dogfood/SKILL.md is its ONE prose. This skill layers aprender's surface-coverage ledger ON TOP of it and must not restate a gate the runner owns. The invariance.sh transport gate likewise belongs to the runner (scripts/invariance.sh), not to this file.

Three documents describe overlapping release work; each owns exactly one scope:

Contracts:

• contracts/apr-cli-qa-v1.yaml — baseline (10 equations, 10 falsification tests)
• contracts/apr-qa-silent-fallback-v1.yaml — bad-input injection (5)
• contracts/apr-qa-metamorphic-v1.yaml — quant equivalence, multi-arch, roundtrip (5)
• contracts/apr-qa-coverage-v1.yaml — category coverage, SATD, complexity (5)
• contracts/apr-qa-chaos-v1.yaml — memory, OOM, signals, overwrite (5)
• contracts/apr-qa-differential-v1.yaml — ollama parity, tokenizer, concurrency (5)
• contracts/apr-dogfood-coverage-v1.yaml — NEW, Phase 2; must be authored

Spec: docs/specifications/apr-cli-qa-spec.md Ledgers: docs/audits/dogfood-<version>-ledger.md, docs/audits/surface_audit.csv

Why this rewrite exists

The v2.0 skill has 19 gates and they work — where they look. The surface audit (832 features across 28 binaries) measured where they look:

And coverage by quality band:

Read the 5–6 band correctly. 672 features scored 6, and 644 of those are uncovered — scored 6 because no ledger finding exists and no gate covers them. That is not "fine" — it is unlooked-at. The 1–2 band's 72.5% coverage is the control: gates find defects at a high rate when pointed at something. 644 unmeasured features is the prediction of where the next 201-finding ledger comes from.

The v2.0 skill cannot see this, because it has no denominator. Every gate answers "did this check pass?" and none answers "what fraction of the shipped surface did we check at all?" A gate suite with no denominator reports a clean sweep over whatever subset it happens to cover — the vacuity failure dogfood_surfaces.sh already guards against per-enumeration, applied one level up.

So the organizing gate of v3.0 is coverage itself.

Doctrine (non-negotiable; do not relitigate)

1. Gates or theater. A check that cannot fail CI is documentation. Every gate in this file carries a named mutation that must turn it RED (§ Mutation registry). A gate with no registered mutation is inadmissible.
2. Never enumerate from a written list. Binaries from cargo metadata, commands from --help on the built binary, routes from the router, tools from a live tools/list. The command count has been claimed as 36, 77, 103, and 111 — every one from a stale hardcoded list, every one failing silently because a shrunken universe reports "all passed."
3. Vacuity-guard every enumeration. An implausibly small surface FAILS. It does not sweep clean.
4. Exit 0 is not a pass. Every probe must EXCLUDE an outcome. The 0.63.0 audit found tests asserting is_ok() on invalid input, which locks the defect in.
5. Silent-pass is the top severity class regardless of surface. Fabricating a green result, ignoring a flag, exiting 0 on failure. The ledger clusters that name it: lint-passes-on-bad-obs, json-flag-emits-text, offline-fails-open, nan-threshold-disarms.
6. Read-only. This skill audits. It never modifies files. Defects become GitHub issues and pmat work add tickets.
7. Deterministic receipts. Sorted enumerations, fixed seeds, fixed prompts, no wall-clock assertions, no timestamps in the receipt body. The same tree yields a byte-identical receipt. Verify with --twice.
8. Stop the line. Any red gate stops the release. Fix the root cause via five-whys to the owning module. Never bypass, never --skip.

Context

• Workspace version: !cargo metadata --no-deps --format-version 1 | jq -r '.packages[]|select(.name=="apr-cli")|.version'
• HEAD: !git rev-parse --short HEAD
• Worktree clean: !git status --porcelain | wc -l modified paths
• apr pinned to HEAD: !. scripts/apr_bin.sh >/dev/null 2>&1 && "$APR" --version || echo "NOT built from HEAD — every verdict below would describe a binary you are not running"
• Models available: !find ~/models -maxdepth 2 \( -name "*.apr" -o -name "*.gguf" -o -name "*.safetensors" \) -type f 2>/dev/null | wc -l
• Surface ledger: !test -f docs/audits/surface_audit.csv && wc -l < docs/audits/surface_audit.csv || echo "ABSENT — Phase 2 will FAIL"
• Clusters: !test -f docs/audits/surface_audit.csv && gawk 'BEGIN{FPAT="([^,]*)|(\"([^\"]|\"\")*\")"}{b=b $0; if(gsub(/"/,"\"",b)%2){b=b "\n"; next}; $0=b; b=""; if(!seen++){next}; n++; l=$8; c[l]=1; y=tolower($6); gsub(/^[ \t]+|[ \t]+$/,"",y); if(y=="yes"){f++; g[l]=1}}END{k=0; for(l in g)k++; m=0; for(l in c)m++; printf "%d/%d clusters gated, %d/%d features gated\n",k,m,f,n}' docs/audits/surface_audit.csv || echo "ABSENT"

Arguments

$ARGUMENTS

Phase 0 — Identity guards

Everything downstream is void if the thing under test is not the thing you think it is. These run first, serially, and a failure here aborts rather than degrading to a warning.

G0.1 — Binary provenance (FALSIFY-QA-005)

cargo install --path crates/apr-cli --force 2>&1 | tail -5
# Resolve through the guard, NEVER a bare `apr`. Sourcing exports $APR and
# hard-fails when it was not built from HEAD.
. scripts/apr_bin.sh || exit 1
APR_BIN_STRICT=1 "$APR" --version
git rev-parse --short HEAD

PASS iff the version string contains the HEAD hash.

This gate exists because of ledger #2384, a P0 closed COMPLETED 2026-08-11 (fixed by #2424; the ledger row still shows no Fixed by, so ledger and GitHub disagree — treat the gate as a regression ratchet, not an open-defect probe): the MCP server and apr code both execute a bare apr resolved from PATH. Every tool ran a different 0.60.0 binary while apr.version reported 0.63.0. Never resolve a bare apr anywhere in this protocol.

G0.2 — Crate identity from cargo metadata, never from sed

Resolve name and version via cargo metadata --no-deps, filtered by manifest_path. Do not grep Cargo.toml.

Measured 2026-08-20 on pforge-cli: sed yielded an empty version for a crate using version.workspace = true, and an empty version made version-unpublished PASS, because the crates.io lookup for "" finds nothing. A false pass in the gate whose entire job is to stop an immutable double-publish.

Never proceed with an empty version. Abort with the candidate member list.

G0.3 — Worktree HEAD sanity (F-WORKTREE-HEAD-001)

Contract: contracts/apr-version-traceability-v1.yaml § FALSIFY-VERSION-004

Body carried verbatim from v2.0 Gate 13 (hardened by #2357 — do not re-derive).

Contract: contracts/apr-version-traceability-v1.yaml § FALSIFY-VERSION-004

Catches #1862 — apr --version reporting a stale commit hash in git worktrees because build.rs watches a hardcoded ../../.git/HEAD path that doesn't exist in a worktree layout.

# After cargo install, the SHA MUST match git rev-parse --short HEAD.
# Run this from inside the source checkout (or worktree) you just built from.
. scripts/apr_bin.sh || exit 1
APR_SHA=$("$APR" --version 2>&1 | grep -oE '\([a-f0-9]{7,}\)' | tr -d '()')
HEAD_SHA=$(git rev-parse --short HEAD)
if [ -n "$APR_SHA" ] && [ "$APR_SHA" = "$HEAD_SHA" ]; then
  echo "G13 PASS: $APR SHA ($APR_SHA) matches HEAD"
elif [ -z "$APR_SHA" ]; then
  # NOT a SKIP. A binary with no embedded SHA cannot be shown to be the code
  # under test, and "cannot prove" must never read as "fine". This branch used
  # to SKIP for "likely crates.io install" — and the binary that wins PATH
  # resolution on this machine is `~/.local/bin/apr`, which reports exactly
  # `apr 0.60.0 (v0.60.0+no-git)`. So the one stale artifact most likely to be
  # executed was also the one that made the protocol's ONLY freshness gate
  # excuse itself. If you are deliberately dogfooding a published crates.io
  # build, say so explicitly with DOGFOOD_ALLOW_UNPINNED=1.
  if [ "${DOGFOOD_ALLOW_UNPINNED:-0}" = "1" ]; then
    echo "G13 SKIP (explicitly allowed): $APR has no embedded SHA"
  else
    echo "G13 FAIL: $APR has NO embedded SHA — cannot prove it is HEAD. Set DOGFOOD_ALLOW_UNPINNED=1 only for a deliberate crates.io dogfood."
  fi
else
  echo "G13 FAIL: $APR SHA=$APR_SHA but HEAD=$HEAD_SHA (#1862)"
fi

Build.rs static check (no install required):

# build.rs MUST use git rev-parse --git-dir / --git-common-dir for worktree-safe
# rerun-if-changed triggers — not a hardcoded ../../.git/HEAD path.
if grep -qE 'rev-parse.*--git-(dir|common-dir)' crates/apr-cli/build.rs \
   && ! grep -qE '\.\./\.\./\.git/HEAD' crates/apr-cli/build.rs; then
  echo "G13 PASS (static): build.rs uses worktree-safe git resolution"
else
  echo "G13 FAIL (static): build.rs still uses hardcoded .git/HEAD path"
fi

PASS if both checks pass (or SHA check SKIPs cleanly on crates.io builds).

G0.4 — Feature-scope resolution

Determine the feature set from [features] only. Two traps, both real:

• cli = { e2e = "…" } under [package.metadata.transports] is not a cargo feature. Matching it makes every gate run --features cli against a crate that has no such feature.
• --all-features is not a safe fallback. pmat declares broken-tests (49 quarantined sites) and red-phase-tests ("expected failures … NEVER add to any feature bundle"). Enabling them measures the quarantine, yielding a permanent RED that says nothing about release readiness — and a gate that is always red is one everybody learns to walk past.

Fallback is every declared feature minus known-broken quarantines, and the exclusion is reported. A silent exclusion is how a gate quietly stops covering what it claims to.

G0.5 — Duplicate-binary divergence check (new)

Two apr bin targets exist and are the same program by hand-duplication, not by sharing: src/bin/apr.rs delegates to apr_cli::cli_main(), while crates/apr-cli/src/main.rs re-implements the same prologue (SIGPIPE reset, aarch64 FPCR.FZ16 clear, NO_COLOR, the --version --json intercept). They have already diverged — the dhat-heap #[global_allocator] exists only in the latter.

Assert both entry points produce byte-identical output for --version, --version --json, and --help. Any divergence is RED.

Phase 1 — Surface derivation

Delegate to scripts/dogfood_surfaces.sh. Do not reimplement it here; do not substitute a list.

Enumerate, sorted LC_ALL=C:

Grepping the source is not an acceptable substitute. A regex over clap Subcommand enums reports 0 subcommands for simular, a clap-derive CLI. The binary is the only thing that knows what the binary accepts.

G1.1 — Vacuity guard

Every enumeration yielding implausibly few items FAILS the run. It does not report a clean sweep over an empty set. Record the threshold per surface and the observed count.

G1.2 — Feature-vs-flag classification

A feature is a distinct user-visible capability with its own code path and its own success/failure semantics. A flag that selects a materially different code path is a feature (--backend cuda, --task pretrain, streaming vs not). A flag that only formats or labels output is not (--json, --verbose, -o).

G1.3 — value_parser domain consistency (new)

For every flag whose value selects a code path, assert a value_parser pins the domain at every declaration site.

Live defect this gate exists to catch: apr serve run --backend is declared with no value_parser — at HEAD, crates/apr-cli/src/serve_commands.rs:73 is exactly #[arg(long, value_name = "BACKEND")], zero BACKEND_VALUES — while apr chat (extended_commands.rs:85, arg line :84) and apr run (commands_enum.rs:154) both pin BACKEND_VALUES (the constant itself is commands_enum.rs:10). The value is consumed (serve/types.rs:57). That constant's own doc-comment says it exists to prevent "a run whose throughput number was taken through a backend the caller did not ask for" — reproduced on the serve path. The doc also claims it covers "apr run / apr chat"; there are three declaration sites, not two.

Also RED: apr bench has no --backend at all, despite being the throughput command.

G1.4 — Help-vs-dispatch completeness (new)

For every command with a --task-style selector, assert the set of documented values equals the set of reachable dispatch arms.

Live defect: apr eval --task dispatches 10 named arms plus a default perplexity path (dispatch_analysis.rs:1389-1425), but --help documents exactly two (extended_commands.rs:138). Nine paths — humaneval, mbpp, code, contamination, compare, verify, correlation, human, plan — are reachable and undocumented. The line range was 1383-1419 when it was measured at 4bbfeb07f and the arms have moved twice since; the count read "Eight" against a list of nine, and both were re-derived from the arms themselves rather than carried forward (PERF-046).

Undocumented-but-reachable is RED. Documented-but-unreachable is RED. This is the phantom-subcommand protocol (v2.0 P9) generalized to flag domains.

Phase 2 — Coverage ledger ⭐ THE GATE THAT ORGANIZES THE REST

Phase 1 produced the denominator. This phase computes the numerator and gates on the ratio.

G2.1 — Ledger freshness

docs/audits/surface_audit.csv must exist and must have been regenerated against the current HEAD. A ledger older than the commit under test FAILS.

This is the staleness arm. Without it the coverage number measures declared state rather than resolved state — the exact enforcement-theater shape this whole protocol exists to prevent.

Schema (10 columns, RFC 4180):

binary,feature,quality_1_10,verified_hardware,top_competitor,in_dogfood_skill,cluster_id,cluster_label,evidence_path,confidence

Every row requires evidence_path and a non-empty cluster_label. A row with no evidence path is an unevidenced claim; a row with no cluster label sits outside every per-cluster floor. Both fail schema validation.

There is ONE ledger. The clustered file replaced the 8-column one; it did not land beside it as surface_audit_clustered.csv. Two ledgers over one surface is the drift hazard this repo keeps re-finding — the second copy goes stale in silence and every consumer then has to be told which is authoritative. The clustered file was a superset in shape (same 830 rows, same order, two extra columns), so replacing cost nothing and keeping both would have bought a permanent divergence. Three cells did disagree — apr run --backend {cpu,cuda,wgpu} cited commands_enum.rs:110 in the clustered snapshot and :154 in the landed ledger; :154 is the backend: arg and :110 a chat-template arg, so the landed value won. That disagreement, found on the day the two files existed side by side, is the argument.

G2.2 — Denominator reconciliation

The Phase 1 runtime enumeration and the CSV row set must agree.

• A feature in the binary but not in the CSV → the ledger is stale. RED.
• A feature in the CSV but not in the binary → the surface shrank without the ledger noticing. RED.

Neither may be silently reconciled. This is what makes the CSV a ledger rather than a list — the list is regenerated, the ledger is checked against it.

G2.3 — Coverage floors

Compute in_dogfood_skill == yes over the reconciled denominator, overall and per binary and per quality band.

Baselines below are measured, not chosen — computed from docs/audits/surface_audit.csv (830 rows, 28 binaries) on 2026-08-22 at 4bbfeb07f. They are the committed starting point of the ratchet.

Band baselines, same measurement: 1–2 → 58/80 (72.5%); 3–4 → 27/49 (55.1%); 5–6 → 29/673 (4.3%); 7–8 → 8/8; 9–10 → 20/20.

Set the initial thresholds from measurement, not from a round number. Commit today's values as the baseline and ratchet. A threshold chosen before measurement either never fires or fires constantly.

The ratchet is the point: coverage is allowed to be 17.1%. It is not allowed to become 17.0%.

G2.5 — The per-cluster floor ⭐ THE THIRD FLOOR

Three floors now, not two: overall, per-binary, per-cluster.

Why a binary is the wrong unit

aprender-orchestrate ships 184 features that are three unrelated subsystems — 95 Banco HTTP routes, a 56-feature agent stack, 17 Pacha secrets commands. A per-binary floor of "≥ 1 gate" lets one gate on Pacha make all 184 look touched. The cluster is the unit whose members share a module, a dispatch path and a failure mode, which is the property that makes a gate on one member evidence about the rest.

What is enforced

per-cluster ratchet   (always)   no cluster_label's gate count may fall
                                 the zero-gate cluster count may not rise
                                 a cluster_label on the comparand may not vanish
per-cluster release arm          every cluster_label carries >= 1 gate   # RED at --release
T2 reporting pairing  (always)   cluster coverage is reported WITH the feature %,
                                 never one alone — ENFORCED, not documented

Every floor is derived from git show origin/main:docs/audits/surface_audit.csv, the same comparand the other floors use. No cluster count is a literal in any gate file. While main still carries the 8-column ledger the ratchet prints a SCHEMA UPGRADE banner instead of passing silently; that branch is self-closing and a half-migrated comparand is a hard failure, not an upgrade.

The four uses of the clustering

1. Stratified sampling frame. 830 gates were never the target. Features in a cluster share a failure mode, so a gate on one member is evidence about the cluster. The goal is n gates per cluster allocated by expected defect yield, with n scaling sub-linearly in cluster size. Nine clusters at zero is nine clusters with no evidence at all — that is the gap, not the 690 uncovered rows.
2. Sibling sweep. A defect in cluster X makes X's remaining members a mandatory sweep list in the same ticket. See Phase 3.
3. Harness amortization. 95 Banco routes is ONE HTTP harness, not 95 gates. 76 pv subcommands is ONE contract-CLI harness. Cluster size estimates gates-per-unit-effort and should drive ticket order: http-orchestrate-banco and contracts-pv are the two cheapest large wins in the repo and both sit at zero.
4. Per-cluster floor replacing per-binary as the release arm — the reason above.

The three traps

scripts/dogfood_cluster.py produces the k-selection evidence only — the inertia/silhouette sweep and docs/audits/surface_audit_elbow.png. It does not write cluster_id/cluster_label and re-running it will not regenerate them. That follows from T1: the labels are human-owned, so the columns are deliberately not reproducible from the script.

G2.4 — The 44

44 features carry quality ≤ 4 (a live ledger defect) and have no gate. All 44 are in apr. Every one is a defect that shipped and would ship again unnoticed.

Enumerate them in the receipt by name. At --release, this list must be empty or every entry must carry an open issue number and an explicit written waiver.

Phase 3 — Tiered gate execution

Tier 0 and 1 are the daily loop. Tier 2 adds cost. Tier 3 is release-only.

G3.0 — The sibling sweep rule (process, not a script)

When a defect lands in cluster X, X's remaining uncovered members become a MANDATORY sweep list in the same ticket.

The prior is measured, not assumed: the 0.63.0 ledger collapsed 201 findings into 37 root causes — ~5.4 findings per cause. Defects arrive in sibling groups. Clustering supplies that prior mechanically and in advance instead of retrospectively, after someone notices the fourth instance.

The rule, exactly:

1. A defect is found in a feature whose cluster_label is X.
2. The ticket enumerates every uncovered member of X, by name, from docs/audits/surface_audit.csv.
3. Each swept sibling ends in one of three states, recorded in the ticket: also broken (its own finding), checked and sound (evidence cited), or not reachable (why, and what would make it reachable).
4. A sweep that finds nothing is still a result. A sweep that was never enumerated is an open obligation, and the ticket does not close.

This is read-only, like the rest of the skill: the sweep produces findings and tickets, never fixes.

Do not skip the sweep because the cluster is large. A 95-member cluster is exactly where the prior pays: one HTTP harness covers the sweep, which is point 3 of the four uses above.

Tier 0 — Cheap, always (parallel)

Bodies carried verbatim from v2.0 Gates 4–7.

Gate 4: Contract Validation

# Verify contract is valid
. scripts/pv_bin.sh || exit 1   # exports $PV, the pinned in-tree pv
"$PV" validate contracts/apr-cli-qa-v1.yaml && echo VALID

# Run integration tests that enforce the contract
cargo test -p apr-cli --test cli_commands 2>&1 | tail -3
cargo test -p aprender-core --test readme_contract 2>&1 | tail -3
cargo test -p aprender-core --test monorepo_invariants 2>&1 | tail -3

PASS if all 3 test suites pass. FAIL if any test fails.

Gate 5: Code Quality

cargo test -p apr-cli --lib 2>&1 | grep "test result:"
cargo clippy -p apr-cli --lib -- -D warnings 2>&1 | grep "^error:" | wc -l

PASS if 0 test failures and 0 clippy errors. WARN if clippy warnings.

Gate 6: Coverage Check

cargo llvm-cov -p aprender-core --lib --no-report 2>&1 | tail -3
cargo llvm-cov report 2>&1 | tail -1

PASS if coverage >= 95%.

Gate 7: Open Issues

gh issue list --repo paiml/aprender --state open --limit 20

Always PASS (informational).

Tier 1 — The protocol grid

The gate bodies below are carried verbatim from v2.0 Gate 2 (full command grid) and Gate 3 (P1–P12). The table indexes them; P13 and P14 are new and still to be authored.

P6 carries the CF-4 constraint. The canonical failure (#1864) is a CPU-vs-GPU parity gate that sampled only decode step 0 and stayed green while GPU output collapsed to gibberish across later steps. Any gate touching an autoregressive, temporal, cached, streaming, or multi-turn path must validate over a horizon, never a single sample. Minimum 64 cached positions for attention parity. A single-step probe on a compounding system is inadmissible.

Gate 2: Full Command Grid (FALSIFY-QA-001, FALSIFY-QA-009)

Auto-discover models:

find ~/models -maxdepth 2 \( -name "*.apr" -o -name "*.gguf" -o -name "*.safetensors" \) -type f 2>/dev/null

Pick one per format. For EACH format, exercise ALL command categories:

2a. Inspection (11 commands)

. scripts/apr_bin.sh || exit 1
for cmd in inspect debug validate lint tensors trace diff hex tree flow explain; do
  echo -n "$cmd: " && timeout 30 "$APR" $cmd $MODEL 2>&1 | head -1 && echo "OK" || echo "FAIL"
done

2b. QA & Evaluation (8 commands)

. scripts/apr_bin.sh || exit 1
for cmd in check qa qualify bench eval canary compare-hf parity; do
  echo -n "$cmd: " && timeout 60 "$APR" $cmd $MODEL 2>&1 | head -1 && echo "OK" || echo "SKIP/FAIL"
done

2c. Transform (9 commands)

. scripts/apr_bin.sh || exit 1
for cmd in convert export import quantize merge prune compile encrypt decrypt; do
  echo -n "$cmd: " && timeout 30 "$APR" $cmd --help 2>&1 | head -1 && echo "OK" || echo "FAIL"
done

2d. Inference (4 commands) — timeout 60

. scripts/apr_bin.sh || exit 1
"$APR" run $MODEL "What is 2+2?" --max-tokens 16 2>&1 | tail -3
"$APR" serve plan $MODEL 2>&1 | head -5

2e. Registry (4 commands)

. scripts/apr_bin.sh || exit 1
"$APR" list 2>&1 | head -5
"$APR" list --json 2>&1 | jq length
"$APR" gpu 2>&1 | head -5

2f. Training & Data (6 commands)

. scripts/apr_bin.sh || exit 1
for cmd in finetune distill train tokenize tune data; do
  echo -n "$cmd: " && timeout 10 "$APR" $cmd --help 2>&1 | head -1 && echo "OK" || echo "FAIL"
done

2g. UI & Pipeline (7 commands)

. scripts/apr_bin.sh || exit 1
for cmd in tui monitor runs experiment pipeline diagnose showcase; do
  echo -n "$cmd: " && "$APR" $cmd --help 2>&1 | head -1 && echo "OK" || echo "FAIL"
done

2h. Remaining (8 commands)

. scripts/apr_bin.sh || exit 1
for cmd in rosetta publish oracle probar ptx ptx-map code cbtop; do
  echo -n "$cmd: " && "$APR" $cmd --help 2>&1 | head -1 && echo "OK" || echo "FAIL"
done

SKIP (not FAIL) if no models found. FAIL if any command panics or crashes.

Gate 3: Protocol Checks (12 protocols from apr-cookbook)

P1. Silent-Flag Protocol (FALSIFY-QA-007)

. scripts/apr_bin.sh || exit 1
diff <("$APR" inspect $M 2>&1) <("$APR" inspect --json $M 2>&1)
diff <("$APR" inspect $M 2>&1) <("$APR" inspect --vocab $M 2>&1)
diff <("$APR" list 2>&1) <("$APR" list --json 2>&1)

FAIL if any flag produces identical output (no-op flag).

P2. Exit-Code Contradiction (FALSIFY-QA-006)

for cmd in "apr lint $M" "apr validate /nonexistent" "apr rm nonexistent-id"; do
  out=$(eval "$cmd" 2>&1); ec=$?
  echo "$out" | grep -qiE 'error|fail' && [ "$ec" -eq 0 ] && echo "P1 EXIT-CODE LIE: $cmd"
done

P3. Flag-Echo Protocol

. scripts/apr_bin.sh || exit 1
out=$("$APR" run $M "test" --max-tokens 8 --temperature 0.5 2>&1)
# Verify temperature is actually 0.5, not default

P4. Cross-Subcommand Consistency (FALSIFY-QA-010)

. scripts/apr_bin.sh || exit 1
F_INSPECT=$("$APR" inspect --json $M 2>/dev/null | jq -r '.architecture // empty')
F_CHECK=$("$APR" check $M 2>&1 | grep -i arch | head -1)
echo "inspect=$F_INSPECT check=$F_CHECK"

P5. Cache Integrity

. scripts/apr_bin.sh || exit 1
BEFORE=$("$APR" list 2>&1 | wc -l)
# pull, list, rm cycle should be consistent

P6. Model capability ladder — GPU/CPU per ARCHITECTURE (F-MCL-001)

Contract: contracts/model-capability-ladder-v1.yaml. Born from #3477: 0.68.1 shipped dense Qwen3 producing garbage on CUDA on both fleet GPUs; the parity guard fell back to CPU so apr run looked fine, apr qa was the only surface that said "gibberish", and no release step ran apr qa on a Qwen3. The old P6 here ("compare apr run --device cpu vs --device gpu") picked ONE model and checked that text came out — the fallback path produces correct text, so it could never fail.

The unit of evidence is the triple (architecture, backend, silicon), one ladder rung per (arch, quant), one receipt per required host:

# on EVERY required host (ladder.hosts: lambda sm_89, gx10 sm_121):
. scripts/apr_bin.sh || exit 1
bash scripts/model_ladder.sh          # apr qa cap+golden per rung, + apr run per claimed
                                      # backend, RED on any "falling back to CPU" line
# writes evidence/dogfood/models/<version>/<host>.json — commit it
# then, anywhere:
bash scripts/check_model_ladder.sh    # the declared gate: every required rung green on
                                      # every required host, receipts fresh for this cut
bash scripts/check_model_ladder.sh --self-test   # the 13-case discrimination table

Rules the gate enforces (each has a RED case in scripts/lib/model_ladder_cases/):

• a Skipped apr qa gate is NOT passed even though --json marks it passed:true;
• a claimed backend that fell back is RED even when the tokens are right;
• a required rung the host does not hold is ABSENT = FAIL, not SKIP;
• a receipt with executed=0, a stale version, or a missing host is RED;
• the ladder at origin/main is a floor: rungs, hosts and backends may be added, never removed.

Missing receipt is FAIL, not DEFER — a dev build measures this; no crates.io needed. At 4a538ddef the gate is RED on both hosts (qwen3 cuda fallback #3413; qwen35 capability_match #3432) — that is its first-green proof of red-capability.

P7. NaN/Inf Sentinel (FALSIFY-QA-004)

for cmd in "apr run $M 'test' --max-tokens 8" "apr bench $M --iterations 1"; do
  timeout 30 eval "$cmd" 2>&1 | grep -qE '\bNaN\b|\bInf\b' && echo "P0 NaN: $cmd"
done

P8. Version Sanity (FALSIFY-QA-005)

. scripts/apr_bin.sh || exit 1
"$APR" --version | grep -qE '\(unknown\)|0000000' && echo "P3 VERSION SENTINEL"

P9. Phantom Subcommand (FALSIFY-QA-008)

. scripts/apr_bin.sh || exit 1
"$APR" --help | awk '/^  [a-z]/{print $1}' | while read cmd; do
  "$APR" "$cmd" --help 2>&1 | grep -qi "not.*implemented" && echo "PHANTOM: $cmd"
done

P10. JSON Schema Stability (FALSIFY-QA-003)

for cmd in "apr inspect --json $M" "apr list --json" "apr gpu --json"; do
  eval "$cmd" 2>&1 | jq . > /dev/null 2>&1 || echo "P2 INVALID JSON: $cmd"
done

P11. Default-Defamation Protocol

. scripts/apr_bin.sh || exit 1
"$APR" eval $M 2>&1 | grep -qi 'garbage\|broken\|corrupt' && echo "P3 DEFAMATION"

P12. Hardware Cascade Protocol

. scripts/apr_bin.sh || exit 1
# If GPU fails, does CPU fallback work?
"$APR" gpu 2>&1 | head -3

Tier 2 — Injection and metamorphic

Gate 8: Silent-Fallback Injection (F-SILENT-001 through F-SILENT-005)

Contract: contracts/apr-qa-silent-fallback-v1.yaml

Bad inputs MUST fail LOUD (non-zero exit + stderr message), never silently degrade.

S1. Truncated file detection (GH-707)

. scripts/apr_bin.sh || exit 1
M_GGUF=$(find ~/models -maxdepth 2 -name "*.gguf" -type f | head -1)
if [ -n "$M_GGUF" ]; then
  SIZE=$(stat -c%s "$M_GGUF")
  head -c $((SIZE / 2)) "$M_GGUF" > /tmp/apr-qa-truncated.gguf
  # IMPORTANT: capture exit code without piping (pipe loses $?)
  OUTPUT=$("$APR" validate /tmp/apr-qa-truncated.gguf 2>&1); EC=$?
  echo "$OUTPUT" | tail -3
  [ "$EC" -ne 0 ] && echo "S1 PASS: truncated file rejected (exit $EC)" || echo "S1 FAIL: truncated file accepted (GH-707)"
fi

S2. Bad file rejection

. scripts/apr_bin.sh || exit 1
OUTPUT=$("$APR" bench /dev/null --iterations 1 2>&1); EC=$?
echo "$OUTPUT" | tail -1
[ "$EC" -ne 0 ] && echo "S2 PASS: /dev/null rejected (exit $EC)" || echo "S2 FAIL: /dev/null accepted"

S3. Unknown architecture handling (GH-704 pattern)

. scripts/apr_bin.sh || exit 1
# Check that Qwen3.5 SSM model gets a clear error, not silent llama fallback
M_SSM=$(find ~/models -maxdepth 2 -name "*Qwen3.5*" -o -name "*qwen35*" 2>/dev/null | head -1)
if [ -n "$M_SSM" ]; then
  OUTPUT=$("$APR" run "$M_SSM" "test" --max-tokens 1 2>&1); EC=$?
  echo "$OUTPUT" | grep -qi "not.*supported\|unsupported\|SSM" && \
    echo "S3 PASS: unsupported arch gives clear error" || echo "S3 FAIL: no clear error for unsupported arch"
else
  echo "S3 SKIP: no SSM model available"
fi

S4. Corrupted metadata detection

. scripts/apr_bin.sh || exit 1
if [ -n "$M_GGUF" ]; then
  cp "$M_GGUF" /tmp/apr-qa-corrupt.gguf
  dd if=/dev/zero of=/tmp/apr-qa-corrupt.gguf bs=1 count=64 seek=8 conv=notrunc 2>/dev/null
  OUTPUT=$("$APR" validate /tmp/apr-qa-corrupt.gguf 2>&1); EC=$?
  echo "$OUTPUT" | tail -1
  [ "$EC" -ne 0 ] && echo "S4 PASS: corrupt metadata rejected (exit $EC)" || echo "S4 FAIL: corrupt metadata accepted"
fi

S5. Missing model graceful (FALSIFY-QA-002)

. scripts/apr_bin.sh || exit 1
OUTPUT=$("$APR" inspect /nonexistent/model.gguf 2>&1); EC=$?
echo "$OUTPUT" | tail -1
[ "$EC" -ne 0 ] && echo "S5 PASS: missing model exits non-zero (exit $EC)" || echo "S5 FAIL: exit 0 for missing model"

PASS if all 5 checks reject bad input. FAIL if any bad input is silently accepted.

Gate 9: Metamorphic Testing (F-META-001 through F-META-005)

Contract: contracts/apr-qa-metamorphic-v1.yaml

M1. Format roundtrip (GGUF→APR→GGUF tensor fidelity)

. scripts/apr_bin.sh || exit 1
if [ -n "$M_GGUF" ]; then
  "$APR" convert "$M_GGUF" --quantize q4k -o /tmp/apr-qa-rt.apr 2>&1 | tail -1  # --quantize is REQUIRED
  # If convert succeeds, check tensor count matches
  if [ -f /tmp/apr-qa-rt.apr ]; then
    ORIG_TENSORS=$("$APR" tensors "$M_GGUF" --json 2>/dev/null | jq length 2>/dev/null || echo 0)
    RT_TENSORS=$("$APR" tensors /tmp/apr-qa-rt.apr --json 2>/dev/null | jq length 2>/dev/null || echo 0)
    echo "M1 orig=$ORIG_TENSORS rt=$RT_TENSORS"
    [ "$ORIG_TENSORS" -gt 0 ] && [ "$RT_TENSORS" -gt 0 ] && echo "M1 PASS" || echo "M1 WARN: tensor count mismatch"
  else
    echo "M1 SKIP: convert not available for this model"
  fi
else
  echo "M1 SKIP: no GGUF model"
fi

M2. Multi-architecture smoke

. scripts/apr_bin.sh || exit 1
# Check that inspect works across all available model architectures
ARCH_COUNT=0
for m in $(find ~/models -maxdepth 2 \( -name "*.gguf" -o -name "*.apr" -o -name "*.safetensors" \) -type f 2>/dev/null); do
  ARCH=$(timeout 10 "$APR" inspect --json "$m" 2>/dev/null | jq -r '.architecture // empty' 2>/dev/null)
  [ -n "$ARCH" ] && ARCH_COUNT=$((ARCH_COUNT + 1)) && echo "  M2 arch=$ARCH ($m)"
done
[ "$ARCH_COUNT" -ge 2 ] && echo "M2 PASS: $ARCH_COUNT architectures inspected" || echo "M2 WARN: only $ARCH_COUNT architectures available"

M3. Temperature determinism (temp=0 → identical output across 3 runs)

. scripts/apr_bin.sh || exit 1
M_APR=$(find ~/models -maxdepth 2 -name "*.apr" -type f | head -1)
if [ -n "$M_APR" ]; then
  OUT1=$(timeout 60 "$APR" run "$M_APR" "Say hello" --max-tokens 4 --temperature 0.0 2>&1 | grep "^Output:" | head -1)
  OUT2=$(timeout 60 "$APR" run "$M_APR" "Say hello" --max-tokens 4 --temperature 0.0 2>&1 | grep "^Output:" | head -1)
  if [ "$OUT1" = "$OUT2" ] && [ -n "$OUT1" ]; then
    echo "M3 PASS: temp=0 deterministic"
  else
    echo "M3 WARN: temp=0 outputs differ (may be non-deterministic sampling)"
  fi
else
  echo "M3 SKIP: no APR model"
fi

PASS if M1+M2+M3 all pass. WARN if any are skipped due to missing models.

Gate 10: Coverage Completeness (F-COV-001 through F-COV-005)

Contract: contracts/apr-qa-coverage-v1.yaml

V1. Contract YAML validity (all 6 QA contracts parse)

. scripts/pv_bin.sh || exit 1   # exports $PV, the pinned in-tree pv
VALID=0; TOTAL=0
for c in contracts/apr-cli-qa-v1.yaml contracts/apr-qa-metamorphic-v1.yaml \
  contracts/apr-qa-silent-fallback-v1.yaml contracts/apr-qa-differential-v1.yaml \
  contracts/apr-qa-chaos-v1.yaml contracts/apr-qa-coverage-v1.yaml; do
  TOTAL=$((TOTAL+1))
  "$PV" validate "$c" > /dev/null 2>&1 && echo "  VALID: $c" && VALID=$((VALID+1))
done
echo "V1: $VALID/$TOTAL contracts valid"
[ "$VALID" -eq "$TOTAL" ] && echo "V1 PASS" || echo "V1 FAIL"

V2. Zero High-severity SATD in apr-cli

SATD_HIGH=$(pmat analyze satd -p crates/apr-cli/ 2>&1 | grep -c "High" 2>/dev/null || echo "0")
echo "V2: $SATD_HIGH High-severity SATD items"
[ "$SATD_HIGH" -eq 0 ] && echo "V2 PASS" || echo "V2 WARN: $SATD_HIGH High SATD items"

V3. Critical modules exercised (no panic on real model)

. scripts/apr_bin.sh || exit 1
M=$(find ~/models -maxdepth 2 \( -name "*.gguf" -o -name "*.apr" \) -type f | head -1)
if [ -n "$M" ]; then
  V3_PASS=0; V3_TOTAL=0
  for cmd in "hex" "profile --iterations 1"; do
    V3_TOTAL=$((V3_TOTAL+1))
    timeout 30 "$APR" $cmd "$M" 2>&1 | head -3 >/dev/null && V3_PASS=$((V3_PASS+1)) && echo "  V3 $cmd: OK" || echo "  V3 $cmd: FAIL/SKIP"
  done
  for cmd in "serve plan" "train plan"; do
    V3_TOTAL=$((V3_TOTAL+1))
    timeout 10 "$APR" $cmd "$M" 2>&1 | head -3 >/dev/null && V3_PASS=$((V3_PASS+1)) && echo "  V3 $cmd: OK" || echo "  V3 $cmd: FAIL/SKIP"
  done
  echo "V3: $V3_PASS/$V3_TOTAL modules exercised"
  [ "$V3_PASS" -ge 2 ] && echo "V3 PASS" || echo "V3 WARN"
else
  echo "V3 SKIP: no model"
fi

V4. Complexity hotspots tracked

# Count true CC>15 functions via JSON. The ANSI-coloured text output also
# contains section headers whose last numeric field exceeds 15 (e.g. the
# refactoring-time estimate or the per-file Cyclomatic totals), so awk over
# stdout was over-counting; use the structured format instead.
HIGH_CC=$(pmat analyze complexity -p crates/apr-cli/ --format json 2>/dev/null \
  | jq '[.files[].functions[] | select(.metrics.cyclomatic > 15)] | length' 2>/dev/null \
  || echo "0")
echo "V4: $HIGH_CC functions with CC > 15"
[ "$HIGH_CC" -le 3 ] && echo "V4 PASS" || echo "V4 WARN: $HIGH_CC high-complexity functions"

Gate 10 Verdict (GH-716)

# Re-compute V1-V4 for a single aggregate verdict. V1 (contracts parse) and
# V3 (critical modules run) are required; V2 (SATD) and V4 (complexity) are
# quality signals that demote PASS → WARN but never cause FAIL on their own.
. scripts/pv_bin.sh || exit 1
V1_OK=0
for c in contracts/apr-cli-qa-v1.yaml contracts/apr-qa-metamorphic-v1.yaml \
  contracts/apr-qa-silent-fallback-v1.yaml contracts/apr-qa-differential-v1.yaml \
  contracts/apr-qa-chaos-v1.yaml contracts/apr-qa-coverage-v1.yaml; do
  "$PV" validate "$c" > /dev/null 2>&1 && V1_OK=$((V1_OK+1))
done
V2_SATD=$(pmat analyze satd -p crates/apr-cli/ 2>&1 | grep -c "High" 2>/dev/null || echo "0")
V4_CC=$(pmat analyze complexity -p crates/apr-cli/ --format json 2>/dev/null \
  | jq '[.files[].functions[] | select(.metrics.cyclomatic > 15)] | length' 2>/dev/null \
  || echo "0")
M=$(find ~/models -maxdepth 2 \( -name "*.gguf" -o -name "*.apr" \) -type f | head -1)
V3_OK=$([ -n "$M" ] && echo 1 || echo 0)

if [ "$V1_OK" -eq 6 ] && [ "$V3_OK" -eq 1 ]; then
  if [ "$V2_SATD" -eq 0 ] && [ "$V4_CC" -le 3 ]; then
    echo "Gate 10: PASS (V1=6/6 V2=0 SATD V3=model V4=$V4_CC CC)"
  else
    echo "Gate 10: WARN (V1+V3 pass; V2=$V2_SATD SATD V4=$V4_CC CC)"
  fi
else
  echo "Gate 10: FAIL (V1=$V1_OK/6 V3=$([ "$V3_OK" = "1" ] && echo ok || echo no-model))"
fi

PASS requires V1+V3. V2 (SATD) and V4 (complexity) demote PASS → WARN.

Gate 11: Chaos Engineering (F-CHAOS-001 through F-CHAOS-005)

Contract: contracts/apr-qa-chaos-v1.yaml

C1. Memory budget (RSS sanity check)

M=$(find ~/models -maxdepth 2 -name "*.gguf" -type f -size -1G | head -1)
if [ -n "$M" ]; then
  MODEL_KB=$(du -k "$M" | cut -f1)
  RSS_KB=$(/usr/bin/time -v timeout 30 apr inspect "$M" 2>&1 | grep "Maximum resident" | awk '{print $NF}' 2>/dev/null || echo 0)
  if [ "$RSS_KB" -gt 0 ]; then
    BUDGET_KB=$(( MODEL_KB * 3 + 524288 ))
    echo "C1: model=${MODEL_KB}KB RSS=${RSS_KB}KB budget=${BUDGET_KB}KB"
    [ "$RSS_KB" -lt "$BUDGET_KB" ] && echo "C1 PASS" || echo "C1 WARN: RSS exceeds 3x model + 512MB"
  else
    echo "C1 SKIP: /usr/bin/time not available"
  fi
else
  echo "C1 SKIP: no small GGUF model"
fi

C2. Overwrite protection

. scripts/apr_bin.sh || exit 1
touch /tmp/apr-qa-existing.apr
"$APR" convert /dev/null -o /tmp/apr-qa-existing.apr 2>&1; EC=$?
# Should either fail (non-zero) or prompt — never silently overwrite
[ "$EC" -ne 0 ] && echo "C2 PASS: existing file not silently overwritten" || echo "C2 WARN: may have overwritten"

C3. SIGINT handling

. scripts/apr_bin.sh || exit 1
M_APR=$(find ~/models -maxdepth 2 -name "*.apr" -type f | head -1)
if [ -n "$M_APR" ]; then
  timeout 5 "$APR" run "$M_APR" "Tell me a very long story about everything" --max-tokens 500 &
  PID=$!
  sleep 2
  kill -INT $PID 2>/dev/null
  wait $PID 2>/dev/null; EC=$?
  # SIGINT should produce exit 130 or similar non-zero, NOT leave zombie
  [ "$EC" -ne 0 ] && echo "C3 PASS: SIGINT handled (exit $EC)" || echo "C3 WARN: SIGINT exit 0"
else
  echo "C3 SKIP: no APR model"
fi

PASS if C1+C2+C3 all pass. WARN on skips.

Gate 12: Differential Testing (F-DIFF-001 through F-DIFF-005)

Contract: contracts/apr-qa-differential-v1.yaml

D1. Cross-format tensor agreement

. scripts/apr_bin.sh || exit 1
M_GGUF=$(find ~/models -maxdepth 2 -name "*.gguf" -type f | head -1)
M_APR=$(find ~/models -maxdepth 2 -name "*.apr" -type f | head -1)
if [ -n "$M_GGUF" ] && [ -n "$M_APR" ]; then
  GGUF_COUNT=$("$APR" tensors "$M_GGUF" --json 2>/dev/null | jq length 2>/dev/null || echo 0)
  APR_COUNT=$("$APR" tensors "$M_APR" --json 2>/dev/null | jq length 2>/dev/null || echo 0)
  echo "D1: GGUF tensors=$GGUF_COUNT APR tensors=$APR_COUNT"
  [ "$GGUF_COUNT" -gt 0 ] && [ "$APR_COUNT" -gt 0 ] && echo "D1 PASS: both formats report tensors" || echo "D1 WARN"
else
  echo "D1 SKIP: need both GGUF and APR models"
fi

D2. Ollama parity (if ollama installed)

if command -v ollama &>/dev/null; then
  OLLAMA_MODELS=$(ollama list 2>/dev/null | tail -n +2 | head -3)
  if [ -n "$OLLAMA_MODELS" ]; then
    echo "D2: ollama available with models — manual parity check recommended"
    echo "D2 SKIP: automated parity not yet wired"
  else
    echo "D2 SKIP: ollama installed but no models"
  fi
else
  echo "D2 SKIP: ollama not installed"
fi

D3. JSON schema consistency across commands

. scripts/apr_bin.sh || exit 1
M=$(find ~/models -maxdepth 2 \( -name "*.gguf" -o -name "*.apr" \) -type f | head -1)
if [ -n "$M" ]; then
  D3_PASS=0
  for cmd in "inspect --json" "check --json" "list --json" "gpu --json"; do
    timeout 15 "$APR" $cmd $M 2>/dev/null | jq . >/dev/null 2>&1 && D3_PASS=$((D3_PASS+1))
  done
  echo "D3: $D3_PASS/4 JSON outputs valid"
  [ "$D3_PASS" -ge 3 ] && echo "D3 PASS" || echo "D3 WARN"
else
  echo "D3 SKIP: no model"
fi

PASS if D1+D3 pass. SKIP for D2 (requires ollama setup).

Gate 14: APR → GGUF Export Round-trip (F-EXPORT-ROUNDTRIP-001)

Contract: contracts/apr-export-num-layers-v1.yaml

Catches #1865 — apr export <model>.apr --format gguf panicking with exit 101 on APR files missing num_layers metadata. Every APR file in the registry must export without panic; exit 5 (clean ValidationFailed) is acceptable, exit 101 is a FAIL.

. scripts/apr_bin.sh || exit 1
G14_PASS=0
G14_TOTAL=0
for apr in $(find ~/models -maxdepth 2 -name "*.apr" -type f 2>/dev/null); do
  G14_TOTAL=$((G14_TOTAL+1))
  OUT=$(timeout 60 "$APR" export "$apr" --format gguf -o /tmp/g14-rt.gguf 2>&1); EC=$?
  # IMPORTANT: capture exit code via OUT=$(...); EC=$? — never via pipe (see Pre-Gate note).
  if [ "$EC" -eq 101 ] || echo "$OUT" | grep -qE "thread .* panicked"; then
    echo "G14 FAIL ($apr): panic exit=$EC"
  elif [ "$EC" -eq 0 ] || [ "$EC" -eq 5 ]; then
    G14_PASS=$((G14_PASS+1))
    echo "G14 OK ($apr): exit=$EC (0=success, 5=clean validation error)"
  else
    echo "G14 WARN ($apr): unexpected exit=$EC"
  fi
  rm -f /tmp/g14-rt.gguf
done
[ "$G14_TOTAL" -eq 0 ] && echo "G14 SKIP: no APR models found" \
  || { [ "$G14_PASS" -eq "$G14_TOTAL" ] && echo "G14 PASS: $G14_PASS/$G14_TOTAL exported without panic" \
       || echo "G14 FAIL: only $G14_PASS/$G14_TOTAL clean"; }

PASS if every APR file either exports successfully or exits 5. FAIL on any panic (exit 101 or stderr panic message). SKIP if no APR models in registry.

Gate 15: validate --quality Sanity (F-VALIDATE-QUALITY-001)

Contract: contracts/apr-validate-quality-threshold-v1.yaml

Catches #1866 — apr validate --quality returning Grade F exit 5 on every working model because 22/25 checks are stubbed Skip(Not implemented) and the threshold gate compared against the full 100-point ceiling.

. scripts/apr_bin.sh || exit 1
# Find a known-good model — one that apr qa says is fine.
M=$(find ~/models -maxdepth 2 \( -name "*.apr" -o -name "*.gguf" \) -type f | head -1)
if [ -z "$M" ]; then
  echo "G15 SKIP: no model available"
else
  OUT=$(timeout 90 "$APR" validate "$M" --quality 2>&1); EC=$?
  # apr qa is the canonical pass/fail (CLAUDE.md). If qa passes, validate --quality
  # MUST NOT exit non-zero solely because checks are unimplemented.
  QA_OUT=$(timeout 120 "$APR" qa "$M" 2>&1 | grep -E "ALL GATES PASSED|FAIL"); QA_PASSES=$?
  if echo "$QA_OUT" | grep -q "ALL GATES PASSED" && [ "$EC" -ne 0 ]; then
    echo "G15 FAIL: apr qa says ✓ ALL GATES PASSED but apr validate --quality exit=$EC (#1866)"
    echo "         likely score threshold counting Skip(Not implemented) against runnable denom"
  else
    echo "G15 PASS: validate --quality consistent with apr qa verdict (exit=$EC)"
  fi
fi

PASS if apr validate --quality exits 0 on any model that apr qa passes. FAIL on the inconsistency that #1866 captured.

Gate 16: apr run Exit Code Reflects Output Validity (F-RUN-EXIT-SANITY-001)

Contract: contracts/apr-cpu-vs-gpu-output-parity-v1.yaml

Catches the secondary defect from #1864 — apr run exiting 0 even when GPU dispatch produced obvious gibberish.

. scripts/apr_bin.sh || exit 1
M=$(find ~/models -maxdepth 2 -name "*.apr" -type f | head -1)
if [ -z "$M" ]; then
  echo "G16 SKIP: no APR model"
else
  OUT=$(timeout 90 "$APR" run "$M" "What is 2+2?" --max-tokens 16 2>&1); EC=$?
  # Heuristic gibberish detectors. Real models answering 2+2 should produce
  # digits or short English. If the output contains chat-template control tokens
  # (e.g. <|im_start|>, <|endoftext|>) repeated, OR is dominated by a single
  # non-numeric word repeating, treat that as a parity-gate-missed regression.
  if echo "$OUT" | grep -qE '<\|im_start\|>.*<\|im_start\|>' \
     || echo "$OUT" | grep -qE '<\|endoftext\|>.*<\|endoftext\|>'; then
    if [ "$EC" -eq 0 ]; then
      echo "G16 FAIL: chat-template gibberish + exit 0 (#1864 secondary)"
    else
      echo "G16 PASS: gibberish detected AND exit=$EC (gate fired)"
    fi
  else
    OUTPUT_LINE=$(echo "$OUT" | sed -n '/^Output:/,$p' | tail -n +2 | tr -d '[:space:]')
    if [ -n "$OUTPUT_LINE" ] && [ "$EC" -eq 0 ]; then
      echo "G16 PASS: clean output, exit=0"
    elif [ "$EC" -ne 0 ]; then
      echo "G16 PASS: non-zero exit=$EC (clean failure path)"
    else
      echo "G16 WARN: output unparseable but exit=0 — inspect manually"
    fi
  fi
fi

PASS if apr run either emits clean output with exit 0, or non-clean output with non-zero exit. FAIL when chat-template gibberish leaks through with exit 0.

Gate 18: Fresh-Convert .apr Inference Parity (F-APR-INFERENCE-PARITY-001)

Contract: contracts/apr-cpu-vs-gpu-output-parity-v1.yaml (the .apr↔GGUF inference invariant)

Catches the PMAT-888 class: a .apr converted by the CURRENT binary produces garbage on inference (mojibake / cosine ~0.7) while the source GGUF is coherent — a converter/loader regression. Gate 16 only runs a PRE-EXISTING ~/models/*.apr (converted by an OLD binary, so it still works), and inspect/validate/tensors (Gate 2a) all pass on a broken-for-inference .apr — so none of them catch it. The native .apr format is the whole project; its inference path MUST be gated on a FRESH convert against the GGUF, on BOTH CPU and GPU. This is the gate the 0.50.0 post-publish QA was missing (it tested .apr inspect/validate but never .apr run).

. scripts/apr_bin.sh || exit 1
M_GGUF=$(find ~/models -maxdepth 2 -name "*.gguf" -type f -size -3G | head -1)
if [ -z "$M_GGUF" ]; then echo "G18 SKIP: no GGUF model"; else
  # NB: `apr convert` REQUIRES --quantize (or --compress). Omitting it fails with
  # "At least one of --quantize or --compress must be specified" and writes NO .apr —
  # which a naive test misreads as empty inference output. ALWAYS pass --quantize.
  "$APR" convert "$M_GGUF" --quantize q4k -o /tmp/g18-fresh.apr 2>&1 | tail -1
  [ -f /tmp/g18-fresh.apr ] || echo "G18 FAIL: apr convert produced no .apr (forgot --quantize?)"
  norm(){ sed -n '/^Output:/,$p' | tail -n +2 | tr -d '[:space:]'; }
  # The P0 (PMAT-888) was GARBAGE, not a verbosity diff. GGUF runs may be response-CACHED
  # (terser, e.g. "4" vs ".apr"'s fresh "2+2 equals 4."), so the gate is the .apr's
  # COHERENCE (the real P0 signal), NOT byte-equality with GGUF.
  coherent(){ [ -n "$1" ] && echo "$1" | grep -qE '[0-9A-Za-z]' \
              && ! echo "$1" | grep -qE 'ä|ã|�|<\|im_start|<\|endoftext'; }
  GGUF_OUT=$(timeout 120 "$APR" run "$M_GGUF"        --no-gpu --prompt "What is 2+2?" --max-tokens 12 2>&1 | norm)
  APR_OUT=$( timeout 120 "$APR" run /tmp/g18-fresh.apr --no-gpu --prompt "What is 2+2?" --max-tokens 12 2>&1 | norm)
  echo "G18 gguf=[$GGUF_OUT] apr=[$APR_OUT]"
  if coherent "$APR_OUT"; then
    [ "$GGUF_OUT" = "$APR_OUT" ] && echo "G18 PASS: fresh .apr CPU inference coherent AND == GGUF" \
                                 || echo "G18 PASS: fresh .apr CPU inference coherent (differs from possibly-cached GGUF; both valid answers)"
  elif [ -z "$APR_OUT" ]; then
    echo "G18 FAIL: fresh .apr produced NO output (broken inference, or convert wrote no model)"
  else
    echo "G18 FAIL: fresh .apr produces garbage while GGUF coherent (PMAT-888 converter/loader regression)"
  fi
  # GPU leg (if a GPU is present): the .apr GPU path must also be coherent
  if apr gpu 2>&1 | grep -qiE 'cuda|gpu.*(yes|available|RTX|GB10)'; then
    APR_GPU=$(timeout 120 "$APR" run /tmp/g18-fresh.apr --prompt "What is 2+2?" --max-tokens 12 2>&1 | norm)
    coherent "$APR_GPU" && echo "G18-GPU PASS: fresh .apr GPU coherent" \
      || echo "G18-GPU FAIL: fresh .apr GPU=[$APR_GPU] not coherent"
  fi
  rm -f /tmp/g18-fresh.apr
fi

PASS if a freshly-converted .apr's CPU (and GPU, when present) inference matches the source GGUF token-for-token. FAIL on garbage (the PMAT-888 regression). SKIP if no GGUF model is available.

Tier 3 — Release only

Gate 17: 7B Inference Smoke (F-7B-INFERENCE-001)

Catches #1864 directly. The README claims Qwen2.5-Coder 7B Q4_K 225+ tok/s RTX 4090 as the headline configuration; if 7B GPU inference produces gibberish, the canonical demo is broken.

. scripts/apr_bin.sh || exit 1
M_7B=$(find ~/models -maxdepth 2 -name "*7b*q4*" -type f 2>/dev/null | head -1)
if [ -z "$M_7B" ]; then
  echo "G17 SKIP: no 7B Q4_K model in registry"
else
  # apr qa Golden Output gate already encodes correctness; reuse it.
  OUT=$(timeout 300 "$APR" qa "$M_7B" 2>&1 | grep -E "Golden Output")
  if echo "$OUT" | grep -q "FAIL"; then
    echo "G17 FAIL: 7B Golden Output gate FAILS — $OUT (#1864)"
  elif echo "$OUT" | grep -q "PASS"; then
    echo "G17 PASS: 7B Golden Output gate passes"
  else
    echo "G17 SKIP: Golden Output gate didn't run (no GPU? --assert-gpu missing?)"
  fi
fi

PASS when apr qa Golden Output gate passes on the 7B Q4_K model. FAIL on the regression that #1864 captured. SKIP when the 7B model isn't available or the gate didn't run.