Test Audit

KeeForge Test Audit

Improve regression protection and reduce maintenance without losing distinct contracts. Test count and deleted lines are not success criteria. Adapted from OpenClaw's test-audit for KeeForge's XCTest, shared platform targets, and independent compatibility gates.

Scope and authority

Read root and affected folder guidance, including local overrides. For an audit, pin the reviewed commit and record dirty files separately. Establish the subsystem and whether the request covers discovery, cleanup, or a complete campaign. Default an audit to read-only discovery and a report under the main checkout's shared scratch/test-audit/; tests may create build artifacts. Find the main checkout with git worktree list --porcelain. A plan-only request excludes test execution and file writes unless requested separately. Apply existing user authorization rather than asking again. An audit alone does not authorize source/test fixes, issue mutations, commits, or pushing.

Use pre-release-review for change-driven release readiness. When working within that static review, contribute coverage gaps to its action list without running tests or creating a separate report. For ordinary test changes, apply the checklist below without producing a separate audit report.

Authoring checklist

Before adding or changing a test, identify:

• The observable outcome or independent contract and a credible regression it detects.
• The production boundary that owns the behavior and why existing tests do not already catch that regression. Extend a case table or shared fixture when it expresses the same contract without obscuring distinct failure paths.
• An expectation independent of the behavior under test. A mock should supply inputs, record effects, or simulate an external boundary; it must not implement the outcome the test claims to prove.
• Any injection or support needed, and why it is the smallest reliable way to exercise the real owner. Constructor injection, fake transports, clocks, notification centers, and system-store fakes can be necessary for deterministic, headless tests. Remove unnecessary hooks, not testability itself.

For bug regressions, demonstrate failure on the old behavior for the intended reason and success with the repair when feasible; record any unverified control explicitly. Use a disposable checkout or reversible isolated patch for controls, preserving user changes. A failing build or an unrelated guard is not a regression control.

Discovery and retention

Read KeeForgeTests/AGENTS.md and relevant support/fixture guidance before judging overlap. Read KeeForgeMac/README.md's testing table for affected Mac behavior, and .github/AGENTS.md, ci_scripts/README.md, and actual target/scheme selection for execution coverage. Read each candidate test in full, its production owner, callers, relevant callees, overlapping suites, and history. Inspect dependency code or types when a claim relies on a dependency contract.

Look for assertions that merely mirror implementation, expected values computed by the same helper under test, assertion-free execution, mocks supplying the alleged result, negative cases rejected by the wrong guard, duplicated scenarios at the same boundary, stale test-only paths, or names claiming more than assertions establish. These are investigation leads, not automatic deletion rules.

Preserve distinct guarantees:

• Shared iOS/Mac execution, compile guards, and Mac App Store/direct-build differences can expose different failures. Check actual CI routing and skips; compiling a test does not prove it executes. Mac UI tests are local-only.
• Parser/XML, encrypted-container, in-memory edit, save safety, and compatibility suites have separate owners in the test docs. Foreign-authored inputs and KeePassXC opening KeeForge output are independent evidence. In-process round trips cannot replace an external oracle; the external reader cannot inspect every preserved byte.
• Keep the compatibility harness's one assertion-bearing execution per artifact scenario, complete artifact inventory, and external expectations. Do not duplicate expensive KDF work to produce artifacts or shrink the matrix to improve timings.
• Security boundaries, defaults, migration/storage formats, localization catalogs, entitlements, extension membership, and release tooling can justify static or exact value checks. Judge the independent contract, not the assertion's appearance.
• Unit tests do not replace real focus/keyboard/window interactions or system AutoFill panels. Prefer headless tests where they reach the behavior; retain the smallest distinct UI proof. Screenshot captures are visual evidence, not assertion-free junk.

Do not delete a baseline failure because it is inconvenient. Separate product defects, test defects, environment failures, skips, and unknown causes before recommending work. Retain uncertain coverage pending evidence. Slow execution alone is not a removal case.