Numerical Check

Numerical Check: Falsify a Self-Authored Math Claim by Sweep

Empirically stress-test a mathematical claim you wrote but have not proven. The goal is falsification: throw many random instances at the claim and try to break it. A single genuine counterexample kills the claim; a large clean sweep is evidence, never proof.

When to Use

• You wrote a Proposition / Theorem / Conjecture (monotonicity, threshold, comparative-static, inequality, closed-form, limit) and want to know if it's actually true before claiming it.
• numerical-check, "stress-test my conjecture", "find a counterexample to X", "is Q(ρ) really monotone", "does the threshold hold for all …".
• The write-time empirical arm of the mark-unverified rule (self-authored math must be checked before assertion).

When NOT to Use

Position in the verification spectrum

R1 — numerical falsification. Can FALSIFY definitively (a confirmed counterexample refutes the claim) but can never VERIFY (no counterexample ≠ proof). The strongest positive result is INCONCLUSIVE (supported): no counterexample in N draws. Pair with lean-check (R3) to prove the claim once it survives.

Procedure

1. Formalize the claim as a predicate over a domain

Restate the claim as P(x) that must hold for all x in a domain D. Make the failure condition explicit and quantitative.

• "Q(ρ) is monotone decreasing in ρ" → P(instance) := max_i (Q(ρ_{i+1}) − Q(ρ_i)) ≤ tol over a ρ-grid.
• "threshold ρ* separates help/hurt" → P := (Q<p_max) iff (ρ>ρ*).
• Write down the domain D precisely (which parameters, which ranges, which side-conditions — e.g. "mean competence > ½, dispersed").

2. Sample the domain to approximate the TRUE object — not finite-n atoms

This is the step that fools people. If the claim is about a continuous or large-n limit object, a tiny discrete instance is NOT that object — it carries finite-n artifacts (ties, atoms, degenerate medians, staircase discontinuities) that manufacture fake violations.

• If the claim is a large-n / continuous-distribution statement, represent each random instance with dense sampling (hundreds of points), so the computed quantity approximates the limit.
• Generate instances from varied shapes (uniform, skewed, bimodal, heavy-tailed) so the sweep is adversarial, not cherry-picked.

3. Evaluate on an INTERIOR grid, smoothly, with a noise-aware tolerance

• Grid the parameter on the interior (e.g. ρ ∈ [0.02, 0.98]); endpoints breed boundary/degeneracy artifacts.
• Prefer a smooth evaluation (e.g. root-find the crossing) over indicator-quadrature, which quantization-jitters and creates false steps.
• Set the violation tolerance an order of magnitude above the numerical noise floor (measure the floor on a case you believe holds). Too tight → false positives; too loose → misses real breaks.

4. Sweep, count, capture the worst — seeded

• Run over thousands of random instances (uv run --no-project --with numpy --with scipy python; never bare python3). Seed the RNG.
• Count genuine violations; capture the worst counterexample (the instance + violation magnitude) for reporting and for the figure.

5. Diagnose a surprise BEFORE trusting it

If you find violations (or a suspiciously high/low rate), do not report the raw number yet. Check it is not an artifact:

• Re-plot the worst case on a fine grid — is the "violation" a real interior feature, or a jump at an endpoint / a finite-n tie?
• Re-run with denser sampling and odd vs even n — does the rate persist as you approach the continuous limit? An artifact shrinks; a real effect stays.
• Only once it survives these is it real. (Document the surprise + the diagnosis — design-before-results.)