Eviction Policy Regret Audit

The round

Workspace: .local/experiments/audit-regret-<yyyy-mm-dd>/ with LEDGER.md (one row per candidate, status updated in place), tree/ (the harness worktree), specs/, traces/, dumps/, data/. Report: .local/audits/<model>/audit-regret.md (docs/audit-output.md). Both survive rebases; nothing checked in may reference them. The commands, in the round's order, run from the workspace (regret.py/search.py default --traces-dir/--dump-dir to ./traces and ./dumps):

A=$PWD/.claude/skills/audit-regret; G=$PWD/.claude/skills/climber-gate
W=.local/experiments/audit-regret-<date>; mkdir -p $W/specs $W/data $W/traces && cd $W
git worktree add --detach tree HEAD
python3 $G/harness.py apply $PWD/tree && python3 $G/harness.py verify $PWD/tree      # counts match
export CAF_TREE=$PWD/tree
python3 $G/<generator> ... --out traces/<row>.lirs                                   # 0: a gate row
python3 $A/regret.py traces/<row>.lirs --size 8192 --seeds 1
python3 $A/search.py eval specs --csv data/screen.csv --seeds 1                      # 2: screen
python3 $A/search.py shrink specs/<c>.json --csv data/shrink.csv --out specs/<c>_min.json  # 4
python3 $A/search.py bisect specs/<c>_min.json --knob <path> --lo <a> --hi <b> --bar 2 \
    --csv data/bisect.csv
python3 $A/regret.py specs/<c>_min.json --seeds 1,2,3,4,5,6,7,8 \
    --variants hybrid,reactive,noaudit --csv data/classify.csv --json data/<c>_min.json   # 5
python3 $A/search.py mutate specs/<c>_min.json --set <knob>=<lo>,<hi> --set max=8192,16384 \
    --out specs/nbhd && python3 $A/search.py eval specs/nbhd --csv data/nbhd.csv --seeds 1  # 6

0. Baseline. Record the tree (git rev-parse HEAD), build the harness worktree, and reproduce one recorded gate row to prove the instruments read the same numbers: regenerate it from the gate's generation block and pick a low-spread row (straywall2_8192_d050; demoflood_8192 is bimodal since the 2026-08-18 re-base and no longer qualifies), not a lottery cell, since the recorded means are unseeded and a --seeds 1 draw must land inside the row's spread. Read hill-climber.md §3 (the recorded families), §5 (the graveyard), §5's last entries before proposing anything, and keep the §3 family list and the gate table at hand for step 3.

1. Propose. Spawn the two proposal lanes in parallel, each returning specs plus a one-paragraph prediction per spec: the class it targets, the state variable it expects to see fail, the terrain argument (why the static curve has a better window than the machine will find), and the regret it expects. The prediction is the finding's hypothesis; a spec whose regret appears for a different reason than predicted is a new finding, not a confirmation. Write each spec and its prediction together before screening that spec.

• The blind lane gets WindowClimber.java, hill-climber.md §2 and §4 (the machine), the member/segment grammar, and the class table above. Nothing about §3, §5, or the gate table. Fresh eyes on the mechanism find shapes the recorded families anchor a sighted reader away from.
• The sighted lane additionally gets §3 and the directions carried from earlier rounds below. Point it to §5 and the gate table by heading rather than copying both into its brief. It must consult the relevant entries, name the nearest recorded family and state what differs, and must not rebuild anything §5 killed.

Give each lane a target count (six to eight specs), a class quota when the argument names one, and the mechanism-directed prompt: for each class, "construct the workload that maximizes this class's regret against this machine, and say which line of densityClimb decides wrongly on it". A lane writes specs; it runs nothing.

Dispatch both lanes on Opus (model: opus), each with an explicit time box of about twenty minutes; dispatch step 7's evaluators on the session model (Fable), since refuting a finding with runs and reading its trajectories is where round 1's errors slipped through on Opus (an evaluator confirmed "the plant at 70% holds" without opening the trajectory, where it was one audit park inside a 64-sample horizon, and let a post-tick misreading of the audit clock stand), and where round 2's Fable evaluators corrected both writeups materially. The lanes differ by what they are shown, not by model: a Fable proposal lane spent four consecutive 96,000-token thinking turns on the open-ended design prompt and was aborted at the output-token cap after two hours with nothing written, while the Opus lane on the same prompt delivered eight specs in fourteen minutes (round 2, 2026-08-16; the earlier Fable attempt at this task shape tripped a safety classifier on the construction framing, 2026-07-30). Ben's direction, 2026-08-17 (either is fine; pick what finds issues), replacing the 2026-08-16 one-lane-per-model rule. If a Fable lane is ever wanted for proposal diversity, run it as a third, non-blocking lane with a different shape: three or four specs, each written to disk the moment it is designed so the harness has tool rounds to deliver a time-box message, and screen whatever it wrote when the round's screen runs; never wait on it. Keep every prompt concrete about the domain it is in (simulator request traces, a cache policy's hit rate, a window-sizing loop) rather than generic red-team language.

2. Screen. search.py eval <specs> --csv screen.csv --seeds 1 on the harness tree at the spec's own maximum (8192 unless the attack is about scale), so every row carries a hint. Apply the validity gates before reading a gap: n ≥ 40, no peak-at-edge unless the attack is structural, and a headroom ≥ 1 prize or a deliberate hold cell. Keep every cell with gap ≥ 2 (or missx ≥ 20% on a low-rate cell) as a candidate.

3. Deduplicate. For each candidate, name the nearest recorded family (§3, the gate table, the frontier sentinels) and the difference; a candidate that is a recorded family at another dose is noted on that family's row and dropped from this round unless its dose is new. pair a candidate against its nearest neighbor when the mechanism is claimed to be the same.

4. Shrink and parameterize. search.py shrink to the minimal witness (class-preserving), then bisect the one continuous knob that carries the effect for its transition and shape, then express the witness as a family with one to three knobs (dose, distance or period relative to max, phase length in samples). Record the transition; a threshold and a gradient are different mechanisms.

5. Classify. regret.py at eight seeds with --variants hybrid,reactive,noaudit, the trajectory read per seed (a bimodal cell has a class per basin; the reactive arm has no trajectory), the layer named by the procedure above, and the responder named in the debug line's vocabulary. If the adjudicated class is not the hint the shrink preserved, the witness may hold a different mechanism than its parent: shrink again from the parent at more seeds. Add --start plants when the question is path dependence, marginal.py when it is the rest point, exposure.py/blindlock.py when it is the clock. Where the arms tie the shipped machine, say so, and name what they share.

6. Neighborhood. mutate the family's knobs one step each way and to a second maximum (16384 or 32768; 4096/4097 when the tier is implicated), eval, and record the surface. The neighborhood is what a fix will be judged on.

7. Challenge. Spawn one evaluator per finding (the session model, Fable) with the finding's writeup, its CSV rows, the anchor sidecars, and the dumps, no source, told to refute it: is it warmup, a basin, an alignment artifact, a peak beyond 80% or between two swept windows, a recorded family re-dosed, or an anchor error? Every objection is answered by a run or the finding is downgraded.

8. Fix, only when asked. Present the classified findings and stop; Ben decides which class to work, one at a time. A candidate change is the smallest one that closes the class: it must close the whole neighborhood (step 6), not the witness, or it has fit the benchmark; it must hold gate.py on the full battery, real.py, floors.py, and the reactive anchor column where the change touches shared state; and it needs a fresh holdout per hill-climber.md §6 (check the unspent inventory first). Two things at once is two rounds.

9. Promote and record. A confirmed family becomes a /climber-gate row (its spec under this skill's specs/, generated by workload.py in the gate's generation block, a gate.py CELLS row with its seeded record) or a frontier sentinel when it is accepted residue; its mechanism goes to hill-climber.md §3 (a family) or §5 (a dated entry) with the class named; open directions go to §3 and §5. Write the report; leave the ledger current.

Directions carried from earlier rounds

What a round did not get to is written here rather than in its report, since a fresh round does not read prior reports. Strike a line when it is done or dead.

• Real-trace instances of shallowmoat's or absolve's shape (a large flat population, a far recency band, a scan or a pulsed short-reuse population): a climber-gate/marginal.py static anatomy over the corpus, or a real.py pass, would say whether a wide shallow valley with a prize behind it exists outside the generator (round 1, 2026-08-15; still open after round 2). Round 3's screen-level answer from the 77-cell start-knob screen (w1/w2/w5/w10/w20 plus the ceiling): no density-tier cell is flat to 20% with its ceiling far above at 50–70%; every recency-favorable cell rises monotonically from 1% and the nearest valley is arc_DS1@256k (a 0.1pp dip, a 0.5pp prize). Only a full anatomy (the 30–70% windows) can close it.

• The anchor-fidelity discrepancy on flat-zipf-plus-scan synthetics: the product's parked position earned ~4pp more than sketch.WindowTinyLfu at the same window on a reference curve flat within 0.5pp (b_scarburn, round 1), and the product held at its 2% floor earned +1.6pp over the reference's static 2% on the veil trace while three such cells beat every reference window by 1.2–3.8pp (round 2), so small gaps on that terrain must not be read to the half-point until a same-window comparison settles whether it is admission jitter, sketch parameters, or the reference's own aging.

• The masked-signal class beyond whisper: neither round's lanes constructed a main-side mirror … Closed in round 7 (2026-08-21): mainsat is the main-side mirror with an interior peak (the two-sided residency condition: the core stays out of the window at 80% and inside main at the peak, with a 52% scan holding the window's residency down), noaudit pins the floor 14pp under the dense-swept ceiling, and the equilibrium audit owns it as it owns whisper; rowed as a sentinel. No new class; the round-2 note on cheap keepalives stands for the window side.

• s_scarburst and s_flashpark are dose notes on recorded classes with open controls (a second trace seed, a neighborhood) (round 1). Still open after rounds 2–5. Round 6's parkveil supplied the class's independent second construction, period ladder and neighborhood, and family-ized it (the reactive-anchor phase-chase class); scarburst's own grid caveat stands on its row. The pair instrument was exercised in round 2 (s_alias_band/s_alias_scan): the pair's premise failed at the deciding field, window hits 30 against 45 around the bar of 32 at the arming sample, so no class-8 pair has yet survived a screen.

• Two dose ladders in round 2 flipped by the parent terrain's own lottery rather than by the knob (the veil share below 3e-4; the lure share at ~1.6%): a regenerated trace jitters a basin selector that sits within ±2 hits of the starvation bar, so read the ladder's dumps at the decision sample before calling a transition. workload.py cannot overlay a rider on a fixed base trace (every member draws from one RNG stream); a per-member seed would remove that confound.

• regret.py's position regret interpolates the static curve linearly between swept windows … a --windows option for a dense re-sweep is the follow-up (round 2). Done in round 3 (--windows; on demoflood the 15%/25% points move the position regret 6.44 → 5.91 and the residual −0.51 → +0.02).

• absolve was repaired 2026-08-17 (hill-climber.md §5): both gate rows moved (24.77 → 41.46, 27.95 → 46.29) by a repeat-confirm memory on the starvation ladder and a park's first audit following its walk; the period-16 witness at 256 samples reads 46.89 on 8 seeds and the family cells at 128 samples 43–44. Still open on the family: absolve_p20 (a 2-of-8 lottery either way) and the period-16 form's audit crash on the lure's off-step at other doses (three of the neighborhood's period-32 cells at 64 samples).

• The proposal lane on the session model (Fable) stalled in round 2 (four consecutive 96,000-token thinking turns with two tool rounds between them, then the output-token cap aborted it after ~2 hours; nothing written): the model, not the machine (continuous timestamps, no permission or classifier events); the blind lane fell back to Opus, which delivered in 14 minutes. The dispatch rule above now puts both lanes on Opus with a ~20-minute box. Rounds 3 and 4 ran the third-lane shape (three or four specs, each written to disk the moment it is designed, aimed at named lines) on Fable: three specs and predictions in nine and eight minutes, done before the screen; none survived either screen (round 3: f_ghostlure recorded, f_cornerloop peak-at-edge, f_crestshelf clean; round 4: f_dipref, f_crestcliff, f_probetax clean or beating the reference), so the shape works and its yield is a fair sample of two.

• hazefloor (round 3): … a sentinel row needs an x4 run at fixed seeds, per-seed bars, a 60/65% re-sweep Done in round 4: rowed at 320 samples (41.09 ± 1.80 at seeds 1–8; the re-sweep moved the ceiling to 52.27 @65%; reactive 36.5, noaudit 37.5; per-seed bars). Still open from that note: an arm with the crash-undo discard disabled, to say how much of the cycle is the discard and how much the corner audit's forced direction; a fix-side question.

• Two round-3 observations for the record rather than the round: after a regime shift on which everything works (the claim discarded, the audit walk confirms), the machine still spends ~27 samples in the valley (16 of audit wait after the crash-scale swing plus a nine-stride ×1 walk; ghostclaim's 16-sample-phase-1 neighborhood cells at n = 80: 10.6pp, all transient), the recovery latency's floor; and a walk armed in one phase of a large square-wave modulation crash-aborts against its own frozen base the moment the other phase returns, at the position that is right for that phase (s_farecho: the level test across a 20pp phase boundary; the recorded exit-bar threshold at an extreme dose).

• Class-8 pairs: round 3's s_pairup/s_pairdown held identical machine state through the decision by construction (a member at share 0.0 in the shared segment, only its reuse distance differing), so the pair instrument's premise can be met; the pair diverged at the shift sample itself (hr 0.30 vs 0.55) and the regret came from the stale claim rather than the shared direction bit, so no class-8 finding has yet survived a screen.

• Round 4's leftovers. b_causepair_b (a hot zipf phasing into main at s6 on a hold cell): the calibration audit's level-step confirm at 33% (absolve's calibration mechanism), the follow rule sending the park's audit up into a crash and back; the reactive law 48.7 against the machine's 42.9 ± 1.9 and noaudit's 47.0, so a reactive-anchor sentinel candidate of scarburst's kind (the hold-cell face); the spec is in the round-4 workspace, not rowed. s_fatbase (the far-crest face of crestpast: regimeramp's cold-reference misconfirm parking the window in a valley 21pp below the ceiling for 32 samples; 29.6 against 44.3 at 45%, LRU 37.2, deterministic) is the trend thread's largest measured dose and would make a sharper sentinel than regimeramp (whose priced cost is ~1.3) if that thread is worked; not rowed. The class-8 pair b_causepair_a/b met the pair premise by construction and failed it at the decision (+22pp against +6pp on the arm's first stride), the third pair in three rounds to do so. The arrival transient (a stride's sample under-reads the position just reached, so a walk's best sample is one to two strides past the static crest on most crestpast cells) bounds any walk's-best verdict and should be measured before one is built. The masked-signal main-side mirror is still untried after four rounds (the sighted lane's s_fatbase attempt at a probation poisoned before the arm did not produce it: no starvation probe armed). Three round-4 specs beat the reference on zipf-plus-scan terrain again (f_dipref −2.3, f_probetax −5.6, s_thrashundo −0.7): the anchor-fidelity note stands.

• Round 5's leftovers (the consolidation round, 2026-08-20; per-tenant instrument tenants.py). The dilute-valley family (h95±1 on the mixture terrain, class 3 reaching the audit walk's exit tests; witness deterministic at 8 seeds, hybrid 45.65 / noaudit 49.99 on the band tenant) awaits Ben's promotion call as a per-tenant sentinel bar. Unmapped: a d-ladder at fixed h95, the h-scan at 16384/d0.5 beyond h93/h95, K>2 tenants, and per-seed per-tenant readout on a bimodal instance (the second trace instance reads +2.29 with aggregate sd 0.25; tenants.py prints seed means). The admission-duel structural construction failed at 2–3% minority share (a uniform and a mild-zipf minority both admitted fine against a heavy zipf majority) — the class-0 consolidation story is unconstructed, and the masked-signal main-side mirror is still untried after five rounds. A weighted (byte) variant of any tenant family needs a weighted trace writer first; named residue, not scheduled. Dedicated-counterfactual arms above 0.5 of the minority's fraction carry warmup (n < 40); keep the load-bearing comparisons at fractions where n clears the bar.

• Round 7's leftovers. latebloom's fix-side questions are the latency face (a stand-down that re-arms the clock; a hold that expires with the shield), and its boundary is the catch window beyond density's ~19% rest (dreq·miss/max ≳ 0.19) for the alignments where the stand-down releases the park, with the held alignments losing at any distance above the floor's residency. mainsat's plants are the rail-return arrival discard (its second face); arrive is the measured arm and needs the plants on its benefit side before the C2 verdict is restated. The class-8 pair instrument has failed its premise at the deciding field five times in six rounds; a pair needs the divergent member invisible to hr, wh, mh and ema at the decision, which a ramped or boundary-introduced member never is, so the next pair must differ only in a reuse distance both beyond the current residency and inside the walk's reach, with the divergence opening during the walk. Class 7 still has no synthetic witness (s_recoil's post-undo deficit sat inside its scatter at one seed; run it seeded and longer). s_floorstraddle is a sharper calibration-park sentinel than regimeramp (13.8 against ~1.3) if that thread is worked; not rowed. b_slowfade's monotone-ramp attack on the rail needs an interior peak (size the lean core to leave room) before its bad_veto question can be asked. The dump file name carries no size, so eval --size N on a spec overwrites the spec's own-size dump (the 8192 s_floorstraddle dump was lost to its 32768 run); regenerate the geometry with mutate --set max= instead, which also keeps the sample count.

• Round 6's leftovers (2026-08-21). The masked-signal main-side mirror is now 0 for 8 across six fable rounds (the parallel Sol synthetic round's two deliberate MAIN mirrors also failed, 0 for 10 across the lanes), and this round's two failures both died at the terrain rather than the mask: s_mainmask's band was worth nothing at any window (its cell is a hold cell with LRU structurally above the whole curve) and b_mainwhisper's peak fled to the 80% edge while the mask itself worked (probes = 0 for the whole trace with main sighted — the first construction to get that far; the spec is in the round-6 workspace for an interior-peak retry). The three constraints — main dense, marginally worthless, and an interior-peak curve — fight each other through the admission filter, and the next attempt should budget the far prize's reuse distance against the miss-rate-scaled residency the round-6 report works out. Two sighted terrains put plain LRU 20+pp above the entire static W-TinyLfu curve (long-distance k=2 bands admission refuses; rep_r6's norank geometry at k=2) — check a screen row's LRU against its ceiling before reading its gap as climber regret. parkveil's 240-sample period ladder was not run (the 60-sample ladder plus the base's p13 record cover the axis for a sentinel, not for a fix); its promotion edits were staged uncommitted for Ben. s_valefloor is preserved as a fast-running class-1 witness (a compressed micro-valley at the floor, 17.99/17.76/23.61 at 1/3/5%, hybrid ≡ noaudit) if marginal steering (§5) is ever worked; the FIU mechanism itself (settling in the near basin) still has no synthetic reproduction — the machine overshoots instead.

• Round 9's leftovers (2026-08-31, opus-5). moatweather was not run against the real corpus: whether any real cell carries a level step of this size and a traverse long enough for it to land on a walk is the anti-correlation §3 already records, so the honest expectation is a constructed-only row, and the check (a real.py pass plus a scatter-versus-terrain scan) belongs before any repair. Its coincidence axis was measured at one period pair (24 at two phases) and its traverse axis on one terrain; whether the same dreq slope appears on shallowmoat or lowbar with weather added is the generalization. b_moatnoise's divergent attribution (there the calibration audit earns +1.7 and the later audits cost) was not chased to a responder. The class-8 pair failed a seventh time in a new place: s_twinloop_a/b met the premise at the trace (identical member sequences, byte-identical non-twin streams, both verified) and failed at the terrain, because a loop member is not resident when the window's capacity equals its cycle length, so both curves were monotone declines and both cells hold cells; a pair built on a residency threshold must measure the threshold on the generated trace rather than compute it. Class 7 is witness-less for the third round running (s_emberloss's capacity-refill construction did reach the top of the curve, but undo_deficit +1.06 sat inside an hr scatter of 3.54). Three blind cells were dropped peak-at-edge without a dense re-sweep, so one may still have an interior peak. Two process facts: a Fable proposal lane and a Fable evaluator both failed without writing anything, so an evaluator needs a completion check and a re-dispatch on another model rather than an open-ended wait; and the harness did not build at HEAD (the jspecify migration made System.getProperty(k, default) @Nullable at five injection sites), while harness.py strip cannot strip a tree applied by an older harness, so reset a worktree with git checkout -- . before re-applying an edited one.

• Round 8's leftovers (2026-08-30, fable). The class-8 pair failed its premise a sixth time, with the failure one layer down: the shared 2.2pp/stride drawdown never tripped the absolute 5pp abort, so both walks ran long enough for A's flipped band to be caught — a distance-flip pair that diverges during the walk diverges by being caught, which is visibility, so round 7's "divergence opening during the walk" requirement is self-defeating in that form. The next pair must first prove its shared abort fires on the common terrain (the drawdown dose is what failed), with the flipped band reachable only past the abort stride. The interior-peak bad_veto construction is 0-for-2 (b_slowfade round 7, s_creepveto round 8: the rail never fired; the trend feeds the confirm and the walk exit bars first). Class 7 is still witness-less (s_scorch repeated s_recoil's inside-scatter fate at gap 1.95). The s_probcap structural note (Belady 20.6pp over a flat curve) needs a geometry witness at cap-residency < D < LRU-residency (~10–11.7k at that terrain's rates) before the fixed-probation tax can be attributed past the sketch's ranking; LRU missed the band at both constructed doses. pricedshift (+0.63 at n=129, +1.80 with the capture tail zeroed at repeat×2 on pinnacle) has an unmeasured cost side on the families that need the discard to fire (ghostclaim, regimeramp, the moat rows). The arrival-transient verdict bound was witnessed twice in one round (settleflip's cold-start inversion; pinnacle's one-stride overshoot); if either family is promoted, the bound belongs in hill-climber.md as a recorded residual rather than a round-4 note.

Rules of evidence

• A claim about a particular decision needs a feasible paired continuation, not just a static ceiling gap. Follow §6's decision-replay protocol for attribution and retention studies. regret.py's trajectory hints and the canonical seeded-stream harness do not themselves establish equivalent decision state or request/contest pairing. Implement the needed hooks against the current source in the local experiment, verify prefix/randomness/observer controls, and retain useful workload specs and conclusions in the existing skills/design record. Reports, raw runs, build snapshots and experiment-specific runners belong in .local; preserving them is not integration into this workflow.
• A gap under 2.5pp needs eight seeded, paired runs; a bimodal cell is read seed by seed and never from a mean; seeded arms pair by seed (regret.py), unseeded comparisons rotate arms inside each run (gate.py), and neither compares across sweeps. Ladder rungs used to establish a classification threshold need the same eight admission seeds as classification cells. Read them seed by seed; a one-seed screen does not establish the boundary.
• A trace holds requests / (4 × max) decisions; classify nothing under 40 and re-check any finding at repeat ×2 before calling it steady state. Repetition restarts the segment-local modulation clock, so it can introduce a phase jump. For a single sine- or square-modulated segment, also lengthen samples with repeat=1 and verify that the original request prefix is unchanged. Keep restarted and continuous controls separate; recovery after a restart does not establish recovery under uninterrupted demand.
• Sign-uniform losses across a family are a mechanism, not noise, whatever their size, and a result that contradicts the round's hypothesis is written up as the result rather than reframed into one the data supports.
• Do not rebuild §5's graveyard, and do not reopen anything in it without a new mechanism; when a proposal resembles one, say which and what differs.
• Holdout discipline is §6's: freeze before tuning by LRU-only characterization (run.py --anchors --variants ''), spend once, and match the holdout to the change.
• One session's fix is presented before it is committed; docs (hill-climber.md, the gate table, rules/design-decisions.md) move in the same commit as the code.

Reporting

.local/audits/<model>/audit-regret.md, with the metadata header (Audit, Date, Commit) and, in this order: the screen table (label, size, n, LRU, start, ceiling@w, cache, gap, headroom, closed, missx, hint vote); one section per finding (the family and its knobs, the minimal witness spec, the transition, the class and secondary classes, the responder and owner, the arm table at eight seeds, the neighborhood surface, the nearest recorded family and the difference, the evaluator's objections and how each was answered, and the promotion status); the candidates dropped and why (dedup, validity gate, unreproduced); and what was not tried. Findings that turn out structural go in their own section addressed to the admission/sketch/SLRU owner. Everything that changed the record goes into hill-climber.md in the same session, and the ledger row is closed with the report's path.