Optimize Loop

The loop (until plateau or <budget>)

Copy this checklist and tick items off:

• Iteration 0 — baseline: run <gate_cmd> (code) — if not green, stop (the loop needs a passing gate to protect behaviour). Run <metric_cmd>; record the metric as the current best, and in sql mode record the baseline hash as the correctness reference. Log the baseline row.
• Snapshot every file in <editable_files> to <sandbox_root>/iter<N>/ so the iteration reverts.
• Apply one focused change (see the per-mode ideas below) — one idea per iteration so each metric delta is attributable.
• Check the gate: code — run <gate_cmd>; sql — read the candidate's hash from <metric_cmd>.
• If the gate fails (tests non-zero / hash ≠ baseline / the tool errored), discard: restore from the snapshot, log the reason, continue.
• Measure the metric and compare to the best: code — the triple (complexity, max_nesting, loc) lexicographically (complexity first; only on a tie consult max_nesting, then loc); sql — median_ms, keeping only on a margin clear of timing noise (default ≥ 3% relative).
• Keep if the metric strictly improves the best (update the best, leave the files in place), else discard (restore from the snapshot).
• Append a ledger row; stop on plateau (<patience>) or <budget>.

Change ideas — code mode: flatten nested if/else into guard clauses, replace a hand-rolled loop with a stdlib call (sum, min, max, statistics.*), collapse duplicated branches, remove dead code. Preserve public behaviour — names, signatures, return shapes, raised exceptions; the test suite is the contract.

Change ideas — sql mode: add an index to <indexes_file> covering filtered/joined/grouped columns; rewrite the query (correlated subquery → JOIN + GROUP BY, hoist a repeated computation, replace SELECT * with needed columns, push a filter earlier, drop a redundant DISTINCT/sort). The hash is over the multiset of rows, so it does not catch a changed row order — if ORDER BY is part of the contract, eyeball that the rewrite preserves it.

Lexicographic keep (code mode), current best (18, 3, 64): (15, 3, 45) keep (lower complexity); (18, 2, 70) keep (tie complexity, lower nesting); (18, 3, 61) keep (tie, fewer lines); (18, 3, 64) discard (no progress); (19, 1, 20) discard (higher complexity outweighs simpler nesting/loc).

Plateau counting: increment the no-improvement counter on every iteration that does not set a new best — discarded for a failed gate, a broken change, or an insufficient metric gain — and reset it to 0 on each keep. <patience> fruitless iterations in a row ends the run; <budget> is the hard cap. On stop, restore the working files to the best iteration (if the latest was a discard) and report: baseline vs best metric (and, in sql mode, the speedup factor), the trajectory, and the winning change set. If you run low on ideas before the budget, look harder rather than stopping early.

Ledger

<sandbox_root>/ledger.tsv, tab-separated, never commas in the description. status ∈ {keep, discard, baseline}. Use the columns for the active <mode>.

Code mode header iter complexity max_nesting loc status description:

iter	complexity	max_nesting	loc	status	description
0	23	7	86	baseline	unmodified module
1	19	5	78	keep	flatten summarize guard clauses
2	19	5	80	discard	extract helper (no complexity gain)
3	13	3	40	keep	use statistics + min/max/median

SQL mode header iter median_ms rows hash_ok status description (hash_ok ∈ {yes,no,-}):

iter	median_ms	rows	hash_ok	status	description
0	1121.06	10	-	baseline	correlated subquery no index
1	6.82	10	yes	keep	rewrite correlated subquery as JOIN + GROUP BY
2	1.18	10	yes	keep	add index orders(customer_id, amount)
4	0.40	10	no	discard	drop ORDER BY — changed result set

Report the best iteration, not necessarily the last.

Constraints

• Only edit files in <editable_files> — the gate's ground truth is read-only: the tests and tools/metrics.py (code), the database and tools/bench.py (sql). Editing what measures you to move the number invalidates the run.
• The correctness gate is non-negotiable. A change that fails it (tests red, or hash ≠ baseline) is a regression, not an optimization — revert it regardless of the metric. A green gate after a behaviour change means the gate is too weak, not that the change is safe; prefer holding behaviour identical over trusting a thin gate.
• One change per iteration, so each metric delta is attributable.
• Measure every candidate the same way (code: the same <gate_cmd>; sql: the same --repeat); compare the metric, not a single noisy run.
• Keep changes within the existing dependency set; do not add imports the project lacks (stdlib is fine).
• The sandbox is self-contained — no ../ escapes beyond the bound <sandbox_root>.
• Do not pause the loop to ask whether to continue; run until plateau or budget.