Optimize Balatro Policy

Optimize Balatro Policy

Build a coherent Policy system that can win complete runs. Treat Balatro strategy, software architecture, replay testing, and noisy evaluation as one engineering problem. Prefer a small shared decision model over disconnected phase heuristics, tooltip tier lists, and encounter-specific patches.

Load the reusable resources

• Before planning or interpreting an evaluation, read references/experiment-protocol.md.
• Before choosing the next capability, use the failure-signature routing and experiment cards in references/experiment-catalog.md.
• Before editing program/, read the applicable implementation pattern in references/implementation-playbook.md.
• Before extracting, splitting, merging, or rewiring Policy modules, read references/modularity-guide.md.
• Before changing a prediction, value, planning, or selection model, localize the lowest incorrect layer with references/modeling-stack.md.
• Before changing scoring, discard, shop, build, or Joker-order behavior, read the relevant part of references/strategy-lessons.md.
• Use scripts/summarize_evidence.py to pool public submission Feedback by immutable digest. Do not hand-calculate repeated evidence summaries.
• Use scripts/compare_indexed_feedback.py to compare separate control and candidate Feedback files. It refuses unmatched Episode-index sets or repeat counts.

Align with the objective

• Distinguish Benchmark reward from in-game Chips and dollars. Optimize expected Episode score: Blinds actually cleared, doubled on a complete-run win.
• Keep win reward and ordinary progress separate. In run-score-v3, clearing 24 Blinds and winning scores 48. Skipped Blinds do not count. Read explicit won and rounds_cleared diagnostics; reward alone cannot identify a win.
• Treat Ante reach and Blinds cleared as development diagnostics, not substitutes for winning. Prefer changes that increase robust Ante 8 completion probability over changes that only improve early progress.
• Group evidence by Program digest and pool all Episodes for that digest. A single win is high-variance evidence, not proof that one observed build or purchase caused it.
• Record wins, Episodes, Policy failures, mean/median Blinds, early deaths, mid/late tails, and Ante reach separately. Never compare only the largest submission mean. Results for the same selected Episode indices are matched; results for different index sets remain unmatched noisy evidence.

Preserve the boundary

• Edit only program/; treat feedback/ as read-only evidence.
• Use only the Benchmark specification, public Feedback and Artifacts, the current observation, and legal_actions.
• Do not inspect or import Jackdaw, Benchmark, Host, seed, or hidden Case internals even when local process execution makes them reachable.
• Never predict a hidden seed, future draw, shop roll, or private Case identity.
• Re-read every entity and hand index from the current observation. Indices are ephemeral.
• Keep learning outside Episodes. Retain only Episode-local intent or plan state in the Policy instance; never update persistent parameters in act().

Establish evidence first

1. Read the Host task's Run-local Episode-index range, Episode budget, submission limits, and candidate limit. Before the first result, partition both train indices and spend into baseline, diagnosis, matched capability development, and frozen confirmation roles.
2. Inspect every file under program/ and the permitted public train Feedback and advertised Artifacts. Record the unchanged Program digest and preserve an exact restore method.
3. Submit the smallest useful baseline batch to establish correctness, distribution, failures, Ante reach, and wins.
4. Keep an evidence ledger with digest, submission IDs, selected Episode indices, preassigned role, Episode count, mean/median Blinds, ≤5, ≥12, ≥18, failures, wins, hypothesis, and decision. Keep the ledger in working reasoning; do not add non-Policy evidence files to the submitted program/.
5. Inspect replay trajectories and omission markers only for baseline, diagnosis, and capability-development submissions. Identify the first consequential bad decision in representative weak, strong, and failed Episodes.
6. Enter frozen confirmation only once. Stop editing strategic behavior, evaluate the frozen shortlist on pre-reserved, previously unseen train indices, and inspect aggregate outcomes only.

All Agent-visible submissions are train evidence from one fixed Host-owned Episode pool. The Agent can choose and reuse Run-local Episode indices but cannot observe their actual Environment or Policy seeds. Reusing an index preserves its hidden Episode specification and Policy seed while creating a fresh Environment and Policy runtime and consuming budget again. Use identical index selectors for matched online A/B, and treat comparisons over different selectors as unmatched. Follow the complete index-partition, budget, and freeze procedure in the experiment protocol.

Build a Policy system

After the unchanged baseline and first minimal tactical correction, establish the Bot-system boundaries before adding a second strategic capability. Do not keep extending a monolithic policy.py. Keep policy.py:make_policy as the small ABI and composition entrypoint, and use the whole program/ directory:

policy.py          ABI adapter and composition only
policy_system/
  state.py         normalized StateView and EpisodePlan
  actions.py       legal catalog, Action construction, exact admission
  hands.py         hand classification and candidate enumeration
  scoring.py       visible scoring model and approximation confidence
  draws.py         outs, draw probability, and discard value
  effects.py       structured visible-effect roles
  planning.py      build, economy, Boss, and horizon model
  strategy.py      phase candidates and final decision
tests/
  test_replays.py  persistent public replay regression

Treat this tree as a responsibility map, not a mandatory file list. Adapt names and merge adjacent small modules according to the ownership and dependency rules in the modularity guide, but preserve these enforceable boundaries:

• Concentrate raw dictionary access and nullable-field normalization in one observation adapter.
• Keep an explicit Episode plan containing the current primary and secondary hand plans, effect roles, economy phase, scaling trajectory, Boss constraints, and confidence. Update only Episode-local state.
• Make hand evaluation and card-effect evaluation pure wherever practical.
• Make every phase planner return an intent or ranked candidate, not an unchecked Action object.
• Construct and admit every emitted Action through one legal-action gateway.
• Use one shared value model for playing hands, discarding, buying, selling, opening packs, using consumables, and planning the build.
• Represent unknown effects explicitly and discount confidence instead of silently treating them as zero or inventing behavior.
• Persist the replay harness and regression fixtures under program/; do not leave essential tests only in one-off shell snippets.
• Give every responsibility one authoritative owner and keep dependencies acyclic toward normalized facts and pure models. Do not preserve parallel scorers, parsers, tier lists, or Action builders after migration.

Use this decision pipeline:

observation -> StateView + LegalCatalog
            -> MechanicsSnapshot + EpisodePlan
            -> OutcomeEstimate -> ActionValue
            -> ranked Intent -> ActionGateway -> Action

Treat the system checkpoint as required work, even though refactoring alone does not increase reward. Keep modular refactors separate from strategic experiments. Validate intent and Action equivalence on public replays, then use a small matched submission only when local evidence cannot cover runtime behavior. Do not defer structural work until the final evidence phase.

Enforce the Action contract

• Index legal_actions by kind on every call.
• Construct exactly the fields described for the selected kind; never emit descriptor metadata as Action fields.
• Validate target indices against the current descriptor, not only against the corresponding observation list.
• Read nested targets for use_consumable and pick_pack_card, including each target's allowed card_indices, minimum, and maximum.
• Preserve selected-card order when selection_order_matters is true.
• If the preferred intent is unavailable, rank a different currently legal intent. Never guess, repair, or reuse a stale index.
• Treat any invalid_action, exception, timeout, or protocol failure as a release-blocking correctness defect, not a bad score.

Model decisions consistently

Build a reusable model of visible game mechanics, not a collection of encounter-name patches. Parse structured rule parameters into effect roles and evaluate actions as changes to survival probability, build strength, economy, and win probability. Use localized text handling only when structured public data cannot express an encountered rule. Diagnose the earliest wrong model layer and fix it there; do not compensate for a mechanics or prediction defect with an upper-layer threshold.

Hands and draws

• Enumerate legal one-to-five-card plays and classify their actual hand types.
• Start from visible poker_hands Chips and Mult, then account for card Chips, enhancements, editions, seals, debuffs, played-card order, and modeled Joker effects.
• Carry an approximation confidence when an effect cannot be modeled exactly.
• Estimate outs and draw probabilities from visible remaining-deck counts. Compare playing now with discarding by expected post-draw value, remaining hands and discards, current build activation, and Blind target pressure.
• Let the established build alter hand value. Do not rank a level-one Straight above a highly upgraded core hand merely by poker category.
• Compare predicted hand score with actual last_hand breakdowns and turn systematic error into effect-model regressions.
• Treat two hidden cards with absent rank or suit as two unknowns, never as a known Pair or Flush relation.
• Test the downstream selector whenever calibration changes. A more accurate immediate score can reduce survival when the play/discard horizon remains myopic.

Effects and build

• Prefer structured rule.parameters and mutable ability values over text. Use rule.summary as semantic evidence or a localized fallback, not as a global substring-based tier list.
• Classify visible effects into roles such as base Chips, additive Mult, multiplicative Mult, retrigger, scaling, economy, hand-specific synergy, deck shaping, and defense.
• Track a primary and optional secondary hand plan, covered scoring roles, scaling trajectory, activation reliability, and switching cost.
• Derive purchase, replacement, pack, consumable, and ordering values from the same effect and build model used by the hand scorer. Do not maintain a disconnected shop tier list.
• Model temporary, scaling, conditional, consumable, and decaying effects differently. Evaluate Joker and played-card ordering when resolution order matters.
• Separate scored-card, held-card, main additive-Mult, main XMult, retrigger, and copy phases. Do not apply a generic Joker sort across unresolved copy dependencies.