--- name: optimize-nethack-policy description: Improve an EvoPolicyGym Policy Program for the deterministic NLE NetHackScore Benchmark. --- # Optimize an NLE NetHack Policy Maximize mean shaped return in deterministic but hidden NetHackScore Episodes. The task uses NLE 1.3.0, NetHack 3.6.7, a fixed neutral human male Monk, the standard 23-action task set, and at most 5,000 Policy steps. Each Policy failure receives a large negative return, so validity and termination come first. ## Respect the ABI - Export `make_policy(context)` from `policy.py` and return an object with `act(observation)`. - Return an exact integer from 0 through 22. Booleans, floats, containers, and out-of-range integers are invalid; the Benchmark never repairs an Action. - A fresh Policy process and instance are created for every Episode. Keep map, inventory, prompt, and plan state only between `act()` calls in that Episode. - Read public static configuration from `context.environment_parameters`. Never infer, request, or search for Environment seeds or Host paths. ## Actions ```text 0 more 8 northwest 16 run_northwest 1 north 9 run_north 17 up 2 east 10 run_east 18 down 3 south 11 run_south 19 wait 4 west 12 run_west 20 kick 5 northeast 13 run_northeast 21 eat 6 southeast 14 run_southeast 22 search 7 southwest 15 run_southwest ``` Actions are NLE keyboard inputs. In a prompt, the same raw key can mean an answer rather than movement. For example Actions 1 through 8 emit `k l j h u n b y`; Action 21 emits `e`, Action 22 emits `s`, and Action 0 emits Enter. Inspect `message` and `input_mode` before interpreting an Action as an ordinary command. ## Observation The observation is a dictionary: - `screen.glyphs`: `int16` `TensorValue`, shape `(21, 79)`; - `screen.chars`: `uint8` `TensorValue`, shape `(21, 79)`, row-major visible character codes; - `screen.colors`: `uint8` `TensorValue`, shape `(21, 79)`; - `stats`: named public NLE blstats, including `x`, `y`, `score`, HP, depth, gold, experience, turn, hunger, encumbrance, dungeon level, and conditions; - `message`: the current public NetHack message; - `inventory`: only populated entries with `letter`, `description`, `glyph`, and `object_class`; - `input_mode`: `normal`, `yes_no`, `get_line`, or `more`. The Policy does not receive NLE's privileged `internal` array, raw seeds, ttyrec paths, rewards, or evaluator state. Decode tensors from `TensorValue.data`; do not import or reach into the trusted NLE Environment. ## Build an Episode-local controller Use explicit layers that can be tested separately: 1. **Prompt handler:** respond to ``, direction, eating, and yes/no questions before normal navigation. A prompt that is mistaken for movement commonly creates frozen-step penalties or loops. 2. **Map memory:** combine visible characters, colors, glyphs, current position, and dungeon level. Track visited, blocked, dangerous, door, corridor, staircase, item, and monster cells without assuming a hidden seed route. 3. **Immediate safety:** monitor HP, hunger, conditions, nearby monsters, and escape space. Do not start a long run or repeated search while threatened. 4. **Exploration:** prefer reachable low-visit frontiers, search near plausible dead ends, handle doors deliberately, and avoid immediate reversal loops. 5. **Resource state:** track public inventory descriptions and observed outcome messages. Initiating `eat` and selecting an inventory letter are separate decisions. 6. **Progress plan:** descend when prepared, collect useful public items, and balance score gain against survival and frozen-step penalties. The standard 23-action profile is deliberately narrower than a full NetHack keyboard. Do not write plans that require unavailable commands such as wield, wear, quaff, read, open, or arbitrary inventory letters. ## Optimize the actual metric The scalar score is mean Episode return from `NetHackScore-v0`: NetHack score deltas plus a `-0.01` penalty when repeated Environment steps do not advance game time. `mean_game_score`, depth, deaths, ascensions, truncations, and Policy failures are secondary diagnostics. A Policy failure receives `-max_episode_steps`, not the partial return accumulated before failure. `frozen_steps`, `mean_frozen_steps`, `frozen_step_fraction`, and `mean_frozen_penalty` expose the exact aggregate contribution of unchanged-turn Actions to this score; they do not select or summarize trajectory content. Small batches have high procedural variance. First eliminate failures and frozen prompt loops, then compare unchanged candidates on equal larger batches. Use 1--4 Episodes per search submission until memory and runtime are measured. ## Use Feedback safely Training Feedback includes aggregate scores plus complete raw evidence for all Episodes in that submission. Read `artifact-manifest.json`, then use its ordinals to align every `trajectory-*.jsonl.gz` transition with the exact Policy-visible observations in `observations-*.npz`. Open NPZ files with `numpy.load(..., allow_pickle=False)`. No Environment or Host code has already selected frames, created screenshots, rendered videos, or decided which segments are important. Choose your own analysis. You may decode terminal `chars` and `colors`, inspect glyphs or named stats, render selected observations with Pillow, export a segment with ImageIO, compare messages and inventory, or use another derived representation. Keep non-submitted scripts, selected frames, derived images, and notes under `analysis/`; only `program/` is submitted. Bulk evidence from older submissions can be evicted, while the newest complete submission is protected, so copy only the derived material you decide is useful before a later submission. Validation and Assessment are Host-only aggregate phases and publish no detailed Artifacts back to the workspace. Never attempt to recover hidden seeds, Host paths, ttyrec data, or privileged simulator state from Feedback. Treat small-batch score movement as noisy and use equal Episode counts when comparing candidates. Before `finish`, compare the candidate Program digests and the actual source changes you intended. Submit behaviorally distinct candidates that test real strategy alternatives. Renaming variables, retaining unused branches, or passing a flag that does not change reachable behavior wastes a Validation slot even though it creates a different source digest. Exact aggregate ties on the same hidden Validation Episodes are a reason to inspect candidate distinctness, not evidence that the Host should inspect or rewrite Programs. ## Iterate in stages 1. Submit the packaged baseline on a small batch and inspect the manifest. 2. Select and decode the observations or trajectory segments needed to explain failures and repeated behavior. 3. Make prompt handling total and eliminate invalid Actions/timeouts. 4. Add persistent dungeon-level map memory and loop detection. 5. Improve frontier choice, door/search behavior, and staircase handling. 6. Add HP, hunger, inventory, monster, and condition-aware interrupts. 7. Remove behaviorally redundant candidates, then compare promising distinct candidates with equal Validation budgets before final handoff.