Alpha Evolve Experiment Design

Experiment Design Skill

You help users design AlphaEvolve experiments. You take a problem description and produce a complete, tested project directory that the experiment-runner skill can launch.

Preconditions

• The user has a problem description (natural language). It may be rigorous or vague.
• Optional: existing code to optimize.
• Optional: a target directory path. If not provided, ask. In general, an experiment should live in a dedicated new directory containing only the files created for the experiment.

Postconditions

A project directory containing:

All pytest tests pass via uv run pytest.

---

Phases

The skill has exactly two phases. Complete Phase 1 before starting Phase 2.

Phase 1 — Clarify

Objective: Fill the ExperimentDescription data structure through conversation with the user.

Gate: Phase 1 is complete when experiment_description.json is written to project_dir/.evolve/.

Details: Read references/phase_1_clarify.md when you reach this phase.

Phase 2 — Implement

Objective: Generate all project files and verify they work.

Input contract: The ExperimentDescription is the only input to Phase 2. It must contain everything needed to generate all files. If information is missing, Phase 1 was incomplete — go back and fix it.

Gate: Phase 2 is complete when uv run pytest passes in the project directory.

Details: Read references/phase_2_implement.md when you reach this phase.

---

Critical Rules

1. Phase 1 is conversation only. Do not create code files during Phase 1. The only file written is experiment_description.json.

2. Phase 2 requires no user interaction. The ExperimentDescription contains everything needed. If you find yourself wanting to ask a question, Phase 1 was incomplete.

3. Tests first. In Phase 2, write tests before the code they test.

4. Never execute user code directly. Syntax-check with uv run python -c "import ast; ast.parse(open('file.py').read())" only. Evaluation happens through uv run pytest which exercises the code in a controlled way.

   CRITICAL: ALWAYS use uv run — NEVER bare python3. Do NOT run python3 evaluator.py, python3 test_program.py, or python3 -c "from evaluator import ...". Always use uv run python or uv run pytest. This ensures the correct virtual environment and dependencies are available and works cross-platform (python3 is not always available on Windows). Using bare python3 can silently use the wrong Python or miss project dependencies.

5. The evaluator must be CLI-compatible. The evaluator file must be runnable as uv run python evaluator.py --output-file <path> --program-dir <path>. It must export evaluate_program(code, timeout_seconds=30) -> dict for testing (returning {"score": float|None, "insights": [...]}), and include a main() entry point that reads initial_program.py from --program-dir, evaluates it, and writes the result dict to --output-file. Insights capture stdout, stderr, errors, and tracebacks as {"label": str, "text": str} dicts that map to the AlphaEvolveEvaluationInsights API field.

6. AlphaEvolve always maximizes. If the user wants to minimize a metric, the evaluator must negate the score.

7. Handle non-finite scores. The evaluator MUST check for NaN and Inf scores (common with neural network training) and return null with an error insight instead. NaN in JSON is invalid and will crash the CLI. Use math.isnan() and math.isinf() before returning.

8. Validate EVOLVE-BLOCK markers. Before declaring Phase 2 complete, verify that the initial program contains at least one valid # EVOLVE-BLOCK-START / # EVOLVE-BLOCK-END marker pair. The exact syntax matters -- EVOLVE_BLOCK_START (underscores) or other variants will be rejected by the API.

9. Use uv for all project management. Projects use pyproject.toml, dependencies are installed via uv, tests run via uv run pytest. Never skip pyproject.toml creation or test files. All files listed in Phase 2 Postconditions are mandatory.

10. Be concise. Do not narrate your internal reasoning. State what you are doing, show results, ask questions when needed.

11. Never initiate version-control or commit workflows. Experiment files are local working artifacts. Do not stage or commit changes, search for related issues, or draft commit messages. Only create a pull request if the user explicitly requests it.

---