Ax Playbook

Agents

a.playbook({ target }) returns an agent-aware AxAgentPlaybook (the stage AxPlaybook handle plus an agent-level evolve). The one playbook the agent renders into its prompt grows three ways:

• Continuous (trust): the construction-time playbook option (see ax-agent) harvests each run's failures automatically — no dataset.
• On-demand (trust): apb.update({ example, prediction, feedback }).
• Batch verified (proof): apb.evolve(dataset, options) runs the full agent over a task set, mines failure clusters, and proposes one playbook bullet per weakness; with verify (default on) it keeps a bullet only if held-in improves AND the validation held-out set does not regress, else exact rollback. verify: false = trust-batch. Bullets-only.

const a = agent('ticket:string -> reply:string', { ai });
const apb = a.playbook({ target: 'actor' }); // agent-aware handle; 'actor' (default) or 'responder'
await apb.update({ example, prediction, feedback }); // online: injected into the live stage prompt
const result = await apb.evolve(
  { train, validation }, // AxAgentEvalDataset
  { metric, runsPerTask: 2 }, // verify:true by default
);

The agent-level evolve(dataset, options) is distinct from the program-level pb.evolve(examples, metric) above: it takes an AxAgentEvalDataset plus options, runs the whole pipeline, and returns baseline/final held-in & held-out with per-bullet outcomes (no { bestScore }). For full-pipeline tuning of agent instructions and demos (not the playbook) use agent.optimize(...) (GEPA).

Generated packages expose that same agent-bound loop with language-shaped APIs:

All five generated packages thread structured failureSignals through agent evaluation predictions. The default verify gate accepts a proposed bullet only when held-in score improves and held-out score stays within epsilon; rejection restores the exact prior snapshot. Scoring is host-shaped: TypeScript uses its metric, Python/Java/Go can accept a metric callback, and all generated ports can use task score/scores values plus the agent evaluation result.

Playbook vs optimize()

• playbook(...) — accumulate reusable, evolving task knowledge; the only path that also learns online via update(...).
• optimize(...) / agent.optimize(...) — tune instruction text and few-shot demos offline to a best/Pareto result.
• They are complementary; a project can use both.

Troubleshooting

• "Cannot convert undefined or null to object" from update() → you passed input fields at the top level; wrap them in example: { ... }.
• Empty playbook after evolve() → the model already scored well, so nothing was curated; use harder/ambiguous examples or a weaker studentAI to surface lessons.
• Playbook not affecting an agent's behavior → ensure apply is not false and you used agent.playbook(...) (not a bare playbook() on an internal program).

See Also

• ax-gepa - optimize(...) and AxGEPA for instruction/demo tuning.
• ax-agent-context - choosing between contextMap, contextPolicy, agent.playbook(...), and recall.
• ax-agent-optimize - agent.optimize(...) GEPA tuning for agents.