Agent skill

Agent Definition Creator

by Margin-Lab in Margin-Lab/evals

Creates or updates Margin Eval agent definitions for new CLI coding agents.

AGPL-3.0Auto-check passedAgent Workflows

Install Agent Definition Creator

skills CLI
$ npx skills add Margin-Lab/evals --skill agent-definition-creator -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install Margin-Lab/evals agent-definition-creator --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/Margin-Lab/evals.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.agents/skills/agent-definition-creator .claude/skills/agent-definition-creator && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
agent-definition-creator
GitHub stars
161
Token cost
~2.4k tokens
SKILL.md length
1,084 words
Files
1
Skills in repo
3
Repo updated
First seen
Licence
AGPL-3.0

At a glance

Creates or updates Margin Eval agent definitions for new CLI coding agents.

  • Works in 7 steps: Choose the nearest template → Scaffold the directories → Design direct mode first → …
  • Codex needs to add support for a new agent
  • SKILL.md covers Preflight Checklist, Definition Components, Reference Files To Read and Workflow, plus 2 more sections
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

Agent Definition Creator is an agent skill from Margin-Lab/evals. Creates or updates Margin Eval agent definitions for new CLI coding agents. Use this skill when Codex needs to add support for a new agent, scaffold a directory under configs/agent-definitions/, define schemas and hooks, add example agent configs, or review an existing definition for missing auth, unified-mode, install, snapshot, or trajectory behavior.

Its SKILL.md is about 2.4k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Agent Workflows. The repository describes itself as: Fast, robust, configurable agent evals. The licence is AGPL-3.0.

When your agent uses it

  • Codex needs to add support for a new agent
  • Scaffold a directory under configs/agent-definitions/
  • Define schemas and hooks
  • Add example agent configs

Example prompts

  • “Use the agent-definition-creator skill to create or updates Margin Eval agent definitions for new CLI coding agents”
  • “/agent-definition-creator”

Requirements

  • Node.js

Workflow steps

7 steps, taken from the step headings in SKILL.md.

  1. Choose the nearest template
  2. Scaffold the directories
  3. Design direct mode first
  4. Write definition.toml
  5. Implement the hooks
  6. Add example configs
  7. Validate and smoke test

What it can do on your machine

Read from SKILL.md and the folder at commit b57dfe9. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md (its code samples are bash).

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Agent Definition Creator loads about 2.4k tokens when it runs. Until then it costs about 96 tokens; SKILL.md has 1,084 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~96
When it runs · the whole SKILL.md, loaded when a task matches
~2.4k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from Margin-Lab/evals at commit b57dfe9, republished under its AGPL-3.0 licence (© Margin-Lab). 1,084 words, ~2,392 tokens.

Download SKILL.mdSave it as .claude/skills/agent-definition-creator/SKILL.md (or your agent's skills folder).
name
agent-definition-creator
description
Creates or updates Margin Eval agent definitions for new CLI coding agents. Use this skill when Codex needs to add support for a new agent, scaffold a directory under `configs/agent-definitions/`, define schemas and hooks, add example agent configs, or review an existing definition for missing auth, unified-mode, install, snapshot, or trajectory behavior.

Agent Definition Creator

Create Margin agent definitions by collecting the missing runtime facts first, selecting the nearest existing definition pattern second, and only then writing definition.toml, schema.json, hooks, and example configs.

Preflight Checklist

Do not start writing files until these questions are answered from docs, --help output, installed CLI behavior, or user input:

  • Agent name, binary name, and install source
  • Version strategy: latest, exact version, semver range, or non-npm install
  • Auth model: single API key, local OAuth file, keychain entry, provider-qualified auth, or no auth
  • Direct config surface: the exact values the hooks need in config.input
  • Launch command: binary, args, env vars, working directory, and non-interactive flags
  • Structured output source: PTY-only, stdout JSONL, event stream, or session files on disk
  • Snapshot capability: whether the agent can resume or provide a lightweight snapshot command
  • Unified-mode mapping: whether shared model, reasoning_level, and mcp.servers[] can be translated cleanly
  • Skills and instruction-file behavior: skill home dir, AGENTS.md, CLAUDE.md, or some other filename
  • Toolchain/runtime needs for hooks and install: Node, Python, or other prerequisites
  • Semantic validation constraints beyond JSON Schema: provider/model coupling, enums, mutually dependent fields

If any item is unknown, investigate it before writing hooks. Most definition failures come from guessing auth, launch flags, or trajectory sources.

Definition Components

Every definition lives under:

text
configs/agent-definitions/<agent>/
├── definition.toml
├── schema.json
└── hooks/
    ├── install-check.*
    ├── install-run.*
    ├── run-prepare.*
    ├── translate-unified.*      # optional
    ├── validate-config.*        # optional
    ├── snapshot-prepare.*       # optional
    └── trajectory-collect.*     # optional

Required pieces:

  • definition.toml: declare auth, schema, hook paths, toolchains, and optional features
  • schema.json: validate the direct-mode [input] shape
  • hooks/install-check.*: report whether the agent is already installed
  • hooks/install-run.*: install the agent and return install metadata
  • hooks/run-prepare.*: write any runtime config files and return the launch exec spec

Optional pieces:

  • hooks/translate-unified.*: map shared unified config into direct input
  • hooks/validate-config.*: enforce semantic rules that JSON Schema cannot express
  • hooks/snapshot-prepare.*: enable POST /v1/run/snapshot
  • hooks/trajectory-collect.*: convert the agent's native logs or session data into ATIF

Common optional manifest sections:

  • [toolchains.node]: declare managed Node/npm for JS hooks or npm-installed CLIs
  • [auth.local_credentials]: support local OAuth or credential file discovery
  • [auth.provider_selection] and [[auth.providers]]: support provider-qualified auth selection
  • [skills]: tell agent-server where to materialize packaged skills inside run home
  • [agents_md]: tell agent-server which instruction filename to write into the project root
  • [config.unified]: advertise unified-mode translation and allowed values

Reference Files To Read

Read these repo files before creating or updating a definition:

  • docs/cli/add-support-for-a-new-agent/01-overview.md
  • agent-server/docs/design.md
  • agent-server/docs/unified-config.md
  • agent-server/docs/agent-config/*.md
  • agent-server/docs/plugins/commands-*.md
  • configs/agent-definitions/*/definition.toml
  • configs/example-agent-configs/*/config.toml

Workflow

1. Choose the nearest template

Do not start from a blank definition if a repo-owned definition already matches the agent's shape.

  • Use the Codex pattern for single-provider agents with a config file and resumable session files
  • Use the Claude Code pattern for single-provider agents with JSON settings and snapshot support
  • Use the Gemini CLI pattern for agents that emit a structured stdout event stream but do not support snapshots
  • Use the Opencode pattern for provider-qualified models plus config-file validation
  • Use the Pi pattern for provider-qualified models where reasoning maps directly to a native runtime flag
2. Scaffold the directories

Run:

bash
margin init agent-definition --definition ./configs/agent-definitions/<agent>
margin init agent-config --agent-config ./configs/example-agent-configs/<agent>-default --definition ./configs/agent-definitions/<agent>

If unified mode will be supported, also plan to add configs/example-agent-configs/<agent>-unified.

3. Design direct mode first

Design schema.json around the exact fields the hooks need, not around the shared unified format.

Prefer:

  • explicit scalar or enum fields when the agent already has stable CLI flags
  • string fields like settings_json, config_jsonc, or config_toml only when the agent truly consumes a raw config file
  • a dedicated provider field when auth or model resolution depends on provider selection

Keep the direct input minimal. Every field should be used by install, run, snapshot, or validation hooks.

4. Write definition.toml

Declare:

  • kind, name, description
  • auth mode
  • config schema path
  • hook paths
  • toolchains if needed
  • optional skills, instruction filename, unified mode, snapshot, and trajectory support

Rules:

  • declare [toolchains.node] whenever hooks are JS or install uses npm
  • only declare [snapshot] if the agent can actually support snapshot collection
  • only declare [config.unified] if translation is real, not aspirational
  • use [auth.provider_selection] when required env depends on provider
Show full SKILL.md (456 more words)Show less
5. Implement the hooks

All hooks:

  • read AGENT_CONTEXT_JSON
  • write only the expected JSON payload to stdout
  • treat stderr as logs

Implement them in this order:

  1. install-check Return installed status and any version details after probing the binary.
  2. install-run Install the requested version, probe again, and return structured install metadata.
  3. run-prepare Write config files into run home, set env vars, and return {path,args,env,dir}.
  4. validate-config if needed Reject semantic mismatches such as provider/model disagreement.
  5. translate-unified if supported Translate shared unified input into direct config.input.
  6. snapshot-prepare if supported Return the command used for snapshot capture.
  7. trajectory-collect if supported Convert native logs or session files into valid ATIF.
6. Add example configs

Create at least one direct config under configs/example-agent-configs/<agent>-default/.

Add a unified example only if:

  • the agent can map shared model
  • the agent can map or intentionally ignore shared reasoning_level
  • the translator can render unified MCP servers if the agent supports them
7. Validate and smoke test

Run a dry-run first:

bash
margin run \
  --suite ./suites/swe-minimal-test-suite \
  --agent-config ./configs/example-agent-configs/<agent>-default \
  --eval ./configs/example-eval-configs/default.toml \
  --dry-run

Then run a real smoke test if credentials are available and inspect the produced artifacts.

Gotchas

  • Do not mix unified and direct responsibilities. Install, run, snapshot, and trajectory hooks always consume resolved direct input.
  • Do not write logs to hook stdout. Any stray text breaks the JSON contract.
  • Do not guess auth precedence. Mirror the actual CLI's behavior for API keys versus local OAuth credentials.
  • Do not overfit direct config to one example profile. Keep the schema reusable across versions and models.
  • Do not claim snapshot support unless the agent has a real resumable or snapshot command.
  • Do not assume reasoning_level means the same thing across agents. Some translators map it directly, some render it into config, and some must ignore it.
  • Do not duplicate provider information in conflicting places. Add a validate hook when fields must agree.
  • Do not write runtime config files into the project root unless the agent requires that. Most belong in run home.
  • Do not forget skill and instruction-file integration. skills.home_rel_dir and agents_md.filename must match what the agent actually reads.
  • Do not leave version checks fuzzy. Install hooks should probe the installed binary after installation and report the resolved version.
  • Do not hardcode trajectory collection to stdout if the agent's real machine-readable history lives in session files or a tee'd artifact.
  • Do not expose unsupported MCP translation. Only translate MCP servers if the target CLI can actually consume them.

Finish Checklist

  • definition.toml matches the actual auth and capability model
  • schema.json matches direct-mode input
  • hook paths in definition.toml exist and are executable
  • example direct config exists
  • example unified config exists only if supported
  • dry-run passes
  • real smoke test passes if credentials are available
  • trajectory output validates if [trajectory] is declared
  • snapshot behavior works if [snapshot] is declared

© Margin-Lab, AGPL-3.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in .agents/skills/agent-definition-creator of Margin-Lab/evals.

Open the folder on GitHubat commit b57dfe9

Compare with similar skills

Agent Definition Creator next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Agent Definition Creator compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Agent Definition Creator this skillMargin-Lab/evals161—~2.4kAutomated safety check: PassAGPL-3.0
Claude Code Agent Developmentanthropics/claude-plugins-official38k8 repos~2.8kAutomated safety check: PassApache-2.0
Copilot Session Failure Analysisdotnet/maui23k—~3.4kAutomated safety check: PassMIT
Mem0 CLI Memory Commandsmem0ai/mem067k—~2kAutomated safety check: NotesApache-2.0
Install and Run Cogneetopoteretes/cognee32k1 repos~1kAutomated safety check: NotesApache-2.0
Create Agentvectorize-io/hindsight47k—~1.1kAutomated safety check: PassMIT

Similar skills

  • Claude Code Agent Development

    anthropics/claude-plugins-official

    Official

    Explains how to write agents for Claude Code plugins: the markdown file with YAML frontmatter, trigger descriptions, model and color settings, and system prompt design.

    38k GitHub starsUsed in 8 repos~2.8k tokens
    Agent WorkflowsAuto-check passed
  • Mines local Copilot CLI session logs for dotnet/maui to rank costly or failing runs, tag recurring failure modes, propose repo edits and emit guard evals.

    23k GitHub stars~3.4k tokensUpdated today
    Agent WorkflowsAuto-check passed
  • Adds, searches, lists, updates and deletes memories on the Mem0 platform from the terminal with the mem0 command, including a JSON mode built for agents.

    67k GitHub stars~2k tokensUpdated yesterday
    Agent WorkflowsAuto-check: notes
  • Install and Run Cognee

    topoteretes/cognee

    Installs the cognee AI memory library in a Python environment, sets the LLM key and gets a first remember and recall script running with the Python SDK.

    32k GitHub starsUsed in 1 repo~1k tokens
    Agent WorkflowsAuto-check: notes
  • Create Agent

    vectorize-io/hindsight

    Create a new Hindsight-powered subagent with long-term memory.

    47k GitHub stars~1.1k tokensUpdated today
    Agent WorkflowsAuto-check passed
  • Google Antigravity SDK

    google-antigravity/antigravity-sdk-python

    Design, implement, and debug autonomous AI agents and multi-agent systems using the Google Antigravity (AGY) SDK.

    3.7k GitHub stars~2.1k tokensUpdated yesterday
    Agent WorkflowsAuto-check: notes

More from Margin-Lab/evals

  • Suite Converter

    Margin-Lab/evals

    Converts test suites from external eval frameworks into the Margin Eval suite format.

    161 GitHub stars~3.7k tokensUpdated 2 mo ago
    Auto-check passed
  • Suite Creator

    Margin-Lab/evals

    Creates new Margin Eval test suites from scratch. An agent skill from Margin-Lab/evals.

    161 GitHub stars~2.3k tokensUpdated 2 mo ago
    Auto-check passed

Questions about Agent Definition Creator

What does Agent Definition Creator do?

Creates or updates Margin Eval agent definitions for new CLI coding agents. Agent Definition Creator is an agent skill from Margin-Lab/evals. Creates or updates Margin Eval agent definitions for new CLI coding agents.

When should I use Agent Definition Creator?

Agent Definition Creator fits situations like: Codex needs to add support for a new agent; scaffold a directory under configs/agent-definitions/; define schemas and hooks; add example agent configs.

How do I install Agent Definition Creator in Claude Code?

Run `npx skills add Margin-Lab/evals --skill agent-definition-creator -a claude-code`. Or copy the skill folder (.agents/skills/agent-definition-creator in Margin-Lab/evals) into .claude/skills/agent-definition-creator in your project. Claude Code loads it when a task matches its description.

How do I install Agent Definition Creator in Codex?

Run `npx skills add Margin-Lab/evals --skill agent-definition-creator -a codex`. Or copy the skill folder (.agents/skills/agent-definition-creator in Margin-Lab/evals) into .agents/skills/agent-definition-creator in your project. Codex loads it when a task matches its description.

Can I use Agent Definition Creator in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add Margin-Lab/evals --skill agent-definition-creator -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/agent-definition-creator, .gemini/skills/agent-definition-creator, .github/skills/agent-definition-creator and .opencode/skills/agent-definition-creator in your project.

What does Agent Definition Creator need to run?

SKILL.md names no scripts, command-line tools or credentials: Agent Definition Creator is instructions for the agent only. Our summary lists: Node.js.

Does Agent Definition Creator access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Agent Definition Creator safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Agent Definition Creator use?

Agent Definition Creator is published under the AGPL-3.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Agent Definition Creator use?

About 2.4k tokens (SKILL.md is roughly 9.6k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Agent Definition Creator?

Skills that share tags, products or a category with Agent Definition Creator: Claude Code Agent Development (anthropics/claude-plugins-official, 38k stars), Copilot Session Failure Analysis (dotnet/maui, 23k stars), Mem0 CLI Memory Commands (mem0ai/mem0, 67k stars) and Install and Run Cognee (topoteretes/cognee, 32k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Agent Definition Creator?

Margin-Lab (a GitHub organization) maintains it in Margin-Lab/evals, which has 161 GitHub stars. The repository holds 3 skills in this directory. The repository was last updated on July 31, 2026.

Source: Margin-Lab/evals on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.