CodeGraph Agent Eval
colbymchenry/codegraph
Benchmarks how much CodeGraph helps a coding agent on a real repository, comparing runs with and without it for a chosen local or published version.
Run and manage CORAL experiments from the operator side — launch agents with coral start (dotlist overrides, model/count, tmux vs local), monitor with coral status / coral log / coral show / the web…
Install Running Coral Experiments
$ npx skills add Human-Agent-Society/CORAL --skill running-coral-experiments -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install Human-Agent-Society/CORAL running-coral-experiments --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/Human-Agent-Society/CORAL.git skills-src && mkdir -p .claude/skills && cp -r skills-src/plugin/skills/running-coral-experiments .claude/skills/running-coral-experiments && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "running-coral-experiments" agent skill from https://github.com/Human-Agent-Society/CORAL/tree/main/plugin/skills/running-coral-experiments into .claude/skills/running-coral-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "running-coral-experiments", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/Human-Agent-Society/CORAL/tree/main/plugin/skills/running-coral-experimentsType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add Human-Agent-Society/CORAL --skill running-coral-experiments -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install Human-Agent-Society/CORAL running-coral-experiments --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Human-Agent-Society/CORAL.git skills-src && mkdir -p .agents/skills && cp -r skills-src/plugin/skills/running-coral-experiments .agents/skills/running-coral-experiments && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "running-coral-experiments" agent skill from https://github.com/Human-Agent-Society/CORAL/tree/main/plugin/skills/running-coral-experiments into .agents/skills/running-coral-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "running-coral-experiments", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add Human-Agent-Society/CORAL --skill running-coral-experiments -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install Human-Agent-Society/CORAL running-coral-experiments --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Human-Agent-Society/CORAL.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/plugin/skills/running-coral-experiments .cursor/skills/running-coral-experiments && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "running-coral-experiments" agent skill from https://github.com/Human-Agent-Society/CORAL/tree/main/plugin/skills/running-coral-experiments into .cursor/skills/running-coral-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "running-coral-experiments", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/Human-Agent-Society/CORAL.git --path plugin/skills/running-coral-experiments--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add Human-Agent-Society/CORAL --skill running-coral-experiments -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install Human-Agent-Society/CORAL running-coral-experiments --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Human-Agent-Society/CORAL.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/plugin/skills/running-coral-experiments .gemini/skills/running-coral-experiments && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "running-coral-experiments" agent skill from https://github.com/Human-Agent-Society/CORAL/tree/main/plugin/skills/running-coral-experiments into .gemini/skills/running-coral-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "running-coral-experiments", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install Human-Agent-Society/CORAL running-coral-experimentsInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add Human-Agent-Society/CORAL --skill running-coral-experiments -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/Human-Agent-Society/CORAL.git skills-src && mkdir -p .github/skills && cp -r skills-src/plugin/skills/running-coral-experiments .github/skills/running-coral-experiments && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "running-coral-experiments" agent skill from https://github.com/Human-Agent-Society/CORAL/tree/main/plugin/skills/running-coral-experiments into .github/skills/running-coral-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "running-coral-experiments", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add Human-Agent-Society/CORAL --skill running-coral-experiments -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install Human-Agent-Society/CORAL running-coral-experiments --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/Human-Agent-Society/CORAL.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/plugin/skills/running-coral-experiments .opencode/skills/running-coral-experiments && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "running-coral-experiments" agent skill from https://github.com/Human-Agent-Society/CORAL/tree/main/plugin/skills/running-coral-experiments into .opencode/skills/running-coral-experiments/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "running-coral-experiments", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
running-coral-experiments- GitHub stars
- 1.1k
- Token cost
- ~1.3k tokens
- SKILL.md length
- 301 words
- Files
- 3 (incl. references)
- Skills in repo
- 7
- Repo updated
- First seen
- Licence
- Apache-2.0
At a glance
Run and manage CORAL experiments from the operator side — launch agents with coral start (dotlist overrides, model/count, tmux vs local), monitor with coral status / coral log / coral show / the web…
- Works in 5 steps: Launch → Monitor → Read results → …
- The user wants to start a CORAL run
- SKILL.md covers 1. Launch, 2. Monitor, 3. Read results and 4. Steer and resume, plus 3 more sections
- Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
What it does
Running Coral Experiments is an agent skill from Human-Agent-Society/CORAL. Run and manage CORAL experiments from the operator side — launch agents with coral start (dotlist overrides, model/count, tmux vs local), monitor with coral status / coral log / coral show / the web dashboard, and drive the loop with coral resume (inject instructions, fork from an attempt), coral heartbeat (tune reflection cadence), and coral stop. Use whenever the user wants to start a CORAL run, check on agents, read scores/leaderboard, steer or resume a run, diagnose agents that keep restarting or fail every…
Its SKILL.md is about 1.3k tokens, which your agent loads only when the skill is triggered. The skill folder holds 3 other files, including reference files (for example `references/scaling-and-ops.md` and `references/steering.md`).
It sits in Agent Workflows. It works with tmux. The repository describes itself as: Open-source autoresearch powered by autonomous coding agents. Run Claude Code, OpenCode, and Codex with grading, shared knowledge, and multi-agent evolution. Accepted at COLM 2026. The licence is Apache-2.0.
When your agent uses it
- The user wants to start a CORAL run
- Check on agents
- Read scores/leaderboard
- Diagnose agents that keep restarting
Example prompts
- “/running-coral-experiments”
Requirements
- Docker
Workflow steps
5 steps, taken from the step headings in SKILL.md.
What it can do on your machine
Read from SKILL.md and the folder at commit 0123dfb. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
No scripts in the folder and no shell commands in SKILL.md (its code samples are bash).
From the folder's file list and the shell code blocks in SKILL.md.
Network
Links to these hosts (documentation or services it may open):
coral.compounding-intelligence.ai
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Running Coral Experiments loads about 1.3k tokens when it runs, and up to ~3.4k if it reads all its reference files. Until then it costs about 178 tokens; SKILL.md has 301 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
The full file from Human-Agent-Society/CORAL at commit 0123dfb, republished under its Apache-2.0 licence (© Human-Agent-Society). 301 words, ~1,315 tokens.
Download SKILL.mdSave it as
.claude/skills/running-coral-experiments/SKILL.md (or your agent's skills folder). This skill also uses 2 other files; get the full folder from GitHub.- name
- running-coral-experiments
- description
- Run and manage CORAL experiments from the operator side — launch agents with `coral start` (dotlist overrides, model/count, tmux vs local), monitor with `coral status` / `coral log` / `coral show` / the web dashboard, and drive the loop with `coral resume` (inject instructions, fork from an attempt), `coral heartbeat` (tune reflection cadence), and `coral stop`. Use whenever the user wants to start a CORAL run, check on agents, read scores/leaderboard, steer or resume a run, diagnose agents that keep restarting or fail every eval, scale to more agents or islands, or stop a run. Deep references for steering/heartbeat tuning and scaling/troubleshooting live alongside this skill.
Running CORAL experiments
You drive a run with five verbs: start → status → log/show → resume → stop. Everything else is a flag on those or a deeper topic in the references. Prefer coral <cmd> --help over guessing flags.
Prereq: a task (task.yaml + seed/ + grader package) that passes coral validate .. No task yet → that's the creating-a-coral-task skill. Each runtime CLI must be installed and authenticated → the setting-up-coral skill.
1. Launch
coral start -c task.yaml # auto-tmux session
coral start -c task.yaml agents.count=4 agents.model=opus # dotlist overrides (no quotes needed)
coral start -c task.yaml run.verbose=true run.ui=true # verbose logs + web dashboard
coral start -c task.yaml run.session=local # foreground, no tmux- Dotlist overrides (
key.subkey=value) beattask.yamlfor this run only — the clean way to sweep count/model without editing the file. run.session:tmux(default, detachable) ·local(foreground) ·docker.- Each run lands in
results/<task-slug>/<timestamp>/; agents work in isolated git worktrees and the grader daemon scores their commits.
2. Monitor
coral status # agent health + leaderboard snapshot (the quick pulse)
coral runs # active runs across tasks; --all includes finished
coral ui --port 8420 # web dashboard: live leaderboard, logs, DAGcoral status answers "who's alive, how many evals, current best". If it looks healthy but scores never move, jump to budget classes + troubleshooting in references/scaling-and-ops.md.
3. Read results
coral log # top 20 real attempts by score
coral log -n 5 --recent # most recent instead of best
coral log --search "kernel" --agent agent-1
coral log --class grader_error # surface crashing graders (first stop when unhealthy)
coral show <hash> # one attempt: score, explanation, files changed
coral show <hash> --diff # full diff — see exactly what the leader did<hash> comes from coral log/coral status. By default coral log hides tune and grader_error attempts; --all shows them, --class {real|tune|grader_error} filters to one. What the classes mean → references/scaling-and-ops.md.
4. Steer and resume
coral resume # resume latest run, sessions restored
coral resume -i "Try greedy approaches first" # inject guidance agents read next loop
coral resume --from <hash> -i "Continue this fork" # reset an agent to an attempt, then steer
coral export <hash> -b winning-idea # export an attempt's commit as a git branchresume -i is how you nudge a run without restarting from scratch (stop → resume with an instruction). --from forks a promising line that later regressed. You can also retune the reflection cadence — coral heartbeat set/remove/reset — to make agents reflect less, pivot sooner, etc. Both topics, with worked examples: references/steering.md.
5. Stop
coral stop # stop the current/latest run (picker if several)
coral stop --all # stop every active runStopping leaves all results, notes, and the leaderboard on disk — coral resume later, or just inspect with coral log/coral show.
Typical loop
coral validate . # grader scores the seed (once)
coral start -c task.yaml agents.count=2 # launch
coral status # ... check periodically
coral log -n 5 --recent # see what agents are trying
coral show <best-hash> --diff # inspect the leader
coral resume -i "Focus on the inner loop" # steer if they plateau
coral stop # doneGoing deeper
- Steer / fork / heartbeat tuning → references/steering.md
- Budget classes, islands, gateway, troubleshooting matrix → references/scaling-and-ops.md
Note: coral eval / diff / revert / checkout / wait are agent-side commands run inside a worktree during a run — agents already know them from the generated CORAL.md. As the operator you rarely touch them; you drive the verbs above. Full CLI reference: https://coral.compounding-intelligence.ai/docs/cli/reference
© Human-Agent-Society, Apache-2.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 2 other files (references) in plugin/skills/running-coral-experiments of Human-Agent-Society/CORAL.
- SKILL.md
- references/scaling-and-ops.md
- references/steering.md
Open the folder on GitHubat commit 0123dfb
Compare with similar skills
Running Coral Experiments next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Running Coral Experiments this skillHuman-Agent-Society/CORAL | 1.1k | — | ~1.3k | Automated safety check: Pass | Apache-2.0 | |
| CodeGraph Agent Evalcolbymchenry/codegraph | 74k | — | ~950 | Automated safety check: Pass | MIT | |
| Agent of Empires Session Manageragent-of-empires/agent-of-empires | 3.3k | — | ~2.5k | Automated safety check: Pass | MIT | |
| Agent Deckasheshgoplani/agent-deck | 1.1k | — | ~1.8k | Automated safety check: Pass | MIT | |
| Agent of Empires Session Manageragent-of-empires/agent-of-empires | 3.3k | — | ~2.1k | Automated safety check: Pass | MIT | |
| Clawteamwin4r/ClawTeam-OpenClaw | 1.5k | — | ~3.1k | Automated safety check: Pass | MIT |
Similar skills
Agent of Empires Session Manager
agent-of-empires/agent-of-empires
Launches, monitors and organizes AI coding agent sessions such as Claude Code or Codex inside tmux, tracking status, capturing output and managing git worktrees for parallel branches.
Agent Deck
asheshgoplani/agent-deck
agent-deck, the terminal session manager for AI coding agents.
Agent of Empires Session Manager
agent-of-empires/agent-of-empires
Starts, monitors and organizes coding agent sessions that run in tmux through the aoe command, including groups, profiles and worktree-based parallel work.
Clawteam
win4r/ClawTeam-OpenClaw
Multi-agent swarm orchestration. An agent skill from win4r/ClawTeam-OpenClaw.
Fleet Manager for Agent Sessions
asgeirtj/system_prompts_leaks
Shows one digest of coding-agent sessions across your connected machines and lets you open, read, steer, approve, stop and close them, over Herdr, tmux or MSP.
More from Human-Agent-Society/CORAL
Creating A Coral Task
Human-Agent-Society/CORAL
Author a new CORAL task — the three pieces that must line up (task.yaml, seed/, a packaged grader/), the coral init → coral validate → smoke-test loop, and how to pick a grader pattern (stdout…
Coral Debug
Human-Agent-Society/CORAL
Verify and debug changes to CORAL itself — smallest reproduce loop per area (grader / daemon / CLI / hooks / manager / workspace / hub / template / config / web), where to look when something breaks…
Coral New Task
Human-Agent-Society/CORAL
End-to-end recipe for adding a new task under examples/ — the three pieces that have to line up (task.yaml, seed/, and grader/), what to put in each, the TaskGrader API surface, the coral validate →…
Promoting Dev To Main
Human-Agent-Society/CORAL
A skill your agent uses when preparing, reviewing, resolving conflicts for, or merging a CORAL release pull request from the long-lived dev branch into main.
Setting Up Coral
Human-Agent-Society/CORAL
One-time machine setup after installing the coral CLI — register local agent runtimes as named bindings with coral setup / coral setup agent, validate them with coral agents doctor (incl.
Coral Extend
Human-Agent-Society/CORAL
Add a new component to the CORAL framework itself — a new agent runtime under coral/agent/builtin/ (claudecode/codex/cursoragent style), a new CLI command in coral/cli/, a new bundled skill or…
Related
Works with
Categories
Questions about Running Coral Experiments
What does Running Coral Experiments do?
Run and manage CORAL experiments from the operator side — launch agents with coral start (dotlist overrides, model/count, tmux vs local), monitor with coral status / coral log / coral show / the web…. Running Coral Experiments is an agent skill from Human-Agent-Society/CORAL. Run and manage CORAL experiments from the operator side — launch agents with coral start (dotlist overrides, model/count, tmux vs local), monitor with coral status / coral log / coral show / the web dashboard, and drive the loop with coral resume (inject instructions, fork from an attempt), coral heartbeat (tune reflection cadence), and coral stop.
When should I use Running Coral Experiments?
Running Coral Experiments fits situations like: the user wants to start a CORAL run; check on agents; read scores/leaderboard; diagnose agents that keep restarting.
How do I install Running Coral Experiments in Claude Code?
Run `npx skills add Human-Agent-Society/CORAL --skill running-coral-experiments -a claude-code`. Or copy the skill folder (plugin/skills/running-coral-experiments in Human-Agent-Society/CORAL) into .claude/skills/running-coral-experiments in your project. Claude Code loads it when a task matches its description.
How do I install Running Coral Experiments in Codex?
Run `npx skills add Human-Agent-Society/CORAL --skill running-coral-experiments -a codex`. Or copy the skill folder (plugin/skills/running-coral-experiments in Human-Agent-Society/CORAL) into .agents/skills/running-coral-experiments in your project. Codex loads it when a task matches its description.
Can I use Running Coral Experiments in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add Human-Agent-Society/CORAL --skill running-coral-experiments -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/running-coral-experiments, .gemini/skills/running-coral-experiments, .github/skills/running-coral-experiments and .opencode/skills/running-coral-experiments in your project.
What does Running Coral Experiments need to run?
SKILL.md names no scripts, command-line tools or credentials: Running Coral Experiments is instructions for the agent only. Our summary lists: Docker.
Does Running Coral Experiments access the network?
SKILL.md names 1 domain. As links in the text: coral.compounding-intelligence.ai. This is read from the text; nothing was executed.
Is Running Coral Experiments safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Running Coral Experiments use?
Running Coral Experiments is published under the Apache-2.0 licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Running Coral Experiments use?
About 1.3k tokens (SKILL.md is roughly 5.3k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 2.1k tokens, read only when the agent opens those files.
What are the alternatives to Running Coral Experiments?
Skills that share tags, products or a category with Running Coral Experiments: CodeGraph Agent Eval (colbymchenry/codegraph, 74k stars), Agent of Empires Session Manager (agent-of-empires/agent-of-empires, 3.3k stars), Agent Deck (asheshgoplani/agent-deck, 1.1k stars) and Agent of Empires Session Manager (agent-of-empires/agent-of-empires, 3.3k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Running Coral Experiments?
Human-Agent-Society (a GitHub organization) maintains it in Human-Agent-Society/CORAL, which has 1,060 GitHub stars. The repository holds 7 skills in this directory. The repository was last updated on September 8, 2026.
Source: Human-Agent-Society/CORAL on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.