Official agent skill

Benchmark Agents

Advanced AI agent benchmark scenarios that push Vercel's cutting-edge platform features — Workflow SDK, AI Gateway, MCP, Chat SDK, Queues, Flags, Sandbox, and multi-agent orchestration.

OfficialCustom licenceAuto-check passedAgent Workflows

Install Benchmark Agents

skills CLI

$ npx skills add vercel/vercel-plugin --skill benchmark-agents -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI

$ gh skill install vercel/vercel-plugin benchmark-agents --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy

$ git clone --depth 1 https://github.com/vercel/vercel-plugin.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.claude/skills/benchmark-agents .claude/skills/benchmark-agents && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Paste into Claude Code

Install the "benchmark-agents" agent skill from https://github.com/vercel/vercel-plugin/tree/main/.claude/skills/benchmark-agents into .claude/skills/benchmark-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "benchmark-agents", then confirm the skill loads.

Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

$skill-installer

$skill-installer install https://github.com/vercel/vercel-plugin/tree/main/.claude/skills/benchmark-agents

Type this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.

skills CLI

$ npx skills add vercel/vercel-plugin --skill benchmark-agents -a codex

Project install goes to .agents/skills/; add -g for ~/.codex/skills/.

GitHub CLI

$ gh skill install vercel/vercel-plugin benchmark-agents --agent codex

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/vercel/vercel-plugin.git skills-src && mkdir -p .agents/skills && cp -r skills-src/.claude/skills/benchmark-agents .agents/skills/benchmark-agents && rm -rf skills-src

Use ~/.agents/skills/ instead of .agents/skills for a personal install.

Codex skills documentation · loads skills from .agents/skills/

Paste into Codex

Install the "benchmark-agents" agent skill from https://github.com/vercel/vercel-plugin/tree/main/.claude/skills/benchmark-agents into .agents/skills/benchmark-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "benchmark-agents", then confirm the skill loads.

Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add vercel/vercel-plugin --skill benchmark-agents -a cursor

Project install goes to .agents/skills/; add -g for ~/.cursor/skills/.

GitHub CLI

$ gh skill install vercel/vercel-plugin benchmark-agents --agent cursor

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/vercel/vercel-plugin.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/.claude/skills/benchmark-agents .cursor/skills/benchmark-agents && rm -rf skills-src

Use ~/.cursor/skills/ instead of .cursor/skills for a personal install.

Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/

Paste into Cursor

Install the "benchmark-agents" agent skill from https://github.com/vercel/vercel-plugin/tree/main/.claude/skills/benchmark-agents into .cursor/skills/benchmark-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "benchmark-agents", then confirm the skill loads.

Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

gemini skills install

$ gemini skills install https://github.com/vercel/vercel-plugin.git --path .claude/skills/benchmark-agents

--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.

skills CLI

$ npx skills add vercel/vercel-plugin --skill benchmark-agents -a gemini-cli

Project install goes to .agents/skills/; add -g for ~/.gemini/skills/.

GitHub CLI

$ gh skill install vercel/vercel-plugin benchmark-agents --agent gemini-cli

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/vercel/vercel-plugin.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/.claude/skills/benchmark-agents .gemini/skills/benchmark-agents && rm -rf skills-src

Use ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.

Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/

Paste into Gemini CLI

Install the "benchmark-agents" agent skill from https://github.com/vercel/vercel-plugin/tree/main/.claude/skills/benchmark-agents into .gemini/skills/benchmark-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "benchmark-agents", then confirm the skill loads.

Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

GitHub CLI

$ gh skill install vercel/vercel-plugin benchmark-agents

Installs for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).

skills CLI

$ npx skills add vercel/vercel-plugin --skill benchmark-agents -a github-copilot

Project install goes to .agents/skills/; add -g for ~/.copilot/skills/.

Manual copy

$ git clone --depth 1 https://github.com/vercel/vercel-plugin.git skills-src && mkdir -p .github/skills && cp -r skills-src/.claude/skills/benchmark-agents .github/skills/benchmark-agents && rm -rf skills-src

Use ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.

GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/

Paste into GitHub Copilot

Install the "benchmark-agents" agent skill from https://github.com/vercel/vercel-plugin/tree/main/.claude/skills/benchmark-agents into .github/skills/benchmark-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "benchmark-agents", then confirm the skill loads.

GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add vercel/vercel-plugin --skill benchmark-agents -a opencode

OpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.

GitHub CLI

$ gh skill install vercel/vercel-plugin benchmark-agents --agent opencode

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/vercel/vercel-plugin.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/.claude/skills/benchmark-agents .opencode/skills/benchmark-agents && rm -rf skills-src

Use ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.

OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/

Paste into OpenCode

Install the "benchmark-agents" agent skill from https://github.com/vercel/vercel-plugin/tree/main/.claude/skills/benchmark-agents into .opencode/skills/benchmark-agents/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "benchmark-agents", then confirm the skill loads.

OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

Facts

Skill name: benchmark-agents
GitHub stars: 301
Token cost: ~3.6k tokens
SKILL.md length: 1,304 words
Files: 2
Skills in repo: 38
Repo updated: Oct 6, 2026
First seen: Oct 7, 2026
Licence: Custom licence

At a glance

Advanced AI agent benchmark scenarios that push Vercel's cutting-edge platform features — Workflow SDK, AI Gateway, MCP, Chat SDK, Queues, Flags, Sandbox, and multi-agent orchestration.

Works in 4 steps: Create test directory and install plugin → Launch session via WezTerm → Find the debug log (wait ~25s for… → …
Tasks that involve LLM evaluation
SKILL.md covers How Evals Work (The Only…, DO NOT (Hard Rules), Setup & Launch (Exact Commands) and Monitoring, plus 8 more sections
Calls npx, bun and node

What it does

Benchmark Agents is an agent skill from vercel/vercel-plugin, published by the product's own GitHub organization. Advanced AI agent benchmark scenarios that push Vercel's cutting-edge platform features — Workflow SDK, AI Gateway, MCP, Chat SDK, Queues, Flags, Sandbox, and multi-agent orchestration. Designed to stress-test skill injection for complex, multi-system builds.

Its SKILL.md is about 3.6k tokens, which your agent loads only when the skill is triggered. The skill folder holds 1 other file (for example `prompts.md`).

It sits in Agent Workflows, covering LLM evaluation, Load testing and Multi-agent orchestration. It works with Vercel, Model Context Protocol and Bash. The repository describes itself as: Comprehensive Vercel ecosystem plugin — relational knowledge graph, skills for every major product, specialized agents, and Vercel conventions. Turns any AI agent into a Vercel…

When your agent uses it

Tasks that involve LLM evaluation
Tasks that involve Load testing
Tasks that involve Multi-agent orchestration

Example prompts

“/benchmark-agents”

Requirements

Node.js

Workflow steps

4 steps, taken from the step headings in SKILL.md.

Create test directory and install plugin
Launch session via WezTerm
Find the debug log (wait ~25s for SessionStart hooks)
Launch multiple sessions in parallel

What it can do on your machine

Read from SKILL.md and the folder at commit 82fa491. It shows what the files ask for, not the result of running them.

Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Shell commands in SKILL.md call:
- npx
- bun
- node
- claude
- bash
- vercel
- git
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md. Its commands use npx, vercel and git, which can reach the network depending on how they are called.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Benchmark Agents loads about 3.6k tokens when it runs. Until then it costs about 69 tokens; SKILL.md has 1,304 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it: ~69
When it runs · the whole SKILL.md, loaded when a task matches: ~3.6k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

Its licence (Custom licence) doesn't allow us to republish the file, so here is its outline and opening line. It has 1,304 words (~3,617 tokens).

“Launch real Claude Code sessions with the plugin installed, verify skill injection, monitor PostToolUse validation catches, and produce a coverage report. This skill covers the full eval loop: setup → launch → monitor → verify → fix → release →…”

— opening of SKILL.md by vercel, Custom licence

name: benchmark-agents

Outline of SKILL.md

Benchmark Agents — Advanced AI Systems
How Evals Work (The Only Correct Method)
DO NOT (Hard Rules)
Setup & Launch (Exact Commands)
Naming convention
1. Create test directory and install plugin
2. Launch session via WezTerm
3. Find the debug log (wait ~25s for SessionStart hooks)
4. Launch multiple sessions in parallel
Monitoring
Skill injection claims (the key metric)
Hook firing
Quick status check for multiple sessions
Verification — What to Check in Generated Code
Project structure
Image generation model
Gateway vs direct provider
AI Elements installed
Workflow API usage
Prompt Design Rules
DO:
DON'T:
Always end prompts with:
Phrases that trigger key skills (via promptSignals):
Common Issues Found in Evals (and Fixes Applied)
Agent-Browser Verification
Coverage Report
Release → Eval Loop
Scenario Table
Complexity Tiers

Read the full SKILL.md on GitHub

Files

SKILL.md and 1 other file in .claude/skills/benchmark-agents of vercel/vercel-plugin.

SKILL.md
prompts.md

Open the folder on GitHubat commit 82fa491

Compare with similar skills

Benchmark Agents next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Benchmark Agents compared with similar skills
Skill	Stars	Used in	Tokens	Auto-check	Licence	Repo updated
Benchmark Agents this skillvercel/vercel-plugin	301	—	~3.6k	Automated safety check: Pass	Custom licence	yesterday
Octocode Graph Eval Loopbgauryy/octocode	946	—	~1.6k	Automated safety check: Pass	MIT	4 days ago
Waza Interactivemicrosoft/waza	1.4k	—	~1.3k	Automated safety check: Pass	MIT	yesterday
Octocode Benchmark Runnerbgauryy/octocode	946	—	~2.1k	Automated safety check: Pass	MIT	4 days ago
Agent Observability Eval Bootstrapdatadog-labs/agent-skills	177	—	~25k	Automated safety check: Pass	MIT	5 days ago
Autocontext for Hermesgreyhaven-ai/autocontext	1.3k	—	~2.5k	Automated safety check: Pass	Apache-2.0	yesterday

Similar skills

Octocode Graph Eval Loop
bgauryy/octocode
Runs a measurable keep-or-discard improvement loop against a runnable sensor, from framing a goal and KPI through baseline, judging and held-out verification.
946 GitHub stars~1.6k tokensUpdated 4 days ago
Agent WorkflowsAuto-check passed
Waza Interactive
microsoft/waza
Official
Walks you through creating, running and reading waza evals for an agent skill, then proposes concrete fixes when tasks fail or the score is low.
1.4k GitHub stars~1.3k tokensUpdated yesterday
Agent WorkflowsAuto-check passed
Octocode Benchmark Runner
bgauryy/octocode
Runs blind pairwise comparisons of Octocode against a gh-based baseline over markdown research questions, scored by total characters through the model rather than self-report.
946 GitHub stars~2.1k tokensUpdated 4 days ago
AI & LLM EngineeringAuto-check passed
Agent Observability Eval Bootstrap
datadog-labs/agent-skills
Bootstrap evaluators from production traces — by default propose online LLM-judge evaluators and, after you confirm, create them in Datadog as disabled drafts (never auto-enabled); on request emit…
177 GitHub stars~25k tokensUpdated 5 days ago
DevOps & CloudAuto-check passed
Autocontext for Hermes
greyhaven-ai/autocontext
Lets a Hermes agent run Autocontext scenarios, inspect Hermes curator state, export reusable knowledge and prepare local MLX or CUDA training data through the autoctx CLI.
1.3k GitHub stars~2.5k tokensUpdated yesterday
Agent WorkflowsAuto-check passed
Swarmclaw
swarmclawai/swarmclaw
Manage your SwarmClaw agent fleet — agents, tasks, chats, chatrooms, goals, schedules, memory, wallets, connectors, autonomy, and 40+ more command groups.
687 GitHub stars~4.2k tokensUpdated 3 mo ago
Agent WorkflowsAuto-check passed

Questions about Benchmark Agents

What does Benchmark Agents do?

Advanced AI agent benchmark scenarios that push Vercel's cutting-edge platform features — Workflow SDK, AI Gateway, MCP, Chat SDK, Queues, Flags, Sandbox, and multi-agent orchestration. Benchmark Agents is an agent skill from vercel/vercel-plugin, published by the product's own GitHub organization. Advanced AI agent benchmark scenarios that push Vercel's cutting-edge platform features — Workflow SDK, AI Gateway, MCP, Chat SDK, Queues, Flags, Sandbox, and multi-agent orchestration.

When should I use Benchmark Agents?

Benchmark Agents fits situations like: tasks that involve LLM evaluation; tasks that involve Load testing; tasks that involve Multi-agent orchestration.

How do I install Benchmark Agents in Claude Code?

Run `npx skills add vercel/vercel-plugin --skill benchmark-agents -a claude-code`. Or copy the skill folder (.claude/skills/benchmark-agents in vercel/vercel-plugin) into .claude/skills/benchmark-agents in your project. Claude Code loads it when a task matches its description.

How do I install Benchmark Agents in Codex?

Run `npx skills add vercel/vercel-plugin --skill benchmark-agents -a codex`. Or copy the skill folder (.claude/skills/benchmark-agents in vercel/vercel-plugin) into .agents/skills/benchmark-agents in your project. Codex loads it when a task matches its description.

Can I use Benchmark Agents in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add vercel/vercel-plugin --skill benchmark-agents -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/benchmark-agents, .gemini/skills/benchmark-agents, .github/skills/benchmark-agents and .opencode/skills/benchmark-agents in your project.

What does Benchmark Agents need to run?

Going by SKILL.md and its folder, Benchmark Agents needs the command-line tools its instructions call (npx, bun, node, claude, bash and vercel). Our summary lists: Node.js.

Does Benchmark Agents access the network?

SKILL.md contains no URLs. Its commands use npx and git, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is Benchmark Agents safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Benchmark Agents use?

Benchmark Agents has a licence file (the repository's licence) that doesn't match a standard licence. Read it on GitHub before reusing the skill.

How many tokens does Benchmark Agents use?

About 3.6k tokens (SKILL.md is roughly 14k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Benchmark Agents?

Skills that share tags, products or a category with Benchmark Agents: Octocode Graph Eval Loop (bgauryy/octocode, 946 stars), Waza Interactive (microsoft/waza, 1.4k stars), Octocode Benchmark Runner (bgauryy/octocode, 946 stars) and Agent Observability Eval Bootstrap (datadog-labs/agent-skills, 177 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Benchmark Agents?

vercel (a GitHub organization, an official publisher) maintains it in vercel/vercel-plugin, which has 301 GitHub stars. The repository holds 38 skills in this directory. The repository was last updated on October 6, 2026.

Source: vercel/vercel-plugin on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.

Benchmark Agents

Install Benchmark Agents

Facts

At a glance

What it does

When your agent uses it

Example prompts

Requirements

Workflow steps

What it can do on your machine

Tool permissions

Runs code

Network

Credentials

Context cost

Safety

SKILL.md

Files

Compare with similar skills

Similar skills

Octocode Graph Eval Loop

Waza Interactive

Octocode Benchmark Runner

Agent Observability Eval Bootstrap

Autocontext for Hermes

Swarmclaw

More from vercel/vercel-plugin

Deployments Cicd

Plugin Audit

Vercel CLI

AI Gateway

Vercel Services

Access Protected Vercel Deployment

Questions about Benchmark Agents

Install Benchmark Agents

Facts

At a glance

What it does

When your agent uses it

Example prompts

Requirements

Workflow steps

What it can do on your machine

Tool permissions

Runs code

Network

Credentials

Context cost

Safety

SKILL.md

Files

Compare with similar skills

Similar skills

More from vercel/vercel-plugin

Related

Questions about Benchmark Agents