New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Add Runner Eval

by paperclipai in paperclipai/paperclip

Add or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.

MITAuto-check passed

Install Add Runner Eval

skills CLI
$ npx skills add paperclipai/paperclip --skill add-runner-eval -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install paperclipai/paperclip add-runner-eval --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/paperclipai/paperclip.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.agents/skills/add-runner-eval .claude/skills/add-runner-eval && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
add-runner-eval
GitHub stars
99k
Token cost
~1k tokens
SKILL.md length
416 words
Files
1
Skills in repo
60
Repo updated
First seen
Licence
MIT

At a glance

Add or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.

  • Calls python3 and git

What it does

Add Runner Eval is an agent skill from paperclipai/paperclip. Add or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.

Its SKILL.md is about 1k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

The repository describes itself as: The open-source app everyone uses to manage agents at work. The licence is MIT.

Example prompts

  • “/add-runner-eval”

Requirements

  • Python 3

What it can do on your machine

Read from SKILL.md and the folder at commit b9750b1. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • python3
    • git

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md. Its commands use git, which can reach the network depending on how they are called.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Add Runner Eval loads about 1k tokens when it runs. Until then it costs about 39 tokens; SKILL.md has 416 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~39
When it runs · the whole SKILL.md, loaded when a task matches
~1k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from paperclipai/paperclip at commit b9750b1, republished under its MIT licence (© paperclipai). 416 words, ~1,037 tokens.

Download SKILL.mdSave it as .claude/skills/add-runner-eval/SKILL.md (or your agent's skills folder).
name
add-runner-eval
description
Add or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.

Add a Runner Eval

Use this skill for the Runner Evals family: a real Runner/provider session against a seeded mock control plane. Product browser/server/database/Daytona coverage belongs in add-product-e2e-eval.

Locate the Paperclip checkout using PAPERCLIP_ROOT when supplied, or git rev-parse --show-toplevel from a checkout. From outside Git, inspect the workspace roots (for example ~/paperclipai/paperclip) and verify that the selected root contains packages/paperclip-runner and tests/runner-e2e. Locate paperclip-evals using PAPERCLIP_EVALS_ROOT or a discovered sibling; a worktree's parent directory need not contain that repository. Read doc/evals.md and packages/paperclip-runner/docs/runner-protocol-live-evals.md, then inspect the nearest existing case, roster, schema, and report test before editing. Definitions and authored cases belong in the sibling paperclip-evals/evals/paperclip-runner; Runner integration, aggregation, viewer, and publication behavior belongs in packages/paperclip-runner. Keep the control-plane boundary explicit in names and documentation.

The sibling eval README is the concrete map: cases live under cases/, company fixtures under fixtures/, runtime/model settings under configs/, selections under rosters/, and maintained campaign membership under campaigns/live-direct-full.json. Update inventory/coverage mappings when the program requires them; a new file alone does not join the maintained campaign. From the Evals repository root, adapt these provider-free checks to the case and roster you changed. Run the reliability-plan validator only when that separate plan changes:

sh
python3 evals/paperclip-runner/tools/eval_program.py validate \
  --case evals/paperclip-runner/cases/get-task-context.json \
  --config evals/paperclip-runner/configs/live-codex-pinned.json
python3 evals/paperclip-runner/tools/run_live_roster.py validate \
  --roster evals/paperclip-runner/rosters/live-mini.json --run-id validate-new-case
python3 evals/paperclip-runner/tools/run_live_campaign.py validate \
  --campaign evals/paperclip-runner/campaigns/live-direct-full.json
python3 evals/paperclip-runner/tools/reliability_campaign.py validate \
  --plan evals/paperclip-runner/campaigns/paperclip-runner-reliability.json

Use nearby positive and negative grader cases/fixtures to calibrate the new assertion, including malformed or missing evidence where the grader must fail closed. Preserve the existing machine disposition and grade; product, model/provider, grading, and infrastructure labels are analytical annotations, not instructions to rewrite classifiers.

Author one bounded case with a deterministic semantic assertion and an inspectable result. Declare its expected operation, state effect, provider lane/profile, timeout and retry policy, and any required evidence. Do not grade hidden reasoning, infer success from a provider terminal message, or invent conversation/tool evidence. Public output follows the reviewed projection: sanitized fixture conversation and allowlisted tool outcomes may be published; raw trusted artifacts, credentials, secrets, private references, and hidden reasoning may not.

Show full SKILL.md (101 more words)Show less

Validate without provider calls first using the commands above and the relevant report/render validation documented in the Runner docs. When a live run is authorized, pin the Paperclip commit and exact 40-character paperclip-evals commit, select the smallest useful roster, and retain the complete provenance and cost record.

Update authoritative detailed docs when the contract or command changes, then link from doc/evals.md rather than duplicating the Runner runbook. Keep public reports immutable and use the reviewed projection; sanitized fixture conversation and allowlisted tool outcomes may be public, while credentials, secrets, private references, raw trusted payloads, and hidden reasoning must not be exposed.

© paperclipai, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in .agents/skills/add-runner-eval of paperclipai/paperclip.

Open the folder on GitHubat commit b9750b1

Compare with similar skills

Add Runner Eval next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Add Runner Eval compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Add Runner Eval this skillpaperclipai/paperclip99k—~1kAutomated safety check: PassMIT
Eval-Driven Development Harnessaffaan-m/ECC276k—~1.5kAutomated safety check: PassMIT
Evalalirezarezvani/claude-skills28k—~618Automated safety check: PassMIT
Eval Harnessaffaan-m/ECC276k—~2.2kAutomated safety check: PassMIT
PaperclipK-Dense-AI/scientific-agent-skills48k1 repos~3.2kAutomated safety check: NotesMIT
Eval Harnessaffaan-m/ECC276k1 repos~1.7kAutomated safety check: PassMIT

Similar skills

  • Eval-Driven Development Harness

    affaan-m/ECC

    Sets up eval-driven development for Claude Code workflows: capability and regression evals, three grader types and pass@k reliability metrics.

    276k GitHub stars~1.5k tokensUpdated 4 days ago
    Agent WorkflowsAuto-check passed

  • Eval

    alirezarezvani/claude-skills

    Evaluate and rank agent results by metric or LLM judge for an AgentHub session.

    28k GitHub stars~618 tokensUpdated 1 mo ago
    AI & LLM EngineeringAuto-check passed

  • Eval Harness

    affaan-m/ECC

    Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k…

    276k GitHub stars~2.2k tokensUpdated 4 days ago
    AI & LLM EngineeringAuto-check passed

  • Paperclip

    K-Dense-AI/scientific-agent-skills

    Searches and reads biomedical papers, FDA/PMDA/EMA documents, clinical trials, and protein records with the GXL Paperclip CLI and Python SDK.

    48k GitHub starsUsed in 1 repo~3.2k tokens
    Research & ScienceAuto-check: notes

  • Eval Harness

    affaan-m/ECC

    Eval-driven development (EDD) ilkelerini uygulayan Claude Code oturumları için formal değerlendirme çerçevesi

    276k GitHub starsUsed in 1 repo~1.7k tokens
    AI & LLM EngineeringAuto-check passed

  • OmniRoute CLI Evals

    diegosouzapw/OmniRoute

    Creates and runs LLM evaluation suites from the omniroute CLI, follows live runs, shows scorecards, compares models and ties eval runs into CI.

    74k GitHub stars~1.3k tokensUpdated today
    AI & LLM EngineeringAuto-check passed

More from paperclipai/paperclip

All 60 skills in this repo

  • Garden Inbox

    paperclipai/paperclip

    Scan a Paperclip user's Mine inbox, classify reversible archive candidates, request checkbox confirmation, and archive only accepted selections.

    99k GitHub stars~1.1k tokensUpdated today
    Auto-check passed

  • Paperclip

    paperclipai/paperclip

    Interact with the Paperclip control plane API for task coordination and governance.

    99k GitHub stars~9.6k tokensUpdated today
    Auto-check passed

  • Paperclip

    paperclipai/paperclip

    A skill your agent uses for Paperclip-managed tasks and heartbeats: reading task context, delivering task documents or files, updating completion or blockers, coordinating or delegating work, and…

    99k GitHub stars~17k tokensUpdated today
    Auto-check passed

  • Design Guide

    paperclipai/paperclip

    Paperclip UI design system guide for building consistent, reusable frontend components.

    99k GitHub starsUsed in 1 repo~3.1k tokens
    Auto-check passed

  • Paperclip Page

    paperclipai/paperclip

    Publish static HTML pages and asset folders to the Paperclip S3/CloudFront page host.

    99k GitHub stars~1k tokensUpdated today
    Auto-check passed

  • Paperclip Create Agent

    paperclipai/paperclip

    Create new agents in Paperclip with governance-aware hiring.

    99k GitHub starsUsed in 1 repo~2.1k tokens
    Auto-check passed

Questions about Add Runner Eval

What does Add Runner Eval do?

Add or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation. Add Runner Eval is an agent skill from paperclipai/paperclip. Add or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.

How do I install Add Runner Eval in Claude Code?

Run `npx skills add paperclipai/paperclip --skill add-runner-eval -a claude-code`. Or copy the skill folder (.agents/skills/add-runner-eval in paperclipai/paperclip) into .claude/skills/add-runner-eval in your project. Claude Code loads it when a task matches its description.

How do I install Add Runner Eval in Codex?

Run `npx skills add paperclipai/paperclip --skill add-runner-eval -a codex`. Or copy the skill folder (.agents/skills/add-runner-eval in paperclipai/paperclip) into .agents/skills/add-runner-eval in your project. Codex loads it when a task matches its description.

Can I use Add Runner Eval in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add paperclipai/paperclip --skill add-runner-eval -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/add-runner-eval, .gemini/skills/add-runner-eval, .github/skills/add-runner-eval and .opencode/skills/add-runner-eval in your project.

What does Add Runner Eval need to run?

Going by SKILL.md and its folder, Add Runner Eval needs the command-line tools its instructions call (python3 and git). Our summary lists: Python 3.

Does Add Runner Eval access the network?

SKILL.md contains no URLs. Its commands use git, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is Add Runner Eval safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Add Runner Eval use?

Add Runner Eval is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Add Runner Eval use?

About 1k tokens (SKILL.md is roughly 4.1k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Add Runner Eval?

Skills that share tags, products or a category with Add Runner Eval: Eval-Driven Development Harness (affaan-m/ECC, 276k stars), Eval (alirezarezvani/claude-skills, 28k stars), Eval Harness (affaan-m/ECC, 276k stars) and Paperclip (K-Dense-AI/scientific-agent-skills, 48k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Add Runner Eval?

paperclipai (a GitHub organization) maintains it in paperclipai/paperclip, which has 98,967 GitHub stars. The repository holds 60 skills in this directory. The repository was last updated on October 9, 2026.

Source: paperclipai/paperclip on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.