New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Evaluation Benchmarking

by AbdelStark in AbdelStark/worldforge

A skill your agent uses for WorldForge evaluation suites, benchmark harness changes, benchmark input fixtures, budget gates, report rendering, metrics semantics, and any claims based on benchmark or…

MITAuto-check passed

Install Evaluation Benchmarking

skills CLI
$ npx skills add AbdelStark/worldforge --skill evaluation-benchmarking -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install AbdelStark/worldforge evaluation-benchmarking --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/AbdelStark/worldforge.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.codex/skills/evaluation-benchmarking .claude/skills/evaluation-benchmarking && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
evaluation-benchmarking
GitHub stars
108
Token cost
~732 tokens
SKILL.md length
302 words
Files
2
Skills in repo
7
Repo updated
First seen
Licence
MIT

At a glance

A skill your agent uses for WorldForge evaluation suites, benchmark harness changes, benchmark input fixtures, budget gates, report rendering, metrics semantics, and any claims based on benchmark or…

  • Works in 6 steps: Read src/worldforge/evaluation/suites.py… → Validate inputs eagerly through… → Keep examples/benchmark-inputs.json and… → …
  • WorldForge evaluation suites
  • SKILL.md covers Ground Rules, Workflow, Definition Of Done and Metric Semantics, plus 1 more section
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

Evaluation Benchmarking is an agent skill from AbdelStark/worldforge. Use for WorldForge evaluation suites, benchmark harness changes, benchmark input fixtures, budget gates, report rendering, metrics semantics, and any claims based on benchmark or evaluation output. Keeps benchmark/eval artifacts deterministic, coherent, and claim-bounded.

Its SKILL.md is about 730 tokens, which your agent loads only when the skill is triggered. The skill folder holds 2 other files (for example `agents/openai.yaml`).

The repository describes itself as: Harness framework to build world model based workflows for physical AI systems. The licence is MIT.

When your agent uses it

  • WorldForge evaluation suites
  • Benchmark harness changes
  • Benchmark input fixtures
  • Report rendering

Example prompts

  • “/evaluation-benchmarking”

Workflow steps

6 steps, taken from the first numbered list in SKILL.md.

  1. Read src/worldforge/evaluation/suites.py for eval changes or src/worldforge/benchmark.py for benchmark changes.
  2. Validate inputs eagerly through BenchmarkInputs and load_benchmark_inputs(...); reject unknown keys and non-finite metrics.
  3. Keep examples/benchmark-inputs.json and examples/benchmark-budget.json reproducible and checkout-safe.
  4. Keep score and policy payloads JSON-native; for provider-native tensors or arrays, preview type and shape rather than forcing raw encoding.
  5. If operation surfaces or CLI text change, update help snapshots, harness diagnostics, README, docs/src/benchmarking.md…
  6. Test direct operation behavior, input parsing, budget pass/fail paths, and renderer output.

What it can do on your machine

Read from SKILL.md and the folder at commit 30b65da. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Evaluation Benchmarking loads about 732 tokens when it runs. Until then it costs about 74 tokens; SKILL.md has 302 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~74
When it runs · the whole SKILL.md, loaded when a task matches
~732

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from AbdelStark/worldforge at commit 30b65da, republished under its MIT licence (© AbdelStark). 302 words, ~732 tokens.

Download SKILL.mdSave it as .claude/skills/evaluation-benchmarking/SKILL.md (or your agent's skills folder). This skill also uses 1 other file; get the full folder from GitHub.
name
evaluation-benchmarking
description
Use for WorldForge evaluation suites, benchmark harness changes, benchmark input fixtures, budget gates, report rendering, metrics semantics, and any claims based on benchmark or evaluation output. Keeps benchmark/eval artifacts deterministic, coherent, and claim-bounded.

Evaluation And Benchmarking

Ground Rules

  • Built-in eval suites are deterministic contract harnesses, not physical-fidelity or media-quality evidence.
  • Benchmark direct provider operations only: predict, embed, score, and policy.
  • plan() is a WorldForge facade workflow. Do not route benchmark score or policy through it.
  • Preserve BenchmarkBudget non-zero exit behavior on violations.
  • Preserve claim-boundary and metric-semantics metadata in JSON, Markdown, and CSV renderers.
  • Treat benchmark/eval output as evidence only when the fixture, command, provider surface, and renderer semantics are all current and inspectable.

Workflow

  1. Read src/worldforge/evaluation/suites.py for eval changes or src/worldforge/benchmark.py for benchmark changes.
  2. Validate inputs eagerly through BenchmarkInputs and load_benchmark_inputs(...); reject unknown keys and non-finite metrics.
  3. Keep examples/benchmark-inputs.json and examples/benchmark-budget.json reproducible and checkout-safe.
  4. Keep score and policy payloads JSON-native; for provider-native tensors or arrays, preview type and shape rather than forcing raw encoding.
  5. If operation surfaces or CLI text change, update help snapshots, harness diagnostics, README, docs/src/benchmarking.md, docs/src/api/python.md, docs/src/playbooks.md, and changelog together.
  6. Test direct operation behavior, input parsing, budget pass/fail paths, and renderer output.

Definition Of Done

  • The changed suite or benchmark path has focused tests for success and failure behavior.
  • Rendered artifacts remain JSON-native and internally coherent before Markdown/CSV output.
  • Claims in README/docs/changelog match the actual command and provider used.
  • The final validation includes the focused tests plus docs/provider checks when public text changed.

Metric Semantics

  • Latency is process-local wall-clock timing for successful samples.
  • Retry counts come from emitted ProviderEvent records.
  • Throughput is successful samples over elapsed time.
  • Event rows can aggregate attempts; sum request_count when reporting actual request volume.

Sharp Edges

SymptomCauseFix
--input-file rejects fixtureUnknown key or non-JSON payloadMatch allowed BenchmarkInputs keys and JSON-native values
Budget gate exits non-zeroThreshold violationPreserve artifact, inspect report, adjust code or documented budget with evidence
CLI help snapshot failsOperation/help text changedUpdate tests/test_cli_help_snapshots.py intentionally

© AbdelStark, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 1 other file in .codex/skills/evaluation-benchmarking of AbdelStark/worldforge.

  • SKILL.md
  • agents/openai.yaml

Open the folder on GitHubat commit 30b65da

Compare with similar skills

Evaluation Benchmarking next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Evaluation Benchmarking compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Evaluation Benchmarking this skillAbdelStark/worldforge108—~732Automated safety check: PassMIT
Agent Benchmark Suiteruvnet/ruflo74k2 repos~4.9kAutomated safety check: PassMIT
AI Agent Evaluation Benchmarkingsickn33/agentic-awesome-skills47k1 repos~1.3kAutomated safety check: PassMIT
LLM Benchmarking with lm-evaluation-harnessOrchestra-Research/AI-Research-SKILLs13k8 repos~3kAutomated safety check: PassMIT
Benchmarkaffaan-m/ECC275k3 repos~654Automated safety check: PassMIT
Benchmarkaffaan-m/ECC275k—~412Automated safety check: PassMIT

Similar skills

  • Agent Benchmark Suite

    ruvnet/ruflo

    Agent skill for benchmark-suite - invoke with $agent-benchmark-suite

    74k GitHub starsUsed in 2 repos~4.9k tokens
    Agent WorkflowsAuto-check passed

  • AI Agent Evaluation Benchmarking

    sickn33/agentic-awesome-skills

    Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.

    47k GitHub starsUsed in 1 repo~1.3k tokens
    Agent WorkflowsAuto-check passed

  • LLM Benchmarking with lm-evaluation-harness

    Orchestra-Research/AI-Research-SKILLs

    Runs lm-evaluation-harness to benchmark language models on academic suites such as MMLU, GSM8K and HumanEval, compare models and track training checkpoints.

    13k GitHub starsUsed in 8 repos~3k tokens
    AI & LLM EngineeringAuto-check passed

  • Benchmark

    affaan-m/ECC

    Measure performance baselines and detect regressions across browser Core Web Vitals (LCP, INP, CLS, page weight), API endpoint latency percentiles, and build/test feedback times, with before/after…

    275k GitHub starsUsed in 3 repos~654 tokens
    Frontend & DesignAuto-check passed

  • Benchmark

    affaan-m/ECC

    このスキルを使用して、パフォーマンスベースラインを測定し、PR前後の回帰を検出し、スタック代替案を比較します. An agent skill from affaan-m/ECC.

    275k GitHub stars~412 tokensUpdated 3 days ago
    Auto-check passed

  • Benchmark

    affaan-m/ECC

    使用此技能测量性能基线，检测PR前后的回归，并比较堆栈替代方案。

    275k GitHub stars~330 tokensUpdated 3 days ago
    Auto-check passed

More from AbdelStark/worldforge

  • Public Docs Release

    AbdelStark/worldforge

    A skill your agent uses for WorldForge README, docs, changelog, generated provider docs, MkDocs navigation, version/release metadata, public positioning, and release or publish readiness checks.

    108 GitHub stars~679 tokensUpdated 19 days ago
    Auto-check passed

  • Testing Validation

    AbdelStark/worldforge

    A skill your agent uses when selecting, running, or fixing WorldForge validation: pytest, coverage, ruff, generated provider docs, MkDocs strict build, package contract, CI failures, and release…

    108 GitHub stars~871 tokensUpdated 19 days ago
    Auto-check passed

  • Optional Runtime Smokes

    AbdelStark/worldforge

    A skill your agent uses for LeWorldModel, GR00T, LeRobot, PushT robotics showcase, real-checkpoint smoke scripts, checkpoint building, and host-owned optional runtime dependencies.

    108 GitHub stars~855 tokensUpdated 19 days ago
    Auto-check passed

  • Persistence State

    AbdelStark/worldforge

    A skill your agent uses for WorldForge local state: run workspaces and artifacts under .worldforge/ (run manifests, evidence bundles, retention/prune), the WorldForge(statedir=...) directory, and…

    108 GitHub stars~921 tokensUpdated 19 days ago
    Auto-check passed

  • Provider Adapter Development

    AbdelStark/worldforge

    A skill your agent uses for WorldForge provider work: adding adapters, changing capability declarations, promoting scaffolds, debugging provider failures, updating provider catalog docs, or touching…

    108 GitHub stars~990 tokensUpdated 19 days ago
    Auto-check: notes

  • Tui Development

    AbdelStark/worldforge

    A skill your agent uses for the robotics showcase Textual UI: report panes, launch helpers, screenshots, visual tests, and changes under src/worldforge/harness/tui.py or robotics view/rendering…

    108 GitHub stars~792 tokensUpdated 19 days ago
    Auto-check passed

Questions about Evaluation Benchmarking

What does Evaluation Benchmarking do?

A skill your agent uses for WorldForge evaluation suites, benchmark harness changes, benchmark input fixtures, budget gates, report rendering, metrics semantics, and any claims based on benchmark or…. Evaluation Benchmarking is an agent skill from AbdelStark/worldforge. Use for WorldForge evaluation suites, benchmark harness changes, benchmark input fixtures, budget gates, report rendering, metrics semantics, and any claims based on benchmark or evaluation output.

When should I use Evaluation Benchmarking?

Evaluation Benchmarking fits situations like: worldForge evaluation suites; benchmark harness changes; benchmark input fixtures; report rendering.

How do I install Evaluation Benchmarking in Claude Code?

Run `npx skills add AbdelStark/worldforge --skill evaluation-benchmarking -a claude-code`. Or copy the skill folder (.codex/skills/evaluation-benchmarking in AbdelStark/worldforge) into .claude/skills/evaluation-benchmarking in your project. Claude Code loads it when a task matches its description.

How do I install Evaluation Benchmarking in Codex?

Run `npx skills add AbdelStark/worldforge --skill evaluation-benchmarking -a codex`. Or copy the skill folder (.codex/skills/evaluation-benchmarking in AbdelStark/worldforge) into .agents/skills/evaluation-benchmarking in your project. Codex loads it when a task matches its description.

Can I use Evaluation Benchmarking in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add AbdelStark/worldforge --skill evaluation-benchmarking -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/evaluation-benchmarking, .gemini/skills/evaluation-benchmarking, .github/skills/evaluation-benchmarking and .opencode/skills/evaluation-benchmarking in your project.

What does Evaluation Benchmarking need to run?

SKILL.md names no scripts, command-line tools or credentials: Evaluation Benchmarking is instructions for the agent only.

Does Evaluation Benchmarking access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Evaluation Benchmarking safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Evaluation Benchmarking use?

Evaluation Benchmarking is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Evaluation Benchmarking use?

About 732 tokens (SKILL.md is roughly 2.9k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Evaluation Benchmarking?

Skills that share tags, products or a category with Evaluation Benchmarking: Agent Benchmark Suite (ruvnet/ruflo, 74k stars), AI Agent Evaluation Benchmarking (sickn33/agentic-awesome-skills, 47k stars), LLM Benchmarking with lm-evaluation-harness (Orchestra-Research/AI-Research-SKILLs, 13k stars) and Benchmark (affaan-m/ECC, 275k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Evaluation Benchmarking?

AbdelStark (a GitHub user) maintains it in AbdelStark/worldforge, which has 108 GitHub stars. The repository holds 7 skills in this directory. The repository was last updated on September 19, 2026.

Source: AbdelStark/worldforge on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.