New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Ebench Setup

by InternRobotics in InternRobotics/EBench

Prepare or check an EBench evaluation environment for OpenPI, X-VLA, InternVLA-A1, or a custom policy.

MITAuto-check passed

Install Ebench Setup

skills CLI
$ npx skills add InternRobotics/EBench --skill ebench-setup -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install InternRobotics/EBench ebench-setup --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/InternRobotics/EBench.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/ebench-setup .claude/skills/ebench-setup && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
ebench-setup
GitHub stars
145
Token cost
~1k tokens
SKILL.md length
469 words
Files
1
Skills in repo
5
Repo updated
First seen
Licence
MIT

At a glance

Prepare or check an EBench evaluation environment for OpenPI, X-VLA, InternVLA-A1, or a custom policy.

  • First-run setup and baseline reproduction prerequisites
  • SKILL.md covers Establish the execution path, Baseline-specific checks and Verify and hand off
  • Calls git and python

What it does

Ebench Setup is an agent skill from InternRobotics/EBench. Prepare or check an EBench evaluation environment for OpenPI, X-VLA, InternVLA-A1, or a custom policy. Use for first-run setup and baseline reproduction prerequisites.

Its SKILL.md is about 1k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

The repository describes itself as: Elemental Diagnosis of Generalist Mobile Manipulation Policies. The licence is MIT.

When your agent uses it

  • First-run setup and baseline reproduction prerequisites

Example prompts

  • “/ebench-setup”

Requirements

  • Python 3

What it can do on your machine

Read from SKILL.md and the folder at commit 355fe56. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • git
    • python

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md. Its commands use git, which can reach the network depending on how they are called.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Ebench Setup loads about 1k tokens when it runs. Until then it costs about 45 tokens; SKILL.md has 469 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~45
When it runs · the whole SKILL.md, loaded when a task matches
~1k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from InternRobotics/EBench at commit 355fe56, republished under its MIT licence (© InternRobotics). 469 words, ~1,035 tokens.

Download SKILL.mdSave it as .claude/skills/ebench-setup/SKILL.md (or your agent's skills folder).
name
ebench-setup
description
Prepare or check an EBench evaluation environment for OpenPI, X-VLA, InternVLA-A1, or a custom policy. Use for first-run setup and baseline reproduction prerequisites.

Prepare EBench evaluation

Resolve paths from the EBench repository root. Read README.md, the selected baselines/<name>/README.md, and third_party/genmanip-client/pyproject.toml. Prefer the checked-out implementation when examples disagree with it.

Establish the execution path

  • Identify the model/checkpoint, online endpoint versus locally hosted GenManip, intended track/split, and available GPUs. Reuse information already supplied; ask only for missing inputs needed for the next action.
  • EBench contains adapters, not the Isaac Sim server. An online client does not need a local Isaac Sim installation. A local server requires a separate GenManip checkout and its environment/assets; inspect that checkout's instructions before proposing server commands.
  • Inspect git submodule status --recursive, the active Python interpreter, installed package locations, GPU availability, and checkpoint existence. Initialize missing pinned submodules with git submodule update --init --recursive when setup is requested; do not advance them to arbitrary upstream heads.
  • Install the pinned client in the selected model/client environment with python -m pip install -e third_party/genmanip-client. Choose full_numpy1 or full_numpy2 extras to match the baseline environment; do not install both. Keep incompatible baseline dependencies in separate environments.

Baseline-specific checks

BaselineSource of truthChecks that change the launch plan
X-VLAbaselines/X-VLA/run.py, scripts/run_xvla_eval.shCheck matching processor/checkpoint and requirements; worker IDs and GPU IDs are separate settings.
OpenPIbaselines/openpi/README.md, baselines/openpi/src/openpi/training/config.py, baselines/openpi/src/openpi/policies/ebench_policy.py, baselines/openpi/scripts/pi_eval_client_online.pyApply EBench overlays together; verify the selected config actually exists and uses checkpoint-compatible normalization. Model WebSocket server and eval client have separate endpoints/environments.
InternVLA-A1baselines/InternVLA-A1/inference.py, baselines/InternVLA-A1/eval_pjsim.shCheck upstream dependencies, checkpoint config/weights/stats.json, and the statistics key. Verify a documented wrapper actually exists before using it.

scripts/launch_pi_onlineeval.sh contains placeholder paths, activation commands, and settings; it is a template, not a ready-to-run launcher. Inspect shell wrappers before execution. Create a concrete local launch command using the user's paths rather than running placeholders or copying another machine's paths.

Show full SKILL.md (180 more words)Show less

Verify and hand off

For a new online evaluation, collect the platform URL and locally configured API token; the user does not need an evaluation endpoint or task ID yet. Check gmp online submit --help and availability of jq if using the shell example in ebench-evaluate. After local prerequisites pass, an evaluation request proceeds through gmp online submit → queue/readiness → returned endpoint and task_id → model launch using that endpoint and run_id=task_id. A setup-only request stops at reporting readiness. Reuse an existing task when supplied.

Check imports and gmp --help in the selected environment before allocating a full run. Inspect CLI source if imports are unavailable. A connectivity test with gmp eval uses fake actions and is not a model evaluation; label it accordingly and keep its run separate from reported model results.

Report the interpreter/environment, pinned revisions, model path/config, dependency gaps, and launch command with credential placeholders. Distinguish checks actually run from inferred compatibility. Do not claim readiness while weights, server access, or imports remain unverified. If asked to proceed with evaluation, continue within the existing request once prerequisites are satisfied.

© InternRobotics, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/ebench-setup of InternRobotics/EBench.

Open the folder on GitHubat commit 355fe56

Compare with similar skills

Ebench Setup next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Ebench Setup compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Ebench Setup this skillInternRobotics/EBench145—~1kAutomated safety check: PassMIT
Arize Evaluatorgithub/awesome-copilot40k2 repos~8.1kAutomated safety check: NotesMIT
LLM Evaluationdavila7/claude-code-templates32k13 repos~3.5kAutomated safety check: PassMIT
Agent Evaluationsickn33/agentic-awesome-skills47k1 repos~2kAutomated safety check: PassMIT
EvaluatorsArize-ai/phoenix12k—~1.7kAutomated safety check: PassCustom licence
Agent Evaluation Reportingsickn33/agentic-awesome-skills47k1 repos~2.1kAutomated safety check: PassMIT

Similar skills

  • Arize Evaluator

    github/awesome-copilot

    Official

    Handles LLM-as-judge evaluation workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and…

    40k GitHub starsUsed in 2 repos~8.1k tokens
    AI & LLM EngineeringAuto-check: notes

  • LLM Evaluation

    davila7/claude-code-templates

    Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.

    32k GitHub starsUsed in 13 repos~3.5k tokens
    AI & LLM EngineeringAuto-check passed

  • Agent Evaluation

    sickn33/agentic-awesome-skills

    Evaluate agent behavior with versioned cases and explicit verifiers.

    47k GitHub starsUsed in 1 repo~2k tokens
    Agent WorkflowsAuto-check passed

  • Evaluators

    Arize-ai/phoenix

    Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output.

    12k GitHub stars~1.7k tokensUpdated today
    EducationAuto-check passed

  • Agent Evaluation Reporting

    sickn33/agentic-awesome-skills

    A skill your agent uses when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.

    47k GitHub starsUsed in 1 repo~2.1k tokens
    Agent WorkflowsAuto-check passed

  • Audit Preparation

    sickn33/agentic-awesome-skills

    Audit preparation register: required document, period covered, request and receipt dates, preparer and reviewer, auditor queries and adjustments.

    47k GitHub starsUsed in 1 repo~5.3k tokens
    Legal & ComplianceAuto-check passed

More from InternRobotics/EBench

  • Ebench Analyze

    InternRobotics/EBench

    Generate and interpret EBench evaluation reports, compare runs and baselines, and diagnose capability or generalization gaps with explicit data coverage and aggregation semantics.

    145 GitHub stars~1.1k tokensUpdated 13 days ago
    Auto-check passed

  • Ebench Evaluate

    InternRobotics/EBench

    Run and monitor an EBench policy evaluation against a local GenManip server or the online service, including baseline launch commands, worker allocation, and reproducible run records.

    145 GitHub stars~1.9k tokensUpdated 13 days ago
    Auto-check passed

  • Ebench Integrate Policy

    InternRobotics/EBench

    Implement or review a custom VLA policy adapter for EBench EvalClient, including observation preprocessing, action semantics, chunking, and episode resets.

    145 GitHub stars~1.3k tokensUpdated 13 days ago
    Auto-check passed

  • Ebench Debug

    InternRobotics/EBench

    Diagnose EBench evaluation failures, stalled workers, transport errors, invalid actions, and unexpectedly low scores using logs and episode artifacts.

    145 GitHub stars~862 tokensUpdated 13 days ago
    Auto-check passed

Questions about Ebench Setup

What does Ebench Setup do?

Prepare or check an EBench evaluation environment for OpenPI, X-VLA, InternVLA-A1, or a custom policy. Ebench Setup is an agent skill from InternRobotics/EBench. Prepare or check an EBench evaluation environment for OpenPI, X-VLA, InternVLA-A1, or a custom policy.

When should I use Ebench Setup?

Ebench Setup fits situations like: first-run setup and baseline reproduction prerequisites.

How do I install Ebench Setup in Claude Code?

Run `npx skills add InternRobotics/EBench --skill ebench-setup -a claude-code`. Or copy the skill folder (skills/ebench-setup in InternRobotics/EBench) into .claude/skills/ebench-setup in your project. Claude Code loads it when a task matches its description.

How do I install Ebench Setup in Codex?

Run `npx skills add InternRobotics/EBench --skill ebench-setup -a codex`. Or copy the skill folder (skills/ebench-setup in InternRobotics/EBench) into .agents/skills/ebench-setup in your project. Codex loads it when a task matches its description.

Can I use Ebench Setup in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add InternRobotics/EBench --skill ebench-setup -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/ebench-setup, .gemini/skills/ebench-setup, .github/skills/ebench-setup and .opencode/skills/ebench-setup in your project.

What does Ebench Setup need to run?

Going by SKILL.md and its folder, Ebench Setup needs the command-line tools its instructions call (git and python). Our summary lists: Python 3.

Does Ebench Setup access the network?

SKILL.md contains no URLs. Its commands use git, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is Ebench Setup safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Ebench Setup use?

Ebench Setup is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Ebench Setup use?

About 1k tokens (SKILL.md is roughly 4.1k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Ebench Setup?

Skills that share tags, products or a category with Ebench Setup: Arize Evaluator (github/awesome-copilot, 40k stars), LLM Evaluation (davila7/claude-code-templates, 32k stars), Agent Evaluation (sickn33/agentic-awesome-skills, 47k stars) and Evaluators (Arize-ai/phoenix, 12k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Ebench Setup?

InternRobotics (a GitHub organization) maintains it in InternRobotics/EBench, which has 145 GitHub stars. The repository holds 5 skills in this directory. The repository was last updated on September 24, 2026.

Source: InternRobotics/EBench on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.