New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Evaluation Tools

by VectorSpaceLab in VectorSpaceLab/AREX-Skill

A skill your agent uses when evaluating RobustVideoMatting alpha/foreground predictions, preparing LR or HR metric directories, generating evaluation composites, or interpreting speed benchmarks.

GPL-3.0Auto-check passed

Install Evaluation Tools

skills CLI
$ npx skills add VectorSpaceLab/AREX-Skill --skill evaluation-tools -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install VectorSpaceLab/AREX-Skill evaluation-tools --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/VectorSpaceLab/AREX-Skill.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/repositories/repo-skills/robust-video-matting/sub-skills/evaluation-tools .claude/skills/evaluation-tools && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
evaluation-tools
GitHub stars
328
Token cost
~933 tokens
SKILL.md length
378 words
Files
4 (incl. scripts, references)
Skills in repo
157
Repo updated
First seen
Licence
GPL-3.0

At a glance

A skill your agent uses when evaluating RobustVideoMatting alpha/foreground predictions, preparing LR or HR metric directories, generating evaluation composites, or interpreting speed benchmarks.

  • Works in 5 steps: Ensure predictions and ground truth use… → Choose LR or HR flow → For a safe tiny assertion check, use the… → …
  • Evaluating RobustVideoMatting alpha/foreground predictions
  • SKILL.md covers Read this when, Evaluation workflow, Bundled references and script and Key decisions, plus 1 more section
  • Runs Python scripts from its folder; calls python

What it does

Evaluation Tools is an agent skill from VectorSpaceLab/AREX-Skill. Use when evaluating RobustVideoMatting alpha/foreground predictions, preparing LR or HR metric directories, generating evaluation composites, or interpreting speed benchmarks.

Its SKILL.md is about 930 tokens, which your agent loads only when the skill is triggered. The skill folder holds 5 other files, including scripts and reference files (for example `references/evaluation-reference.md`, `references/troubleshooting.md` and `scripts/rvm_evaluate_lr_tiny.py`).

The repository describes itself as: A Skill Library for Automated Machine Learning. The licence is GPL-3.0.

When your agent uses it

  • Evaluating RobustVideoMatting alpha/foreground predictions
  • HR metric directories
  • Generating evaluation composites
  • Interpreting speed benchmarks

Example prompts

  • “/evaluation-tools”

Requirements

  • Python 3

Workflow steps

5 steps, taken from the first numbered list in SKILL.md.

  1. Ensure predictions and ground truth use the exact same dataset/clip/frame
  2. Choose LR or HR flow
  3. For a safe tiny assertion check, use the bundled JSON evaluator
  4. For full repo-style evaluation, mirror the metric names and directory
  5. If the user asks for speed, separate tensor throughput from media IO. The

What it can do on your machine

Read from SKILL.md and the folder at commit ac3fe1a. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships 1 file in scripts/ (Python), which the agent can run.

    Shell commands in SKILL.md call:

    • python

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Evaluation Tools loads about 933 tokens when it runs, and up to ~2.7k if it reads all its reference files. Until then it costs about 48 tokens; SKILL.md has 378 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~48
When it runs · the whole SKILL.md, loaded when a task matches
~933
With references · SKILL.md plus every file in references/, read only if the agent opens them
~2.7k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from VectorSpaceLab/AREX-Skill at commit ac3fe1a, republished under its GPL-3.0 licence (© VectorSpaceLab). 378 words, ~933 tokens.

Download SKILL.mdSave it as .claude/skills/evaluation-tools/SKILL.md (or your agent's skills folder). This skill also uses 3 other files; get the full folder from GitHub.
name
evaluation-tools
description
Use when evaluating RobustVideoMatting alpha/foreground predictions, preparing LR or HR metric directories, generating evaluation composites, or interpreting speed benchmarks.
disable-model-invocation
true
metadata.disco-role
operating
license
GPL 3.0

RobustVideoMatting Evaluation Tools

Use this sub-skill when the task is about measuring RVM predictions, preparing prediction/ground-truth trees, understanding LR versus HR metrics, or explaining published speed numbers.

Read this when

  • The user asks to compute pha_mad, pha_mse, pha_grad, pha_conn, pha_dtssd, fgr_mad, or fgr_mse.
  • The task mentions evaluate_lr.py, evaluate_hr.py, videomatte_512x288, videomatte_1920x1080, or Excel metric output.
  • The user needs to generate synthetic evaluation composites from matte and background datasets.
  • The user asks why converter FPS differs from the README speed table.

Route other tasks elsewhere:

Evaluation workflow

  1. Ensure predictions and ground truth use the exact same dataset/clip/frame tree. The evaluator expects alpha under pha/ and foreground under fgr/ when foreground metrics are selected.

  2. Choose LR or HR flow:

    • LR evaluation is CPU/NumPy/OpenCV oriented and includes pha_conn.
    • HR evaluation hardcodes CUDA tensors and Kornia for high-resolution gradient metrics.

  3. For a safe tiny assertion check, use the bundled JSON evaluator:

    bash
    python scripts/rvm_evaluate_lr_tiny.py \
  --pred-dir pred/videomatte_512x288 \
  --true-dir true/videomatte_512x288 \
  --metrics pha_mad pha_mse pha_dtssd fgr_mad fgr_mse

  4. For full repo-style evaluation, mirror the metric names and directory constraints in references/evaluation-reference.md.

  5. If the user asks for speed, separate tensor throughput from media IO. The README speed table is based on a CUDA tensor loop, not the Python converter's full decode/encode pipeline.

Show full SKILL.md (166 more words)Show less

Bundled references and script

  • Read references/evaluation-reference.md for LR/HR metrics, directory schemas, compositing scripts, and speed-test caveats.
  • Read references/troubleshooting.md for empty result directories, frame mismatches, missing dependencies, HR CUDA-only behavior, deprecated NumPy aliases, and foreground-mask issues.
  • Run scripts/rvm_evaluate_lr_tiny.py for a self-contained CPU JSON summary on small prediction/ground-truth folders. It intentionally does not depend on the original checkout.

Key decisions

  • Use LR/tiny evaluation for quick validation of directory matching and metric plumbing.
  • Use HR evaluation only in a CUDA environment with Kornia and matching high-resolution data; it is not a CPU fallback.
  • Do not run synthetic composite generation as a default check. Those scripts require large external matte/background datasets and can create many files.
  • If foreground metrics are requested, require matching fgr/ frames and a non-empty alpha mask.

Acceptance check for evaluation answers

A good answer states the exact directory shape, names the metrics and runtime requirements, checks frame-name equality, chooses LR/HR appropriately, and explains whether any skipped compositing or CUDA benchmark is outside the safe verification scope.

© VectorSpaceLab, GPL-3.0. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 3 other files (scripts, references) in skills/repositories/repo-skills/robust-video-matting/sub-skills/evaluation-tools of VectorSpaceLab/AREX-Skill.

  • SKILL.md
  • references/evaluation-reference.md
  • references/troubleshooting.md
  • scripts/rvm_evaluate_lr_tiny.py

Open the folder on GitHubat commit ac3fe1a

Compare with similar skills

Evaluation Tools next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Evaluation Tools compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Evaluation Tools this skillVectorSpaceLab/AREX-Skill328—~933Automated safety check: PassGPL-3.0
Arize Evaluatorgithub/awesome-copilot40k2 repos~8.1kAutomated safety check: NotesMIT
LLM Evaluationdavila7/claude-code-templates32k13 repos~3.5kAutomated safety check: PassMIT
Agent Evaluationsickn33/agentic-awesome-skills47k1 repos~2kAutomated safety check: PassMIT
EvaluatorsArize-ai/phoenix12k—~1.7kAutomated safety check: PassCustom licence
Agent Evaluation Reportingsickn33/agentic-awesome-skills47k1 repos~2.1kAutomated safety check: PassMIT

Similar skills

  • Arize Evaluator

    github/awesome-copilot

    Official

    Handles LLM-as-judge evaluation workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and…

    40k GitHub starsUsed in 2 repos~8.1k tokens
    AI & LLM EngineeringAuto-check: notes

  • LLM Evaluation

    davila7/claude-code-templates

    Master comprehensive evaluation strategies for LLM applications, from automated metrics to human evaluation and A/B testing.

    32k GitHub starsUsed in 13 repos~3.5k tokens
    AI & LLM EngineeringAuto-check passed

  • Agent Evaluation

    sickn33/agentic-awesome-skills

    Evaluate agent behavior with versioned cases and explicit verifiers.

    47k GitHub starsUsed in 1 repo~2k tokens
    Agent WorkflowsAuto-check passed

  • Evaluators

    Arize-ai/phoenix

    Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output.

    12k GitHub stars~1.7k tokensUpdated today
    EducationAuto-check passed

  • Agent Evaluation Reporting

    sickn33/agentic-awesome-skills

    A skill your agent uses when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.

    47k GitHub starsUsed in 1 repo~2.1k tokens
    Agent WorkflowsAuto-check passed

  • Audit Preparation

    sickn33/agentic-awesome-skills

    Audit preparation register: required document, period covered, request and receipt dates, preparer and reviewer, auditor queries and adjustments.

    47k GitHub starsUsed in 1 repo~5.3k tokens
    Legal & ComplianceAuto-check passed

More from VectorSpaceLab/AREX-Skill

All 157 skills in this repo

  • Agent Lightning

    VectorSpaceLab/AREX-Skill

    Use this repo skill for Agent Lightning package tasks: authoring trainable agents, tracing rewards and spans, running LightningStore/Trainer loops, using agl CLI services, choosing examples, and…

    328 GitHub stars~1.4k tokensUpdated 1 mo ago
    Auto-check passed

  • Agent Tools

    VectorSpaceLab/AREX-Skill

    A skill your agent uses when configuring LiteLLM for MCP tools, A2A agents, Claude Code/Cursor agent gateway traffic, MCP auth/OAuth, tool permissions, semantic filtering, or agent-specific proxy…

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents And Awel

    VectorSpaceLab/AREX-Skill

    Build and debug DB-GPT agents, tools, skills, teams, and AWEL workflows, including deterministic local DAG runs and HTTP-trigger topology without assuming an LLM, credential, or external service.

    328 GitHub stars~2.3k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents And Middleware

    VectorSpaceLab/AREX-Skill

    Work on the actively maintained LangChain v1 agent package: initchatmodel, createagent, structured output, tools, middleware, embeddings initialization, provider routing, and agent runtime…

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

  • Agents Workflows

    VectorSpaceLab/AREX-Skill

    A skill your agent uses for giskard.agents async chat workflows, tools, prompt templates, structured outputs, retries, rate limiting, embeddings, and optional LiteLLM backend.

    328 GitHub stars~500 tokensUpdated 1 mo ago
    Auto-check passed

  • Alphafold3

    VectorSpaceLab/AREX-Skill

    A skill your agent uses for AlphaFold 3 input preparation, prediction command planning, output interpretation, and Python API inspection.

    328 GitHub stars~1.2k tokensUpdated 1 mo ago
    Auto-check passed

Questions about Evaluation Tools

What does Evaluation Tools do?

A skill your agent uses when evaluating RobustVideoMatting alpha/foreground predictions, preparing LR or HR metric directories, generating evaluation composites, or interpreting speed benchmarks. Evaluation Tools is an agent skill from VectorSpaceLab/AREX-Skill. Use when evaluating RobustVideoMatting alpha/foreground predictions, preparing LR or HR metric directories, generating evaluation composites, or interpreting speed benchmarks.

When should I use Evaluation Tools?

Evaluation Tools fits situations like: evaluating RobustVideoMatting alpha/foreground predictions; HR metric directories; generating evaluation composites; interpreting speed benchmarks.

How do I install Evaluation Tools in Claude Code?

Run `npx skills add VectorSpaceLab/AREX-Skill --skill evaluation-tools -a claude-code`. Or copy the skill folder (skills/repositories/repo-skills/robust-video-matting/sub-skills/evaluation-tools in VectorSpaceLab/AREX-Skill) into .claude/skills/evaluation-tools in your project. Claude Code loads it when a task matches its description.

How do I install Evaluation Tools in Codex?

Run `npx skills add VectorSpaceLab/AREX-Skill --skill evaluation-tools -a codex`. Or copy the skill folder (skills/repositories/repo-skills/robust-video-matting/sub-skills/evaluation-tools in VectorSpaceLab/AREX-Skill) into .agents/skills/evaluation-tools in your project. Codex loads it when a task matches its description.

Can I use Evaluation Tools in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add VectorSpaceLab/AREX-Skill --skill evaluation-tools -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/evaluation-tools, .gemini/skills/evaluation-tools, .github/skills/evaluation-tools and .opencode/skills/evaluation-tools in your project.

What does Evaluation Tools need to run?

Going by SKILL.md and its folder, Evaluation Tools needs Python for the scripts in its folder and the command-line tools its instructions call (python). Our summary lists: Python 3.

Does Evaluation Tools access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Evaluation Tools safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.

What licence does Evaluation Tools use?

Evaluation Tools is published under the GPL-3.0 licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Evaluation Tools use?

About 933 tokens (SKILL.md is roughly 3.7k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 1.8k tokens, read only when the agent opens those files.

What are the alternatives to Evaluation Tools?

Skills that share tags, products or a category with Evaluation Tools: Arize Evaluator (github/awesome-copilot, 40k stars), LLM Evaluation (davila7/claude-code-templates, 32k stars), Agent Evaluation (sickn33/agentic-awesome-skills, 47k stars) and Evaluators (Arize-ai/phoenix, 12k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Evaluation Tools?

VectorSpaceLab (a GitHub organization) maintains it in VectorSpaceLab/AREX-Skill, which has 328 GitHub stars. The repository holds 157 skills in this directory. The repository was last updated on September 3, 2026.

Source: VectorSpaceLab/AREX-Skill on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.