Official agent skill

RAG Eval

Filesystem RAG benchmarks: corpus/, train.json, evaluaterag.py (RAGAS quality).

OfficialApache-2.0Auto-check: notesAI & LLM Engineering

Install RAG Eval

skills CLI

$ npx skills add NVIDIA/skills --skill rag-eval -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills rag-eval --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/rag-eval .claude/skills/rag-eval && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Paste into Claude Code

Install the "rag-eval" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/rag-eval into .claude/skills/rag-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "rag-eval", then confirm the skill loads.

Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

$skill-installer

$skill-installer install https://github.com/NVIDIA/skills/tree/main/skills/rag-eval

Type this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.

skills CLI

$ npx skills add NVIDIA/skills --skill rag-eval -a codex

Project install goes to .agents/skills/; add -g for ~/.codex/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills rag-eval --agent codex

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/rag-eval .agents/skills/rag-eval && rm -rf skills-src

Use ~/.agents/skills/ instead of .agents/skills for a personal install.

Codex skills documentation · loads skills from .agents/skills/

Paste into Codex

Install the "rag-eval" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/rag-eval into .agents/skills/rag-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "rag-eval", then confirm the skill loads.

Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add NVIDIA/skills --skill rag-eval -a cursor

Project install goes to .agents/skills/; add -g for ~/.cursor/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills rag-eval --agent cursor

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/rag-eval .cursor/skills/rag-eval && rm -rf skills-src

Use ~/.cursor/skills/ instead of .cursor/skills for a personal install.

Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/

Paste into Cursor

Install the "rag-eval" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/rag-eval into .cursor/skills/rag-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "rag-eval", then confirm the skill loads.

Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

gemini skills install

$ gemini skills install https://github.com/NVIDIA/skills.git --path skills/rag-eval

--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.

skills CLI

$ npx skills add NVIDIA/skills --skill rag-eval -a gemini-cli

Project install goes to .agents/skills/; add -g for ~/.gemini/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills rag-eval --agent gemini-cli

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/rag-eval .gemini/skills/rag-eval && rm -rf skills-src

Use ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.

Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/

Paste into Gemini CLI

Install the "rag-eval" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/rag-eval into .gemini/skills/rag-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "rag-eval", then confirm the skill loads.

Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

GitHub CLI

$ gh skill install NVIDIA/skills rag-eval

Installs for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).

skills CLI

$ npx skills add NVIDIA/skills --skill rag-eval -a github-copilot

Project install goes to .agents/skills/; add -g for ~/.copilot/skills/.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/rag-eval .github/skills/rag-eval && rm -rf skills-src

Use ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.

GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/

Paste into GitHub Copilot

Install the "rag-eval" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/rag-eval into .github/skills/rag-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "rag-eval", then confirm the skill loads.

GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add NVIDIA/skills --skill rag-eval -a opencode

OpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.

GitHub CLI

$ gh skill install NVIDIA/skills rag-eval --agent opencode

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/NVIDIA/skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/rag-eval .opencode/skills/rag-eval && rm -rf skills-src

Use ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.

OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/

Paste into OpenCode

Install the "rag-eval" agent skill from https://github.com/NVIDIA/skills/tree/main/skills/rag-eval into .opencode/skills/rag-eval/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "rag-eval", then confirm the skill loads.

OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

Facts

Skill name: rag-eval
GitHub stars: 3.5k
Token cost: ~2.3k tokens
SKILL.md length: 721 words
Files: 10 (incl. references)
Skills in repo: 380
Repo updated: Oct 7, 2026
First seen: Oct 7, 2026
Licence: Apache-2.0

At a glance

Filesystem RAG benchmarks: corpus/, train.json, evaluaterag.py (RAGAS quality).

Works in 5 steps: Prepare data — Ensure each dataset… → Run eval — uv run --project scripts/eval… → Tune quality — Adjust --top_k /… → …
Tasks that involve Retrieval-augmented generation
SKILL.md covers Purpose, When not to use, Prerequisites and Instructions, plus 6 more sections
Calls uv and python3; needs NVIDIA_API_KEY

What it does

RAG Eval is an agent skill from NVIDIA/skills, published by the product's own GitHub organization. Filesystem RAG benchmarks: corpus/, train.json, evaluaterag.py (RAGAS quality). Not for prod monitoring, latency/throughput benchmarking (use rag-perf), or evals outside this repo layout.

Its SKILL.md is about 2.3k tokens, which your agent loads only when the skill is triggered. The skill folder holds 11 other files, including reference files (for example `BENCHMARK.md`, `eval/h100.json` and `eval/nvidia_hosted.json`). Compatibility notes: Repository checkout with uv; Python 3.11+; run from repo root; uv sync --project scripts/eval (eval deps live in scripts/eval/pyproject.toml); network to RAG…

It sits in AI & LLM Engineering, covering Retrieval-augmented generation and LLM evaluation. It works with NVIDIA AI Platform. The repository describes itself as: Agent Skills for NVIDIA products — install into Claude Code, Codex, and other coding agents to run Physical AI, robotics, simulation, CUDA, and RAG workflows end to end. The licence is Apache-2.0.

When your agent uses it

Tasks that involve Retrieval-augmented generation
Tasks that involve LLM evaluation

Example prompts

“/rag-eval”

Requirements

Python 3
A credential in NVIDIA_API_KEY
Compatibility (from SKILL.md): Repository checkout with uv; Python 3.11+; run from repo root; uv sync --project scripts/eval (eval deps live in scripts/eval/pyproject.toml); network to RAG, ingestor, and vdb endpoints; NVIDIA_API_KEY for RAGAS; optional RAG_EVAL_JUDGE_MODEL (default mistralai/mixtral-8x22b-instruct-v0.1).
Pre-approved tools (allowed-tools): Read, Grep, Glob, Bash(ls *), Bash(python3 *), Bash(uv *), Write, Edit

Workflow steps

5 steps, taken from the first numbered list in SKILL.md.

Prepare data — Ensure each dataset directory matches the layout and train.json rules in references/dataset-and-conversion.md. When sources…
Run eval — uv run --project scripts/eval python scripts/eval/evaluate_rag.py with --dataset-paths, --host, and --port. See…
Tune quality — Adjust --top_k / --vdb_top_k, reranker and query-rewriting toggles, and generation overrides (--temperature, --top-p…
Analyze results — Use references/result-analysis.md for scripts; scan rag_*_evaluation_summary.json for headline RAGAS metrics.
Triage errors — Use the error signal table and the Troubleshooting section below.

What it can do on your machine

Read from SKILL.md and the folder at commit 0e0d506. It shows what the files ask for, not the result of running them.

Tool permissions
Pre-approves these tools, so the agent can use them without asking each time:
- Read
- Grep
- Glob
- Bash(ls *)
- Bash(python3 *)
- Bash(uv *)
- Write
- Edit
From allowed-tools in the SKILL.md frontmatter.
Runs code
Shell commands in SKILL.md call:
- uv
- python3
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md. Its commands use uv, which can reach the network depending on how they are called.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names these keys or tokens, usually read from environment variables:
- NVIDIA_API_KEY
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Compatibility
Repository checkout with uv; Python 3.11+; run from repo root; uv sync --project scripts/eval (eval deps live in scripts/eval/pyproject.toml); network to RAG, ingestor, and vdb endpoints; NVIDIA_API_KEY for RAGAS; optional RAG_EVAL_JUDGE_MODEL (default mistralai/mixtral-8x22b-instruct-v0.1).
From compatibility in the SKILL.md frontmatter.

Context cost

RAG Eval loads about 2.3k tokens when it runs, and up to ~8.3k if it reads all its reference files. Until then it costs about 49 tokens; SKILL.md has 721 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it: ~49
When it runs · the whole SKILL.md, loaded when a task matches: ~2.3k
With references · SKILL.md plus every file in references/, read only if the agent opens them: ~8.3k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check: notes

The automated check noted patterns worth knowing about, such as sudo or a known installer.

NoteMentions a .env fileSKILL.md:64

le or secrets manager; avoid committing `.env`; rotate keys if exposed. Details: [`references/benchmark-execution.md#cre

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from NVIDIA/skills at commit 0e0d506, republished under its Apache-2.0 licence (© NVIDIA). 721 words, ~2,346 tokens.

Download SKILL.mdSave it as .claude/skills/rag-eval/SKILL.md (or your agent's skills folder). This skill also uses 9 other files; get the full folder from GitHub.

name: rag-eval
description: Filesystem RAG benchmarks: corpus/, train.json, evaluate_rag.py (RAGAS quality). Not for prod monitoring, latency/throughput benchmarking (use rag-perf), or evals outside this repo layout.
allowed-tools: Read, Grep, Glob, Bash(ls *), Bash(python3 *), Bash(uv *), Write, Edit
compatibility: Repository checkout with uv; Python 3.11+; run from repo root; uv sync --project scripts/eval (eval deps live in scripts/eval/pyproject.toml); network to RAG, ingestor, and vdb endpoints; NVIDIA_API_KEY for RAGAS; optional RAG_EVAL_JUDGE_MODEL (default mistralai/mixtral-8x22b-instruct-v0.1).
version: 2.6.0
license: Apache-2.0
metadata.author: NVIDIA RAG <foundational-rag-dev@exchange.nvidia.com>
metadata.github-url: https://github.com/NVIDIA-AI-Blueprints/rag
metadata.endpoint-openapi-schemas: docs/api_reference/openapi_schema_rag_server.json, docs/api_reference/openapi_schema_ingestor_server.json
metadata.argument-hint: RAGAS eval | evaluate_rag | train.json | corpus | results json | error triage | uv run --project scripts/eval | enable_reranker | query_rewriting |…
metadata.tags: nvidia, blueprint, rag, evaluation, ragas, benchmarking, nvidia-rag-blueprint
metadata.languages: python, shell
metadata.frameworks: ragas, fastapi
metadata.domain: ai-ml

On-disk RAG evaluation (`corpus/` + `train.json`)

Purpose

Guide agents through NVIDIA RAG Blueprint filesystem benchmarks: preparing corpus/ and train.json, running scripts/eval/evaluate_rag.py, tuning retrieval and generation flags for quality comparisons, interpreting RAGAS JSON outputs, and triaging failures (HTTP/stream errors, empty contexts, collection mismatch, judge API).

For latency, throughput, and load testing, use the rag-perf skill (scripts/rag-perf, docs/performance-benchmarking.md) — not this skill.

When not to use

Do not use this skill for: deploying or repairing services (use rag-blueprint); evaluating APIs without the corpus/ + train.json layout; general ML experimentation unrelated to this evaluator; production monitoring/alerting; or latency/throughput benchmarking (use rag-perf).

Prerequisites

Repo cloned; run commands from repo root (imports and paths assume this).
Python 3.11+ and uv; eval deps: uv sync --project scripts/eval.
Reachable RAG server and ingestor (defaults often localhost:8081 / 8082).
NVIDIA_API_KEY for RAGAS (see credential hygiene); optional RAG_EVAL_JUDGE_MODEL.
Dataset roots passed to --dataset-paths each contain corpus/ and train.json.

Instructions

Prepare data — Ensure each dataset directory matches the layout and train.json rules in references/dataset-and-conversion.md. When sources arrive as public links (sites or dataset pages), materialize documents under corpus/—prefer PDF for multimodal content so images stay embedded; convert CSV/JSONL/etc. using the patterns there.
Run eval — uv run --project scripts/eval python scripts/eval/evaluate_rag.py with --dataset-paths, --host, and --port. See references/benchmark-execution.md for command examples, outputs, and errors. Use references/evaluate-rag-cli.md for flag-level detail.
Tune quality — Adjust --top_k / --vdb_top_k, reranker and query-rewriting toggles, and generation overrides (--temperature, --top-p, --max-tokens) as documented in references/benchmark-execution.md when comparing retrieval/generation configs for RAGAS scores.
Analyze results — Use references/result-analysis.md for scripts; scan rag_*_evaluation_summary.json for headline RAGAS metrics.
Triage errors — Use the error signal table and the Troubleshooting section below.

Examples

Set API key without putting secrets in shell history (preferred patterns): load from a gitignored env file or secrets manager; avoid committing .env; rotate keys if exposed. Details: references/benchmark-execution.md#credential-hygiene-nvidia_api_key.

Minimal eval (key already in environment):

bash

uv sync --project scripts/eval
uv run --project scripts/eval python scripts/eval/evaluate_rag.py \
  --dataset-paths /path/to/my_dataset \
  --host localhost \
  --port 8081

Pretty-print summary JSON:

bash

python3 -m json.tool results/my_dataset/rag_my_dataset_evaluation_summary.json

More examples (skip ingestion, quality sweeps): references/benchmark-execution.md.

Limitations

Evaluator behavior is fixed to the filesystem contract and evaluate_rag.py; it does not substitute for custom offline judges or non-RAG benchmarks.
Vector DB / embedding choices follow deployed ingestor and RAG env — not overridden by this CLI alone.
Scores depend on retrieval quality, judge model availability, and NVIDIA_API_KEY; empty contexts yield partial RAGAS metrics (see references).
Large procedural detail lives under references/ to keep routing concise; read those files when the user needs step-by-step conversion, full flags, or error tables.

Show full SKILL.md (326 more words)Show less

Troubleshooting

Error / signal	Likely cause	What to do
Immediate exit mentioning `NVIDIA_API_KEY`	Missing or invalid key	Set key via secure channel; see credential hygiene in `references/benchmark-execution.md`.
`train.json must be a JSON array`	Wrong JSON shape	Top-level array of objects; validate per `references/dataset-and-conversion.md`.
Fewer rows in `evaluation_data.json` than `train.json`	Per-query failures	Check stderr: network or stream JSON errors; see error table in benchmark-execution.
Empty `generated_contexts` everywhere	Retrieval gap	Verify collection, ingestion, `top_k` / `vdb_top_k`, and `ingestor_server_url` without `/v1` suffix.
Ingestor 404 on upload	Bad ingestor base URL	Pass `http://host:port` only — code appends `/v1/`.

Full signal table: references/benchmark-execution.md#common-error-cases-and-signals.

Gotchas

Run from repo root: paths and imports in scripts/eval/evaluate_rag.py assume this; a wrong directory silently breaks imports.
--ingestor_server_url: pass http://host:port without /v1—the code appends /v1/ automatically. Including /v1 causes 404s on ingestor calls.
Vector DB / embedding settings: not set by this CLI; configure via the deployed ingestor and RAG server env vars (e.g. APP_VECTORSTORE_URL, embedding model).
--model / --llm_endpoint: forwarded verbatim only when explicitly set; omit to keep the server's configured LLM.
Stale collections: a previous run's ingested data persists unless you use --force_ingestion. Use --collection with a unique name when comparing quality across isolated runs.
Empty context metrics: if all generated_contexts are empty, RAGAS scores only nv_accuracy and leaves the other two metrics blank—this is not a silent success.

Source of truth

Piece	Location
Driver	`scripts/eval/evaluate_rag.py` (`CORPUS_DIRECTORY` = `corpus`, `EVAL_DATA` = `train.json`)
Human README (always in-repo)	`scripts/eval/README.md`
Full CLI (flags, defaults)	`scripts/eval/evaluate_rag.py --help`; `references/evaluate-rag-cli.md`
Dataset / conversion	`references/dataset-and-conversion.md`
Runs, outputs, errors	`references/benchmark-execution.md`
Result analysis scripts	`references/result-analysis.md`
Latency / throughput	rag-perf skill, `docs/performance-benchmarking.md`

Agent playbook

Run eval — uv sync --project scripts/eval then uv run --project scripts/eval python scripts/eval/evaluate_rag.py with required --dataset-paths, --host, and --port (and env NVIDIA_API_KEY). Argument --ingestor_server_url is optional (defaults to http://localhost:8082); pass it only when overriding the ingestor endpoint.
Quality tuning — See references/benchmark-execution.md: --top_k/--vdb_top_k, reranker and query-rewriting toggles, --temperature, --top-p, --max-tokens.
Data conversion — Follow references/dataset-and-conversion.md.
Analyze results — references/result-analysis.md; quick scan: python3 -m json.tool results/<dataset>/rag_<dataset>_evaluation_summary.json.
Error triage — references/benchmark-execution.md#common-error-cases-and-signals.

Files

SKILL.md and 9 other files (references) in skills/rag-eval of NVIDIA/skills.

SKILL.md
BENCHMARK.md
eval/h100.json
eval/nvidia_hosted.json
references/benchmark-execution.md
references/dataset-and-conversion.md
references/evaluate-rag-cli.md
references/result-analysis.md
skill-card.md
skill.oms.sig

Open the folder on GitHubat commit 0e0d506

Compare with similar skills

RAG Eval next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

RAG Eval compared with similar skills
Skill	Stars	Used in	Tokens	Auto-check	Licence	Repo updated
RAG Eval this skillNVIDIA/skills	3.5k	—	~2.3k	Automated safety check: Notes	Apache-2.0	today
Evaluate RAGai-evals-course/evals-skills	1.5k	—	~1.9k	Automated safety check: Pass	Apache-2.0	12 days ago
RAG ArchitectJeffallan/claude-skills	12k	1 repos	~2k	Automated safety check: Pass	MIT	4 days ago
Jd Gap Analysisstarkyru/learn-ai	105	—	~1.9k	Automated safety check: Pass	MIT	2 mo ago
Agent Evalericrisco/rsc-harness	156	—	~3.2k	Automated safety check: Pass	MIT	today
RAG Observability Evalssickn33/agentic-awesome-skills	47k	2 repos	~3.1k	Automated safety check: Pass	MIT	yesterday

Similar skills

Evaluate RAG
ai-evals-course/evals-skills
Guides evaluation of a RAG system by diagnosing failures in traces, building a retrieval test set and scoring retrieval and generation separately.
1.5k GitHub stars~1.9k tokensUpdated 12 days ago
AI & LLM EngineeringAuto-check passed
RAG Architect
Jeffallan/claude-skills
Designs retrieval-augmented generation systems: document chunking, embeddings, vector store setup, hybrid search, reranking and retrieval evaluation, with checks at each step.
12k GitHub starsUsed in 1 repo~2k tokens
AI & LLM EngineeringAuto-check passed
Jd Gap Analysis
starkyru/learn-ai
Analyze a job description (pasted text OR a URL) and find the AI/ML/GenAI topics it requires that this learn-ai course does NOT yet cover.
105 GitHub stars~1.9k tokensUpdated 2 mo ago
AI & LLM EngineeringAuto-check passed
Agent Eval
ericrisco/rsc-harness
A skill your agent uses when measuring whether an LLM or agent system actually got better and gating merges on it: golden sets, fixing an inflated LLM-as-judge, scoring RAG (faithfulness, contextual…
156 GitHub stars~3.2k tokensUpdated today
AI & LLM EngineeringAuto-check passed
RAG Observability Evals
sickn33/agentic-awesome-skills
Monitor and evaluate RAG systems with retrieval quality metrics, groundedness checks, hallucination detection, and continuous regression testing.
47k GitHub starsUsed in 2 repos~3.1k tokens
AI & LLM EngineeringAuto-check passed
RAG Evaluation Harness
davepoon/buildwithclaude
Evaluate retrieval and citation behavior for RAG pipelines from deterministic JSONL fixtures.
3.6k GitHub stars~813 tokensUpdated yesterday
AI & LLM EngineeringAuto-check passed

More from NVIDIA/skills

All 380 skills in this repo

Vss Deploy Detection Tracking 2D
NVIDIA/skills
Official
A skill your agent uses when the user wants to deploy, run, debug, tear down, or call the REST API of the RTVI-CV 2D detection / tracking microservice.
3.5k GitHub starsUsed in 1 repo~4.5k tokens
Auto-check passed
HSB IP Def File Tool
NVIDIA/skills
Official
Generates, validates, compares and explains HOLOLINK_def.svh macro files for the HSB IP, using bundled Python scripts and asking before it writes anything.
3.5k GitHub stars~2.9k tokensUpdated today
Auto-check passed
Isaac Sim Mission Control Showcase
NVIDIA/skills
Official
Runs and validates an end-to-end Mission Control demo in a locally installed Isaac Sim, with a Nova Carter robot driven through a Python server.
3.5k GitHub stars~4.8k tokensUpdated today
Auto-check passed
Defect Image Generation with Cosmos AnomalyGen
NVIDIA/skills
Official
Orchestrates defect image generation for PCBA, metal surface and glass inspection with NVIDIA Cosmos AnomalyGen on OSMO, from cold-start Day 0 to real-photo Day 1 labeling.
3.5k GitHub stars~5k tokensUpdated today
Auto-check: notes
Physical AI Video Augmentation on OSMO
NVIDIA/skills
Official
Orchestrates video data augmentation and auto-labeling workflows on OSMO, from flow selection and preflight checks to submission, monitoring and output download.
3.5k GitHub stars~4.7k tokensUpdated today
Auto-check: notes
TAO Detection KPI Analysis
NVIDIA/skills
Official
Runs NVIDIA TAO Data Services KPI analysis on object detection results, comparing predictions to ground truth and writing per-class precision, recall and AP to a CSV.
3.5k GitHub stars~2.7k tokensUpdated today
Auto-check: notes

Works with

NVIDIA AI Platform

Questions about RAG Eval

What does RAG Eval do?

Filesystem RAG benchmarks: corpus/, train.json, evaluaterag.py (RAGAS quality). RAG Eval is an agent skill from NVIDIA/skills, published by the product's own GitHub organization.py (RAGAS quality).

When should I use RAG Eval?

RAG Eval fits situations like: tasks that involve Retrieval-augmented generation; tasks that involve LLM evaluation.

How do I install RAG Eval in Claude Code?

Run `npx skills add NVIDIA/skills --skill rag-eval -a claude-code`. Or copy the skill folder (skills/rag-eval in NVIDIA/skills) into .claude/skills/rag-eval in your project. Claude Code loads it when a task matches its description.

How do I install RAG Eval in Codex?

Run `npx skills add NVIDIA/skills --skill rag-eval -a codex`. Or copy the skill folder (skills/rag-eval in NVIDIA/skills) into .agents/skills/rag-eval in your project. Codex loads it when a task matches its description.

Can I use RAG Eval in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add NVIDIA/skills --skill rag-eval -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/rag-eval, .gemini/skills/rag-eval, .github/skills/rag-eval and .opencode/skills/rag-eval in your project.

What does RAG Eval need to run?

Going by SKILL.md and its folder, RAG Eval needs the command-line tools its instructions call (uv and python3) and credentials named NVIDIA_API_KEY. Our summary lists: Python 3; A credential in NVIDIA_API_KEY. Its frontmatter pre-approves these tools: Read, Grep, Glob, Bash(ls *), Bash(python3 *), Bash(uv *), Write, Edit. Compatibility (from SKILL.md): Repository checkout with uv; Python 3.11+; run from repo root; uv sync --project scripts/eval (eval deps live in scripts/eval/pyproject.toml); network to RAG, ingestor, and vdb endpoints; NVIDIA_API_KEY for RAGAS; optional RAG_EVAL_JUDGE_MODEL (default mistralai/mixtral-8x22b-instruct-v0.1)..

Does RAG Eval access the network?

SKILL.md contains no URLs. Its commands use uv, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is RAG Eval safe to install?

Our automated static check of SKILL.md found notes only (mentions a .env file), nothing it rates as a warning. It is not a guarantee. Review the folder before installing.

What licence does RAG Eval use?

RAG Eval is published under the Apache-2.0 licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does RAG Eval use?

About 2.3k tokens (SKILL.md is roughly 9.4k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 6k tokens, read only when the agent opens those files.

What are the alternatives to RAG Eval?

Skills that share tags, products or a category with RAG Eval: Evaluate RAG (ai-evals-course/evals-skills, 1.5k stars), RAG Architect (Jeffallan/claude-skills, 12k stars), Jd Gap Analysis (starkyru/learn-ai, 105 stars) and Agent Eval (ericrisco/rsc-harness, 156 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains RAG Eval?

NVIDIA (a GitHub organization, an official publisher) maintains it in NVIDIA/skills, which has 3,534 GitHub stars. The repository holds 380 skills in this directory. The repository was last updated on October 7, 2026.

Source: NVIDIA/skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.

Install RAG Eval

Facts

At a glance

What it does

When your agent uses it

Example prompts

Requirements

Workflow steps

What it can do on your machine

Tool permissions

Runs code

Network

Credentials

Compatibility

Context cost

Safety

SKILL.md

On-disk RAG evaluation (corpus/ + train.json)

Purpose

When not to use

Prerequisites

Instructions

Examples

Limitations

Troubleshooting

Gotchas

Source of truth

Agent playbook

Files

Compare with similar skills

Similar skills

More from NVIDIA/skills

Related

Questions about RAG Eval

On-disk RAG evaluation (`corpus/` + `train.json`)