Agent skill

Eval Harness

MITAuto-check passedAI & LLM Engineering

Install Eval Harness

skills CLI

$ npx skills add affaan-m/ECC --skill eval-harness -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI

$ gh skill install affaan-m/ECC eval-harness --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy

$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/eval-harness .claude/skills/eval-harness && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Paste into Claude Code

Install the "eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/skills/eval-harness into .claude/skills/eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "eval-harness", then confirm the skill loads.

Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

$skill-installer

$skill-installer install https://github.com/affaan-m/ECC/tree/main/skills/eval-harness

Type this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.

skills CLI

$ npx skills add affaan-m/ECC --skill eval-harness -a codex

Project install goes to .agents/skills/; add -g for ~/.codex/skills/.

GitHub CLI

$ gh skill install affaan-m/ECC eval-harness --agent codex

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/eval-harness .agents/skills/eval-harness && rm -rf skills-src

Use ~/.agents/skills/ instead of .agents/skills for a personal install.

Codex skills documentation · loads skills from .agents/skills/

Paste into Codex

Install the "eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/skills/eval-harness into .agents/skills/eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "eval-harness", then confirm the skill loads.

Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add affaan-m/ECC --skill eval-harness -a cursor

Project install goes to .agents/skills/; add -g for ~/.cursor/skills/.

GitHub CLI

$ gh skill install affaan-m/ECC eval-harness --agent cursor

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/eval-harness .cursor/skills/eval-harness && rm -rf skills-src

Use ~/.cursor/skills/ instead of .cursor/skills for a personal install.

Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/

Paste into Cursor

Install the "eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/skills/eval-harness into .cursor/skills/eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "eval-harness", then confirm the skill loads.

Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

gemini skills install

$ gemini skills install https://github.com/affaan-m/ECC.git --path skills/eval-harness

--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.

skills CLI

$ npx skills add affaan-m/ECC --skill eval-harness -a gemini-cli

Project install goes to .agents/skills/; add -g for ~/.gemini/skills/.

GitHub CLI

$ gh skill install affaan-m/ECC eval-harness --agent gemini-cli

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/eval-harness .gemini/skills/eval-harness && rm -rf skills-src

Use ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.

Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/

Paste into Gemini CLI

Install the "eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/skills/eval-harness into .gemini/skills/eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "eval-harness", then confirm the skill loads.

Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

GitHub CLI

$ gh skill install affaan-m/ECC eval-harness

Installs for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).

skills CLI

$ npx skills add affaan-m/ECC --skill eval-harness -a github-copilot

Project install goes to .agents/skills/; add -g for ~/.copilot/skills/.

Manual copy

$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/eval-harness .github/skills/eval-harness && rm -rf skills-src

Use ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.

GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/

Paste into GitHub Copilot

Install the "eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/skills/eval-harness into .github/skills/eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "eval-harness", then confirm the skill loads.

GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add affaan-m/ECC --skill eval-harness -a opencode

OpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.

GitHub CLI

$ gh skill install affaan-m/ECC eval-harness --agent opencode

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/eval-harness .opencode/skills/eval-harness && rm -rf skills-src

Use ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.

OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/

Paste into OpenCode

Install the "eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/skills/eval-harness into .opencode/skills/eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "eval-harness", then confirm the skill loads.

OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

Facts

Skill name: eval-harness
GitHub stars: 276k
Token cost: ~2.2k tokens
SKILL.md length: 641 words
Files: 1
Skills in repo: 683
Repo updated: Oct 10, 2026
First seen: Oct 7, 2026
Licence: MIT

At a glance

Works in 7 steps: Code-Based Grader → Model-Based Grader → Human Grader → …
Defining pass/fail criteria for agent tasks
SKILL.md covers When to Activate, Philosophy, Eval Types and Grader Types, plus 8 more sections
Calls npm and node

What it does

Eval Harness is an agent skill from affaan-m/ECC. Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability. Use when defining pass/fail criteria for agent tasks, measuring agent reliability, building regression suites for prompt or agent changes, or benchmarking across model versions.

Its SKILL.md is about 2.2k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in AI & LLM Engineering, covering LLM evaluation. The repository describes itself as: The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond. The licence is MIT.

When your agent uses it

Defining pass/fail criteria for agent tasks
Measuring agent reliability
Building regression suites for prompt
Benchmarking across model versions

Example prompts

“/eval-harness”

Workflow steps

7 steps, taken from the step headings in SKILL.md.

Code-Based Grader
Model-Based Grader
Human Grader
Define (Before Coding)
Implement
Evaluate
Report

What it can do on your machine

Read from SKILL.md and the folder at commit 4eb71d9. It shows what the files ask for, not the result of running them.

Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Shell commands in SKILL.md call:
- npm
- node
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md. Its commands use npm, which can reach the network depending on how they are called.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Eval Harness loads about 2.2k tokens when it runs. Until then it costs about 103 tokens; SKILL.md has 641 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it: ~103
When it runs · the whole SKILL.md, loaded when a task matches: ~2.2k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from affaan-m/ECC at commit 4eb71d9, republished under its MIT licence (© affaan-m). 641 words, ~2,184 tokens.

Download SKILL.mdSave it as .claude/skills/eval-harness/SKILL.md (or your agent's skills folder).

name: eval-harness
description: Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability. Use when defining pass/fail criteria for agent tasks, measuring agent reliability, building regression suites for prompt or agent changes, or benchmarking across model versions.
metadata.origin: ECC
tools: Read, Write, Edit, Bash, Grep, Glob

Eval Harness Skill

A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.

When to Activate

Setting up eval-driven development (EDD) for AI-assisted workflows
Defining pass/fail criteria for Claude Code task completion
Measuring agent reliability with pass@k metrics
Creating regression test suites for prompt or agent changes
Benchmarking agent performance across model versions

Philosophy

Eval-Driven Development treats evals as the "unit tests of AI development":

Define expected behavior BEFORE implementation
Run evals continuously during development
Track regressions with each change
Use pass@k metrics for reliability measurement

Eval Types

Capability Evals

Test if Claude can do something it couldn't before:

markdown

[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
  - [ ] Criterion 1
  - [ ] Criterion 2
  - [ ] Criterion 3
Expected Output: Description of expected result

Regression Evals

Ensure changes don't break existing functionality:

markdown

[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
  - existing-test-1: PASS/FAIL
  - existing-test-2: PASS/FAIL
  - existing-test-3: PASS/FAIL
Result: X/Y passed (previously Y/Y)

Grader Types

1. Code-Based Grader

Deterministic checks using code:

bash

# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"

# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"

# Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"

2. Model-Based Grader

Use Claude to evaluate open-ended outputs:

markdown

[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
4. Is error handling appropriate?

Score: 1-5 (1=poor, 5=excellent)
Reasoning: [explanation]

3. Human Grader

Flag for manual review:

markdown

[HUMAN REVIEW REQUIRED]
Change: Description of what changed
Reason: Why human review is needed
Risk Level: LOW/MEDIUM/HIGH

Metrics

pass@k

"At least one success in k attempts"

pass@1: First attempt success rate
pass@3: Success within 3 attempts
Typical target: pass@3 > 90%

pass^k

"All k trials succeed"

Higher bar for reliability
pass^3: 3 consecutive successes
Use for critical paths

Eval Workflow

1. Define (Before Coding)

markdown

## EVAL DEFINITION: feature-xyz

### Capability Evals
1. Can create new user account
2. Can validate email format
3. Can hash password securely

### Regression Evals
1. Existing login still works
2. Session management unchanged
3. Logout flow intact

### Success Metrics
- pass@3 > 90% for capability evals
- pass^3 = 100% for regression evals

2. Implement

Write code to pass the defined evals.

3. Evaluate

bash

# Run capability evals
[Run each capability eval, record PASS/FAIL]

# Run regression evals
npm test -- --testPathPattern="existing"

# Generate report

4. Report

markdown

EVAL REPORT: feature-xyz
========================

Capability Evals:
  create-user:     PASS (pass@1)
  validate-email:  PASS (pass@2)
  hash-password:   PASS (pass@1)
  Overall:         3/3 passed

Regression Evals:
  login-flow:      PASS
  session-mgmt:    PASS
  logout-flow:     PASS
  Overall:         3/3 passed

Metrics:
  pass@1: 67% (2/3)
  pass@3: 100% (3/3)

Status: READY FOR REVIEW

Integration Patterns

Pre-Implementation

/eval define feature-name

Creates eval definition file at .claude/evals/feature-name.md

During Implementation

/eval check feature-name

Runs current evals and reports status

Post-Implementation

/eval report feature-name

Generates full eval report

Eval Storage

Store evals in project:

.claude/
  evals/
    feature-xyz.md      # Eval definition
    feature-xyz.log     # Eval run history
    baseline.json       # Regression baselines

Best Practices

Define evals BEFORE coding - Forces clear thinking about success criteria
Run evals frequently - Catch regressions early
Track pass@k over time - Monitor reliability trends
Use code graders when possible - Deterministic > probabilistic
Human review for security - Never fully automate security checks
Keep evals fast - Slow evals don't get run
Version evals with code - Evals are first-class artifacts

Example: Adding Authentication

markdown

## EVAL: add-authentication

### Phase 1: Define (10 min)
Capability Evals:
- [ ] User can register with email/password
- [ ] User can login with valid credentials
- [ ] Invalid credentials rejected with proper error
- [ ] Sessions persist across page reloads
- [ ] Logout clears session

Regression Evals:
- [ ] Public routes still accessible
- [ ] API responses unchanged
- [ ] Database schema compatible

### Phase 2: Implement (varies)
[Write code]

### Phase 3: Evaluate
Run: /eval check add-authentication

### Phase 4: Report
EVAL REPORT: add-authentication
==============================
Capability: 5/5 passed (pass@3: 100%)
Regression: 3/3 passed (pass^3: 100%)
Status: SHIP IT

Show full SKILL.md (352 more words)Show less

Local Framework Utilities

The mechanical utilities ship in scripts/lib/eval-harness/:

node scripts/eval-harness.js example

Capsule: hash-linked journal with five lineages and local integrity checks.
Inspection: source digests, validated variant paths, and syntactic warnings.
Replay: declared tools and content-addressed fixtures. Missing fixtures fail closed; SE3 and above are refused in replay. Record mode invokes the registered implementation, so only register trusted functions.
Receipt: offline verification of capsule and artifact bytes, with named checks.
Retrospective preparation: node scripts/eval-harness.js capsule group <dir> [<dir> ...] groups 1 to 100 explicitly selected, verified local capsule snapshots from one task family by declared harness version. Repeated snapshots count once; conflicting identities or invalid capsules reject the whole report. This is read-only record counting, with no new rollouts, scores or promotion. Use small, quiescent capsules. Payloads, directory arguments and raw run/capsule IDs are omitted, but task-family/version labels are verbatim and digest references are linkable; review them before sharing. Operational validation remains pending.

Candidate execution is disabled on every OS because no verified OS containment backend is implemented. gate run, runGate, runVariant, direct child launch, and the retired effect preload refuse with gate.isolation_required. No trust flag or caller-supplied executor can bypass the refusal. The example records that refusal and inspects source without executing or scoring it.

Do not present static warnings, a capsule receipt, or successful utility tests as candidate containment or promotion evidence. A future gate requires an independently reviewed OS boundary, protected checker and audit channels, and fatal baseline rejection. See docs/architecture/eval-harness-frameworks.md.

Product Evals (v1.8)

Use product evals when behavior quality cannot be captured by unit tests alone.

Grader Types

Code grader (deterministic assertions)
Rule grader (regex/schema constraints)
Model grader (LLM-as-judge rubric)
Human grader (manual adjudication for ambiguous outputs)

pass@k Guidance

pass@1: direct reliability
pass@3: practical reliability under controlled retries
pass^3: stability test (all 3 runs must pass)

Recommended thresholds:

Capability evals: pass@3 >= 0.90
Regression evals: pass^3 = 1.00 for release-critical paths

Eval Anti-Patterns

Overfitting prompts to known eval examples
Measuring only happy-path outputs
Ignoring cost and latency drift while chasing pass rates
Allowing flaky graders in release gates

Minimal Eval Artifact Layout

.claude/evals/<feature>.md definition
.claude/evals/<feature>.log run history
docs/releases/<version>/eval-summary.md release snapshot

Files

Just SKILL.md in skills/eval-harness of affaan-m/ECC.

Open the folder on GitHubat commit 4eb71d9

Compare with similar skills

Eval Harness next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Eval Harness compared with similar skills
Skill	Stars	Used in	Tokens	Auto-check	Licence	Repo updated
Eval Harness this skillaffaan-m/ECC	276k	—	~2.2k	Automated safety check: Pass	MIT	yesterday
LLM Benchmarking with lm-evaluation-harnessOrchestra-Research/AI-Research-SKILLs	13k	8 repos	~3k	Automated safety check: Pass	MIT	3 mo ago
Hugging Face Local Model Evalshuggingface/skills	11k	2 repos	~1.6k	Automated safety check: Pass	Apache-2.0	2 days ago
Looperksimback/looper	710	—	~2.7k	Automated safety check: Notes	MIT	2 mo ago
Agent Eval Engineeringlangchain-ai/langchain-skills	1.3k	—	~4k	Automated safety check: Pass	MIT	2 days ago
Quality FlywheelGoogleCloudPlatform/vertex-ai-samples	792	—	~2k	Automated safety check: Pass	Apache-2.0	yesterday

Similar skills

LLM Benchmarking with lm-evaluation-harness
Orchestra-Research/AI-Research-SKILLs
Runs lm-evaluation-harness to benchmark language models on academic suites such as MMLU, GSM8K and HumanEval, compare models and track training checkpoints.
13k GitHub starsUsed in 8 repos~3k tokens
AI & LLM EngineeringAuto-check passed
Hugging Face Local Model Evals
huggingface/skills
Official
Runs evaluations of Hugging Face Hub models on local hardware with inspect-ai or lighteval, and helps choose between vLLM, Transformers and accelerate backends.
11k GitHub starsUsed in 2 repos~1.6k tokens
AI & LLM EngineeringAuto-check passed
Looper
ksimback/looper
Scaffold a well-designed agent loop with best-practice coaching and a cross-model review council.
710 GitHub stars~2.7k tokensUpdated 2 mo ago
AI & LLM EngineeringAuto-check: notes
Agent Eval Engineering
langchain-ai/langchain-skills
Official
Builds agent evaluations in stages: inspect the repository and traces, agree a Task Spec with you, then build, audit and run a Harbor task with an independent verifier.
1.3k GitHub stars~4k tokensUpdated 2 days ago
AI & LLM EngineeringAuto-check passed
Quality Flywheel
GoogleCloudPlatform/vertex-ai-samples
Evaluate and improve GenAI models and agents using the Google GenAI Evaluation SDK.
792 GitHub stars~2k tokensUpdated yesterday
AI & LLM EngineeringAuto-check passed
Eval Harness
cloudnative-co/claude-code-starter-kit
Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles.
153 GitHub starsUsed in 9 repos~1.3k tokens
AI & LLM EngineeringAuto-check passed

Questions about Eval Harness

What does Eval Harness do?

Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k…. Eval Harness is an agent skill from affaan-m/ECC. Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k and pass^k reliability.

When should I use Eval Harness?

Eval Harness fits situations like: defining pass/fail criteria for agent tasks; measuring agent reliability; building regression suites for prompt; benchmarking across model versions.

How do I install Eval Harness in Claude Code?

Run `npx skills add affaan-m/ECC --skill eval-harness -a claude-code`. Or copy the skill folder (skills/eval-harness in affaan-m/ECC) into .claude/skills/eval-harness in your project. Claude Code loads it when a task matches its description.

How do I install Eval Harness in Codex?

Run `npx skills add affaan-m/ECC --skill eval-harness -a codex`. Or copy the skill folder (skills/eval-harness in affaan-m/ECC) into .agents/skills/eval-harness in your project. Codex loads it when a task matches its description.

Can I use Eval Harness in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add affaan-m/ECC --skill eval-harness -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/eval-harness, .gemini/skills/eval-harness, .github/skills/eval-harness and .opencode/skills/eval-harness in your project.

What does Eval Harness need to run?

Going by SKILL.md and its folder, Eval Harness needs the command-line tools its instructions call (npm and node).

Does Eval Harness access the network?

SKILL.md contains no URLs. Its commands use npm, which can reach the network depending on how they are called. This is read from the text; nothing was executed.

Is Eval Harness safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Eval Harness use?

Eval Harness is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Eval Harness use?

About 2.2k tokens (SKILL.md is roughly 8.7k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Eval Harness?

Skills that share tags, products or a category with Eval Harness: LLM Benchmarking with lm-evaluation-harness (Orchestra-Research/AI-Research-SKILLs, 13k stars), Hugging Face Local Model Evals (huggingface/skills, 11k stars), Looper (ksimback/looper, 710 stars) and Agent Eval Engineering (langchain-ai/langchain-skills, 1.3k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Eval Harness?

affaan-m (a GitHub user) maintains it in affaan-m/ECC, which has 276,111 GitHub stars. The repository holds 683 skills in this directory. The repository was last updated on October 10, 2026.

Source: affaan-m/ECC on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.

Install Eval Harness

Facts

At a glance

What it does

When your agent uses it

Example prompts

Workflow steps

What it can do on your machine

Tool permissions

Runs code

Network

Credentials

Context cost

Safety

SKILL.md

Eval Harness Skill

When to Activate

Philosophy

Eval Types

Capability Evals

Regression Evals

Grader Types

1. Code-Based Grader

2. Model-Based Grader

3. Human Grader

Metrics

pass@k

pass^k

Eval Workflow

1. Define (Before Coding)

2. Implement

3. Evaluate

4. Report

Integration Patterns

Pre-Implementation

During Implementation

Post-Implementation

Eval Storage

Best Practices

Example: Adding Authentication

Local Framework Utilities

Product Evals (v1.8)

Grader Types

pass@k Guidance

Eval Anti-Patterns

Minimal Eval Artifact Layout

Files

Compare with similar skills

Similar skills

More from affaan-m/ECC

Related

Questions about Eval Harness