Agent evaluation and testing

Benchmarks AMD's GAIA agent against Claude Code and across models on quality, honesty, steps, tokens, time and real cost, using gaia eval tasks.

amd/gaia

Run agent benchmarks, create tasks, analyze results, and manage agents using BenchFlow.

benchflow-ai/benchflow

Collects evidence from a failed or confusing skill-driven task and triggers skvm jit-optimize to propose concrete fixes to that skill's files.

SJTU-IPADS/SkVM

Runs Darwin Mode on an agent harness: mutates one policy file per generation in a sandbox, scores each variant against your tests, and archives only variants that measurably improve.

ruvnet/RuView

Operational notes for generating, evaluating and debugging strategies in the autocontext grid_ctf scenario, with tier rules and parameter ranges that worked or failed.

greyhaven-ai/autocontext

Operates or analyzes a LoopX-managed benchmark experiment: launching runs, maintaining the experiment board, qualifying integrity, and writing case insights.

loopx-project/loopx

Walks you through creating, running and reading waza evals for an agent skill, then proposes concrete fixes when tasks fail or the score is low.

microsoft/waza

Evaluate mathematical expressions and unit conversions. An agent skill from NVIDIA/SkillEvaluator.

NVIDIA/SkillEvaluator

Audits recent agent conversation history and turns repeated failures and successes into testable harness improvement proposals that later audits can check.

KonghaYao/peri

Run and maintain Con's terminal-agent benchmark against a live app session.

nowledge-co/con-terminal

Author and safely run deterministic email-agent evaluation suites with dedicated e2a test agents.

tokencanopy/e2a

A skill your agent uses when the user's Copilot Studio agent evaluations have come back and they need to interpret scores, diagnose root causes of underperforming test cases, find remediation steps…

microsoft/eval-guide

Interviews you about what a skill should do, writes an eval spec from your answers, then loops through run, diagnose and improve until the skill is ready to ship.

edonadei/caliper

Create new skills, modify and improve existing skills, and measure skill performance.

ZS520L/HanakoPro

Static safety audit of a SKILL.md that scores five dimensions and acts as a gate: skills below the pass line do not ship, whatever else they score.

openJiuwen-ai/agent-core

Evaluate Agent Skill design quality with an opinionated, practice-derived rubric informed by public specifications and examples.

shareAI-lab/lab-skills

Analyze text content and produce statistics including word count, line count, character count, most frequent words, and readability metrics.

NVIDIA/SkillEvaluator

[omh] Choosing between coding agents on evidence: compare executor or agent choices on reproducible tasks using quality, cost, time, tool, and evidence metrics.

rlaope/oh-my-hermes

Test and validate AgentUse agents without real side effects.

agentuse/agentuse

Evaluate catalog candidates against either the standard five agent-native criteria or the narrow operator-surface track, and check URL Onboarding.

haoruilee/awesome-agent-native-services

A skill your agent uses when changing Deep Researcher Agent continuous integration, pre-commit, or contributor governance — editing .github/workflows/ (ci, ui, skills-eval, request-nvskills-ci)…

NVIDIA-AI-Blueprints/deep-researcher-agent

Eval enablement accelerator — help customers think through "what does good look like" for their AI agent, then generate a structured eval plan and test cases they can use immediately.

microsoft/eval-guide

Framework for measuring and tracking agent response quality over time.

vibeeval/vibecosystem

Runbook for the five-arm Codex harness comparison on Terminal-Bench 4.0, pointing out four differences from SWE-Marathon that silently produce wrong scores.

loopx-project/loopx

Runs a measurable keep-or-discard improvement loop against a runnable sensor, from framing a goal and KPI through baseline, judging and held-out verification.

bgauryy/octocode

Run one specified Test Agent on one specified Benchmark Case exactly once, privately score that execution, and return one protocol result.

Prism-Shadow/penguin-harness

Evaluate agent and skill behavior or routing. An agent skill from TheGoat395/Codex-Skills.

TheGoat395/Codex-Skills

Writes turn-level tests for a LiveKit agent in the user's normal test suite: pytest (Python) or Vitest (Node.js).

livekit-examples/agent-starter-python

Run evaluation pipelines on Claude Code skills to test triggering accuracy, workflow correctness, and output quality.

AgriciDaniel/skill-forge

Answers AI agent evaluation methodology questions with practical, opinionated guidance grounded primarily in Microsoft's agent evaluation ecosystem (MS Learn, Eval Scenario Library, Triage &…

microsoft/eval-guide

Pulls pattern, trend and benchmark data from the claude-view MCP server to present a behavioral analysis of your Claude Code usage, grouped by category.

tombelieber/claude-view

Runs Chat Customizations Evaluations analysis for the active customization file and summarizes the findings from the Problems panel.

microsoft/vscode-chat-customizations-evaluation

Creates, edits and benchmarks skills; supersedes the official skill-creator plugin, so when both are listed, use this one.

daymade/claude-code-skills

Points agents to the right sources when changing the Smithers workspace graph, generated docs, benchmark and eval evidence, or durable flows in the Smithers repository.

smithersai/smithers

Run and improve the c0 agent evaluation loop using the local and remote Cloudflare MCP harness, versioned history notes, and current Cloudflare product updates.

Consensys/c0

Run skill evals via evals/runner.js — wrapper validates skill names, captures stdout/stderr, persists JSON results.

ww-w-ai/bkit-claude-code

Analyzes a finished pull request that involved an agent to find what slowed or helped it, and recommends specific changes to instruction files, skills and docs.

dotnet/maui

Guidance for using A-Evolve to improve an AI agent automatically, evolving its prompts, skills and memory against a benchmark through solve, observe and evolve cycles.

Orchestra-Research/AI-Research-SKILLs

Analyzes Copilot Studio evaluation results using Practical Guidance on Agent Evaluation's 10-step playbook (Steps 6, 7, and 9) plus Microsoft's triage diagnostics.

microsoft/eval-guide

Improves a single skill through a test, fix and retest loop, keeping or reverting each change based on how the static and category scores move.

Donchitos/Claude-Code-Game-Studios

Agentforce agent testing with dual-track workflow and 100-point scoring.

Jaganpro/sf-skills

Creates or changes a druxt.js contributor skill in .agents/skills, with its evals and the tests that gate it.

druxt/druxt.js

Investigates what went wrong in a superpowers session by reading its transcript, reports findings with path and line citations, and can draft a GitHub issue or redacted bundle.

jnMetaCode/superpowers-zh

Improves Agent Skills via validate → rule-based eval cases → plugin-eval → prompt evals → bounded edits with held-out gates.

Arenukvern/mcp_flutter

A skill your agent uses when evaluating, designing, or pressure-testing the business model of an AI agent product.

harperreed/dotfiles

Plan standalone — populates the Eval Suite Planning & Logging Template from an Agent Vision or plain-English agent description.

microsoft/eval-guide

Agent skill for benchmark-suite - invoke with $agent-benchmark-suite

ruvnet/ruflo

Agent skill for test-long-runner - invoke with $agent-test-long-runner

ruvnet/ruflo