LLM evaluation

6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building…

sickn33/agentic-awesome-skills

Eval-driven development (EDD) framework for AI coding sessions — define capability and regression evals before coding, grade with code-based, model-based, rule, or human graders, and track pass@k…

affaan-m/ECC

Generates python code that evaluates SageMaker models. An agent skill from awslabs/agent-plugins.

awslabs/agent-plugins

A skill your agent uses when you want guided bottom-up error analysis, structured failure taxonomy discovery, or comprehensive trace coding before documenting eval criteria.

tikalk/adlc-team-skills

Entry point for Pydantic Logfire — an observability, monitoring, and evals platform.

pydantic/skills

Build a regression + eval harness for AI-written code and AI features.

Houseofmvps/ultraship

Turn student course evaluations (free-text + numeric) into an actionable teaching-improvement plan — the teaching analogue of /respond-to-referees.

pedrohcgs/claude-code-my-workflow

Build the evaluation harness that gates every fine-tuning run — golden sets, per-failure-mode graders, judge calibration, and base-model baselines.

wshobson/agents

Create offline evaluation tests for Output SDK workflows using @outputai/evals.

growthxai/output

Multi-agent collaboration plugin that spawns N parallel subagents competing on the same task via git worktree isolation.

alirezarezvani/claude-skills

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise…

aiskillstore/marketplace

Teaches how to write and run evals on the products/posthogai/evalharness/ harness — sandboxed agent suites that execute the real coding agent in a Docker or Modal sandbox against a seeded Hedgebox…

PostHog/posthog

Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology.

google/skills

Author a new Dex skill that actually fires and passes the quality bar.

davekilleen/Dex

lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.). An agent skill from Luciole-Studio/Misaka-Agent.

Luciole-Studio/Misaka-Agent

A skill your agent uses when a goldset with graders is ready to run — executes the evaluation pyramid and validates evaluator quality (SLA compliance, TPR/TNR, statistical accuracy).

tikalk/adlc-team-skills

Sets up eval-driven development for Claude Code workflows: capability and regression evals, three grader types and pass@k reliability metrics.

affaan-m/ECC

用于医疗应用部署的患者安全评估工具。针对CDSS准确性、PHI暴露、临床工作流完整性和集成合规性的自动化测试套件。在安全故障时阻止部署。

affaan-m/ECC

A skill your agent uses when the user has evaluation principles or a dataset but needs help choosing the right graders, designing evaluation metrics, creating LLM-as-judge prompts, combining…

agentscope-ai/OpenJudge

Teaches agents how to write correct riteway ai prompt evals (.sudo files) for multi-step flows that involve tool calls.

paralleldrive/aidd

Run Azure SDK QA bot evaluations on curated datasets locally, including a single test case.

Azure/azure-sdk-tools

Author continuously-running online evaluations in PostHog AI observability, grounded in real failure modes you've identified.

PostHog/posthog

Evaluate WooAIAssistant against a structured scenario suite with hard invariants + LLM-as-judge rubric scoring.

woocommerce/woocommerce-ios

Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates.

sickn33/agentic-awesome-skills

Auto-discover all skills with evals in RConsortium/pharma-skills, benchmark each with vs.

RConsortium/pharma-skills

Evaluate LLM systems using automated metrics, LLM-as-judge, and benchmarks.

ancoleman/ai-design-components

Improve AI application with evaluation-driven development. An agent skill from github/awesome-copilot.

github/awesome-copilot

Run an improve-my-MCP campaign: an autoresearch-style loop that measures the MCP agent experience with the eval harness, picks the highest-impact tool problem from production data, makes one bounded…

PostHog/posthog

INVOKE THIS SKILL when building, testing, or deploying Managed Deep Agents in LangSmith.

langchain-ai/langchain-skills

Autonomously optimize any Claude Code skill by running it repeatedly, scoring outputs against binary evals, mutating the prompt, and keeping improvements.

pedronauck/skills

Author or review software tests and LLM/agent evals; choose test placement and mocks, diagnose flaky CI, or repair brittle suites.

pedronauck/skills

Create, improve, and evaluate agent skills (SKILL.md plus reference files).

himself65/finance-skills

Investigate AI observability evaluations — hog (deterministic code-based), llmjudge (LLM-prompt-based), and sentiment (user-message sentiment).

PostHog/posthog

Set up an LLM-judge evaluation that extracts canonical use cases for a PostHog feature at scale and streams the results to a Slack channel as a live feed.

PostHog/posthog

A skill your agent uses when designing a study that benchmarks AI systems against a human-expert panel, before data collection.

Aperivue/medsci-skills

[Beta] Creates permanent eval cases from promoted learnings and runs regression checks against them.

pskoett/pskoett-ai-skills

[Beta] Session-start scan that surfaces relevant learnings, recent errors, and eval status before work begins.

pskoett/pskoett-ai-skills

Work with the Ambient Quality Agent (AQuA) added to this agents-cli project: augment the agents-cli agent with AQuA or attach the agent to an AQuA deployed elsewhere, read the quality insights it…

google/adk-recipes

Build or refresh a product README showcase using a seeded Bag of Words workspace, polished in-product screenshots, and repository-ready visual assets.

bagofwords1/bagofwords

Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes.

aiskillstore/marketplace

Audit an existing eval suite for trustworthiness. An agent skill from growthxai/output.

growthxai/output

Design effective LLM judge .prompt files for evaluators. An agent skill from growthxai/output.

growthxai/output

Evaluate and rank agent results by metric or LLM judge for an AgentHub session.

alirezarezvani/claude-skills

Configures and runs LLM evaluation using Promptfoo framework.

daymade/claude-code-skills

Eval-driven skill tuning. An agent skill from ClawBio/ClawBio.

ClawBio/ClawBio

Launch agentic Harbor evals through the OT-Agent unified eval listener (eval/unifiedevallistener.py) on any cluster: select models (queryunevaledmodels.py / priority lists), wire the pinggy…

open-thoughts/OpenThoughts-Agent

Skill validation framework PLUS daily test-suite health and regression intelligence.

inbrainfun/inbrain

A skill your agent uses when choosing between engines or models for an agent pipeline and the answer must come from measurement on your own data, not from marketing pages: speech recognition for…

serejaris/personal-corp-os