Agent Eval Engineering
langchain-ai/langchain-skills
Builds agent evaluations in stages: inspect the repository and traces, agree a Task Spec with you, then build, audit and run a Harbor task with an independent verifier.
用于医疗应用部署的患者安全评估工具。针对CDSS准确性、PHI暴露、临床工作流完整性和集成合规性的自动化测试套件。在安全故障时阻止部署。
$ npx skills add affaan-m/ECC --skill healthcare-eval-harness -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install affaan-m/ECC healthcare-eval-harness --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .claude/skills && cp -r skills-src/docs/zh-CN/skills/healthcare-eval-harness .claude/skills/healthcare-eval-harness && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "healthcare-eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/docs/zh-CN/skills/healthcare-eval-harness into .claude/skills/healthcare-eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "healthcare-eval-harness", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/affaan-m/ECC/tree/main/docs/zh-CN/skills/healthcare-eval-harnessType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add affaan-m/ECC --skill healthcare-eval-harness -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install affaan-m/ECC healthcare-eval-harness --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .agents/skills && cp -r skills-src/docs/zh-CN/skills/healthcare-eval-harness .agents/skills/healthcare-eval-harness && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "healthcare-eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/docs/zh-CN/skills/healthcare-eval-harness into .agents/skills/healthcare-eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "healthcare-eval-harness", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add affaan-m/ECC --skill healthcare-eval-harness -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install affaan-m/ECC healthcare-eval-harness --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/docs/zh-CN/skills/healthcare-eval-harness .cursor/skills/healthcare-eval-harness && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "healthcare-eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/docs/zh-CN/skills/healthcare-eval-harness into .cursor/skills/healthcare-eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "healthcare-eval-harness", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/affaan-m/ECC.git --path docs/zh-CN/skills/healthcare-eval-harness--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add affaan-m/ECC --skill healthcare-eval-harness -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install affaan-m/ECC healthcare-eval-harness --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/docs/zh-CN/skills/healthcare-eval-harness .gemini/skills/healthcare-eval-harness && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "healthcare-eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/docs/zh-CN/skills/healthcare-eval-harness into .gemini/skills/healthcare-eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "healthcare-eval-harness", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install affaan-m/ECC healthcare-eval-harnessInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add affaan-m/ECC --skill healthcare-eval-harness -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .github/skills && cp -r skills-src/docs/zh-CN/skills/healthcare-eval-harness .github/skills/healthcare-eval-harness && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "healthcare-eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/docs/zh-CN/skills/healthcare-eval-harness into .github/skills/healthcare-eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "healthcare-eval-harness", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add affaan-m/ECC --skill healthcare-eval-harness -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install affaan-m/ECC healthcare-eval-harness --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/affaan-m/ECC.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/docs/zh-CN/skills/healthcare-eval-harness .opencode/skills/healthcare-eval-harness && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "healthcare-eval-harness" agent skill from https://github.com/affaan-m/ECC/tree/main/docs/zh-CN/skills/healthcare-eval-harness into .opencode/skills/healthcare-eval-harness/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "healthcare-eval-harness", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
healthcare-eval-harness用于医疗应用部署的患者安全评估工具。针对CDSS准确性、PHI暴露、临床工作流完整性和集成合规性的自动化测试套件。在安全故障时阻止部署。
Healthcare Eval Harness is an agent skill from affaan-m/ECC. 用于医疗应用部署的患者安全评估工具。针对CDSS准确性、PHI暴露、临床工作流完整性和集成合规性的自动化测试套件。在安全故障时阻止部署。
Its SKILL.md is about 1.4k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.
It sits in AI & LLM Engineering, covering LLM evaluation and Clinical and healthcare research. The repository describes itself as: The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond. The licence is MIT.
Read from SKILL.md and the folder at commit 4eb71d9. It shows what the files ask for, not the result of running them.
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Shell commands in SKILL.md call:
npxjqFrom the folder's file list and the shell code blocks in SKILL.md.
No URLs in SKILL.md. Its commands use npx, which can reach the network depending on how they are called.
From URLs in SKILL.md, links to its own repository left out.
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Healthcare Eval Harness loads about 1.4k tokens when it runs. Until then it costs about 23 tokens; SKILL.md has 120 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
The full file from affaan-m/ECC at commit 4eb71d9, republished under its MIT licence (© affaan-m). 120 words, ~1,411 tokens.
.claude/skills/healthcare-eval-harness/SKILL.md (or your agent's skills folder).医疗应用部署的自动化验证系统。单个严重故障将阻止部署。患者安全不容妥协。
注意: 示例使用 Jest 作为参考测试运行器。请根据您的框架(Vitest、pytest、PHPUnit 等)调整命令——测试类别和通过阈值与框架无关。
评估框架按顺序运行五个测试类别。前三个(CDSS 准确性、PHI 暴露、数据完整性)是严重关卡,要求 100% 通过率——单个故障即阻止部署。其余两个(临床工作流、集成)是高优先级关卡,要求 95% 以上通过率。
每个类别对应一个 Jest 测试路径模式。CI 流水线使用 --bail(首次失败即停止)运行严重关卡,并使用 --coverage --coverageThreshold 强制执行覆盖率阈值。
1. CDSS 准确性(严重 — 要求 100%)
测试所有临床决策支持逻辑:药物相互作用对(双向)、剂量验证规则、临床评分与发布规范的对比、无假阴性、无静默故障。
npx jest --testPathPattern='tests/cdss' --bail --ci --coverage2. PHI 暴露(严重 — 要求 100%)
测试受保护健康信息泄露:API 错误响应、控制台输出、URL 参数、浏览器存储、跨机构隔离、未认证访问、服务角色密钥缺失。
npx jest --testPathPattern='tests/security/phi' --bail --ci3. 数据完整性(严重 — 要求 100%)
测试临床数据安全:锁定就诊记录、审计追踪条目、级联删除保护、并发编辑处理、无孤立记录。
npx jest --testPathPattern='tests/data-integrity' --bail --ci4. 临床工作流(高优先级 — 要求 95% 以上)
测试端到端流程:就诊生命周期、模板渲染、用药集、药物/诊断搜索、处方 PDF、红色警报。
tmp_json=$(mktemp)
npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$tmp_json" || true
total=$(jq '.numTotalTests // 0' "$tmp_json")
passed=$(jq '.numPassedTests // 0' "$tmp_json")
if [ "$total" -eq 0 ]; then
echo "No clinical tests found" >&2
exit 1
fi
rate=$(echo "scale=2; $passed * 100 / $total" | bc)
echo "Clinical pass rate: ${rate}% ($passed/$total)"5. 集成合规性(高优先级 — 要求 95% 以上)
测试外部系统:HL7 消息解析(v2.x)、FHIR 验证、实验室结果映射、格式错误消息处理。
tmp_json=$(mktemp)
npx jest --testPathPattern='tests/integration' --ci --json --outputFile="$tmp_json" || true
total=$(jq '.numTotalTests // 0' "$tmp_json")
passed=$(jq '.numPassedTests // 0' "$tmp_json")
if [ "$total" -eq 0 ]; then
echo "No integration tests found" >&2
exit 1
fi
rate=$(echo "scale=2; $passed * 100 / $total" | bc)
echo "Integration pass rate: ${rate}% ($passed/$total)"| 类别 | 阈值 | 失败时操作 |
|---|---|---|
| CDSS 准确性 | 100% | 阻止部署 |
| PHI 暴露 | 100% | 阻止部署 |
| 数据完整性 | 100% | 阻止部署 |
| 临床工作流 | 95% 以上 | 警告,允许经审查后部署 |
| 集成 | 95% 以上 | 警告,允许经审查后部署 |
name: Healthcare Safety Gate
on: [push, pull_request]
jobs:
safety-gate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
- run: npm ci
# CRITICAL gates — 100% required, bail on first failure
- name: CDSS Accuracy
run: npx jest --testPathPattern='tests/cdss' --bail --ci --coverage --coverageThreshold='{"global":{"branches":80,"functions":80,"lines":80}}'
- name: PHI Exposure Check
run: npx jest --testPathPattern='tests/security/phi' --bail --ci
- name: Data Integrity
run: npx jest --testPathPattern='tests/data-integrity' --bail --ci
# HIGH gates — 95%+ required, custom threshold check
# HIGH gates — 95%+ required
- name: Clinical Workflows
run: |
TMP_JSON=$(mktemp)
npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$TMP_JSON" || true
TOTAL=$(jq '.numTotalTests // 0' "$TMP_JSON")
PASSED=$(jq '.numPassedTests // 0' "$TMP_JSON")
if [ "$TOTAL" -eq 0 ]; then
echo "::error::No clinical tests found"; exit 1
fi
RATE=$(echo "scale=2; $PASSED * 100 / $TOTAL" | bc)
echo "Pass rate: ${RATE}% ($PASSED/$TOTAL)"
if (( $(echo "$RATE < 95" | bc -l) )); then
echo "::warning::Clinical pass rate ${RATE}% below 95%"
fi
- name: Integration Compliance
run: |
TMP_JSON=$(mktemp)
npx jest --testPathPattern='tests/integration' --ci --json --outputFile="$TMP_JSON" || true
TOTAL=$(jq '.numTotalTests // 0' "$TMP_JSON")
PASSED=$(jq '.numPassedTests // 0' "$TMP_JSON")
if [ "$TOTAL" -eq 0 ]; then
echo "::error::No integration tests found"; exit 1
fi
RATE=$(echo "scale=2; $PASSED * 100 / $TOTAL" | bc)
echo "Pass rate: ${RATE}% ($PASSED/$TOTAL)"
if (( $(echo "$RATE < 95" | bc -l) )); then
echo "::warning::Integration pass rate ${RATE}% below 95%"
fi--no-bail--coveragenpx jest --testPathPattern='tests/cdss' --bail --ci --coverage && \
npx jest --testPathPattern='tests/security/phi' --bail --ci && \
npx jest --testPathPattern='tests/data-integrity' --bail --citmp_json=$(mktemp)
npx jest --testPathPattern='tests/clinical' --ci --json --outputFile="$tmp_json" || true
jq '{
passed: (.numPassedTests // 0),
total: (.numTotalTests // 0),
rate: (if (.numTotalTests // 0) == 0 then 0 else ((.numPassedTests // 0) / (.numTotalTests // 1) * 100) end)
}' "$tmp_json"
# Expected: { "passed": 21, "total": 22, "rate": 95.45 }## 医疗评估:2026-03-27 [commit abc1234]
### 患者安全:通过
| 类别 | 测试数 | 通过 | 失败 | 状态 |
|----------|-------|------|------|--------|
| CDSS 准确性 | 39 | 39 | 0 | 通过 |
| PHI 暴露 | 8 | 8 | 0 | 通过 |
| 数据完整性 | 12 | 12 | 0 | 通过 |
| 临床工作流 | 22 | 21 | 1 | 95.5% 通过 |
| 集成 | 6 | 6 | 0 | 通过 |
### 覆盖率:84%(目标:80%以上)
### 结论:可安全部署© affaan-m, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Just SKILL.md in docs/zh-CN/skills/healthcare-eval-harness of affaan-m/ECC.
Open the folder on GitHubat commit 4eb71d9
Healthcare Eval Harness next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Healthcare Eval Harness this skillaffaan-m/ECC | 276k | — | ~1.4k | Automated safety check: Pass | MIT | |
| Agent Eval Engineeringlangchain-ai/langchain-skills | 1.3k | — | ~4k | Automated safety check: Pass | MIT | |
| Skill Conductorsmixs/skill-conductor | 179 | — | ~6.6k | Automated safety check: Pass | MIT | |
| Evaluate RAGai-evals-course/evals-skills | 1.5k | — | ~1.9k | Automated safety check: Pass | Apache-2.0 | |
| Evalagentevals-dev/agentevals | 163 | — | ~904 | Automated safety check: Pass | Apache-2.0 | |
| Synthetic Eval Data Generatorai-evals-course/evals-skills | 1.5k | — | ~1.4k | Automated safety check: Pass | Apache-2.0 |
langchain-ai/langchain-skills
Builds agent evaluations in stages: inspect the repository and traces, agree a Task Spec with you, then build, audit and run a Harbor task with an independent verifier.
smixs/skill-conductor
Create, edit, evaluate, and package agent skills. An agent skill from smixs/skill-conductor.
ai-evals-course/evals-skills
Guides evaluation of a RAG system by diagnosing failures in traces, building a retrieval test set and scoring retrieval and generation separately.
agentevals-dev/agentevals
Evaluate and score agent behavior against a golden reference.
ai-evals-course/evals-skills
Builds diverse synthetic test inputs for LLM pipeline evaluation by defining failure-focused dimensions, drafting tuples with you and turning them into realistic queries.
Orchestra-Research/AI-Research-SKILLs
Benchmarks code generation models with the BigCode Evaluation Harness across HumanEval, MBPP, MultiPL-E and other suites using pass@k metrics.
affaan-m/ECC
Audits your installed Claude skills and commands for quality, with a quick mode for recently changed skills and a full mode that evaluates all of them through subagents.
affaan-m/ECC
Ingests, indexes, searches, edits and monitors video, audio and live streams through the VideoDB Python SDK, returning stream links, clips and timestamps.
affaan-m/ECC
Route broad documentation-governance requests to existing ECC skills and run an opt-in, read-only audit of mapped documentation roles, links, ADR indexes, and evidence references.
affaan-m/ECC
Scans installed skills for principles that recur across them and proposes rule-file changes: append, revise, add a section, create a file or leave as covered.
affaan-m/ECC
Builds DRAFT counterparty agreements from one markdown template and a small JSON spec per party, with clauses picked by the party's role.
affaan-m/ECC
Set an ECC-specific frontend design direction for production UI work.
用于医疗应用部署的患者安全评估工具。针对CDSS准确性、PHI暴露、临床工作流完整性和集成合规性的自动化测试套件。在安全故障时阻止部署。. Healthcare Eval Harness is an agent skill from affaan-m/ECC.
Healthcare Eval Harness fits situations like: tasks that involve LLM evaluation; tasks that involve Clinical and healthcare research.
Run `npx skills add affaan-m/ECC --skill healthcare-eval-harness -a claude-code`. Or copy the skill folder (docs/zh-CN/skills/healthcare-eval-harness in affaan-m/ECC) into .claude/skills/healthcare-eval-harness in your project. Claude Code loads it when a task matches its description.
Run `npx skills add affaan-m/ECC --skill healthcare-eval-harness -a codex`. Or copy the skill folder (docs/zh-CN/skills/healthcare-eval-harness in affaan-m/ECC) into .agents/skills/healthcare-eval-harness in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add affaan-m/ECC --skill healthcare-eval-harness -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/healthcare-eval-harness, .gemini/skills/healthcare-eval-harness, .github/skills/healthcare-eval-harness and .opencode/skills/healthcare-eval-harness in your project.
Going by SKILL.md and its folder, Healthcare Eval Harness needs the command-line tools its instructions call (npx and jq). Our summary lists: Node.js.
SKILL.md contains no URLs. Its commands use npx, which can reach the network depending on how they are called. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
Healthcare Eval Harness is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
About 1.4k tokens (SKILL.md is roughly 5.6k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
Skills that share tags, products or a category with Healthcare Eval Harness: Agent Eval Engineering (langchain-ai/langchain-skills, 1.3k stars), Skill Conductor (smixs/skill-conductor, 179 stars), Evaluate RAG (ai-evals-course/evals-skills, 1.5k stars) and Eval (agentevals-dev/agentevals, 163 stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
affaan-m (a GitHub user) maintains it in affaan-m/ECC, which has 276,111 GitHub stars. The repository holds 683 skills in this directory. The repository was last updated on October 10, 2026.
Source: affaan-m/ECC on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.