LLM Benchmarking with lm-evaluation-harness
Orchestra-Research/AI-Research-SKILLs
Runs lm-evaluation-harness to benchmark language models on academic suites such as MMLU, GSM8K and HumanEval, compare models and track training checkpoints.
在明确的发现 scope 中构建受限的 Agent Skill 目录，并在加载指令正文前报告冲突. An agent skill from fancyboi999/ai-engineering-from-scratch-zh.
Install Skill Catalog Builder
$ npx skills add fancyboi999/ai-engineering-from-scratch-zh --skill skill-catalog-builder -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install fancyboi999/ai-engineering-from-scratch-zh skill-catalog-builder --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/fancyboi999/ai-engineering-from-scratch-zh.git skills-src && mkdir -p .claude/skills && cp -r skills-src/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder .claude/skills/skill-catalog-builder && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "skill-catalog-builder" agent skill from https://github.com/fancyboi999/ai-engineering-from-scratch-zh/tree/main/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder into .claude/skills/skill-catalog-builder/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "skill-catalog-builder", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/fancyboi999/ai-engineering-from-scratch-zh/tree/main/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builderType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add fancyboi999/ai-engineering-from-scratch-zh --skill skill-catalog-builder -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install fancyboi999/ai-engineering-from-scratch-zh skill-catalog-builder --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/fancyboi999/ai-engineering-from-scratch-zh.git skills-src && mkdir -p .agents/skills && cp -r skills-src/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder .agents/skills/skill-catalog-builder && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "skill-catalog-builder" agent skill from https://github.com/fancyboi999/ai-engineering-from-scratch-zh/tree/main/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder into .agents/skills/skill-catalog-builder/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "skill-catalog-builder", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add fancyboi999/ai-engineering-from-scratch-zh --skill skill-catalog-builder -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install fancyboi999/ai-engineering-from-scratch-zh skill-catalog-builder --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/fancyboi999/ai-engineering-from-scratch-zh.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder .cursor/skills/skill-catalog-builder && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "skill-catalog-builder" agent skill from https://github.com/fancyboi999/ai-engineering-from-scratch-zh/tree/main/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder into .cursor/skills/skill-catalog-builder/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "skill-catalog-builder", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/fancyboi999/ai-engineering-from-scratch-zh.git --path phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add fancyboi999/ai-engineering-from-scratch-zh --skill skill-catalog-builder -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install fancyboi999/ai-engineering-from-scratch-zh skill-catalog-builder --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/fancyboi999/ai-engineering-from-scratch-zh.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder .gemini/skills/skill-catalog-builder && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "skill-catalog-builder" agent skill from https://github.com/fancyboi999/ai-engineering-from-scratch-zh/tree/main/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder into .gemini/skills/skill-catalog-builder/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "skill-catalog-builder", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install fancyboi999/ai-engineering-from-scratch-zh skill-catalog-builderInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add fancyboi999/ai-engineering-from-scratch-zh --skill skill-catalog-builder -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/fancyboi999/ai-engineering-from-scratch-zh.git skills-src && mkdir -p .github/skills && cp -r skills-src/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder .github/skills/skill-catalog-builder && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "skill-catalog-builder" agent skill from https://github.com/fancyboi999/ai-engineering-from-scratch-zh/tree/main/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder into .github/skills/skill-catalog-builder/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "skill-catalog-builder", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add fancyboi999/ai-engineering-from-scratch-zh --skill skill-catalog-builder -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install fancyboi999/ai-engineering-from-scratch-zh skill-catalog-builder --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/fancyboi999/ai-engineering-from-scratch-zh.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder .opencode/skills/skill-catalog-builder && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "skill-catalog-builder" agent skill from https://github.com/fancyboi999/ai-engineering-from-scratch-zh/tree/main/phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder into .opencode/skills/skill-catalog-builder/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "skill-catalog-builder", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
skill-catalog-builder- GitHub stars
- 1.2k
- Token cost
- ~133 tokens
- SKILL.md length
- 38 words
- Files
- 4 (incl. scripts, references, assets)
- Skills in repo
- 14
- Repo updated
- First seen
- Licence
- MIT
At a glance
在明确的发现 scope 中构建受限的 Agent Skill 目录，并在加载指令正文前报告冲突. An agent skill from fancyboi999/ai-engineering-from-scratch-zh.
- Works in 5 steps: 阅读 references/discovery-contract.md。 → 查看 assets/scope-policy.json… → 以从最高到最低的优先级列出 scope，运行 python3… → …
- AI & LLM Engineering work in your project
- Runs Python scripts from its folder; calls python3
What it does
Skill Catalog Builder is an agent skill from fancyboi999/ai-engineering-from-scratch-zh. 在明确的发现 scope 中构建受限的 Agent Skill 目录，并在加载指令正文前报告冲突。
Its SKILL.md is about 130 tokens, which your agent loads only when the skill is triggered. The skill folder holds 6 other files, including scripts, reference files and assets (for example `assets/scope-policy.json`, `references/discovery-contract.md` and `scripts/build_catalog.py`).
It sits in AI & LLM Engineering. It works with Python. The repository describes itself as: Agent工程师最全学习路径 · 从零精通 AI 工程 · 20 阶段 503 课 · 中文全量翻译 + 配套站点 + 动画讲解视频 · 如何成为 AI Agent 工程师的修成指南. The licence is MIT.
When your agent uses it
- AI & LLM Engineering work in your project
Example prompts
- “/skill-catalog-builder”
Requirements
- Python 3
Workflow steps
5 steps, taken from the first numbered list in SKILL.md.
- 阅读 references/discovery-contract.md。
- 查看 assets/scope-policy.json 中的宿主策略示例；不要假设其顺序具有通用性。
- 以从最高到最低的优先级列出 scope，运行 python3 scripts/build_catalog.py project=PATH user=PATH。
- 激活 skill 前检查 JSON 的 collisions 和 omitted 数组。
- 只加载选中的 SKILL.md 正文；仅当正文点名时才加载直接引用。
What it can do on your machine
Read from SKILL.md and the folder at commit 94b9888. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
Ships 1 file in scripts/ (Python), which the agent can run.
Shell commands in SKILL.md call:
python3
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Skill Catalog Builder loads about 133 tokens when it runs, and up to ~280 if it reads all its reference files. Until then it costs about 18 tokens; SKILL.md has 38 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
SKILL.md
The full file from fancyboi999/ai-engineering-from-scratch-zh at commit 94b9888, republished under its MIT licence (© fancyboi999). 38 words, ~133 tokens.
Download SKILL.mdSave it as
.claude/skills/skill-catalog-builder/SKILL.md (or your agent's skills folder). This skill also uses 3 other files; get the full folder from GitHub.- name
- skill-catalog-builder
- description
- 在明确的发现 scope 中构建受限的 Agent Skill 目录，并在加载指令正文前报告冲突。
- license
- MIT
- metadata.lesson
- 24
Skill 目录构建器
当 agent 宿主需要跨多个 skill 目录进行确定性发现时使用此 skill。
- 阅读
references/discovery-contract.md。 - 查看
assets/scope-policy.json中的宿主策略示例；不要假设其顺序具有通用性。 - 以从最高到最低的优先级列出 scope，运行
python3 scripts/build_catalog.py project=PATH user=PATH。 - 激活 skill 前检查 JSON 的
collisions和omitted数组。 - 只加载选中的 SKILL.md 正文；仅当正文点名时才加载直接引用。
发现期间绝不执行打包脚本；绝不按偶然的文件系统顺序选择同优先级重复项。
返回目录预算、选中条目、冲突解决结果和省略项。
© fancyboi999, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 3 other files (scripts, references, assets) in phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder of fancyboi999/ai-engineering-from-scratch-zh.
- SKILL.md
- assets/scope-policy.json
- references/discovery-contract.md
- scripts/build_catalog.py
Open the folder on GitHubat commit 94b9888
Compare with similar skills
Skill Catalog Builder next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Skill Catalog Builder this skillfancyboi999/ai-engineering-from-scratch-zh | 1.2k | — | ~133 | Automated safety check: Pass | MIT | |
| LLM Benchmarking with lm-evaluation-harnessOrchestra-Research/AI-Research-SKILLs | 13k | 8 repos | ~3k | Automated safety check: Pass | MIT | |
| Segment Anything Model GuideOrchestra-Research/AI-Research-SKILLs | 13k | 8 repos | ~3.3k | Automated safety check: Pass | MIT | |
| Chroma Vector DatabaseOrchestra-Research/AI-Research-SKILLs | 13k | 7 repos | ~2.3k | Automated safety check: Pass | MIT | |
| CLIP Image-Text MatchingOrchestra-Research/AI-Research-SKILLs | 13k | 7 repos | ~1.7k | Automated safety check: Pass | MIT | |
| Paddle Design DistributedPaddlePaddle/Paddle | 24k | — | ~660 | Automated safety check: Pass | Apache-2.0 |
Similar skills
Segment Anything Model Guide
Orchestra-Research/AI-Research-SKILLs
Guide to using Meta's Segment Anything Model for zero-shot image segmentation with point, box or mask prompts, or automatic mask generation.
Chroma Vector Database
Orchestra-Research/AI-Research-SKILLs
Shows how to store documents and embeddings in Chroma, query them by similarity with metadata filters, and persist them to disk for RAG and semantic search projects.
CLIP Image-Text Matching
Orchestra-Research/AI-Research-SKILLs
Explains OpenAI's CLIP model for zero-shot image classification, image-text similarity, semantic image search and content moderation, with install steps and code patterns.
Paddle Design Distributed
PaddlePaddle/Paddle
A skill your agent uses when working with Paddle's distributed training system: understanding parallelism strategies (DP, ZeRO, TP, PP, SP), semi-automatic parallel with ProcessMesh + shardtensor…
Onnxtxt
onnx/onnx
Read or write ONNX text format ("onnxtxt"). An agent skill from onnx/onnx.
More from fancyboi999/ai-engineering-from-scratch-zh
All 14 skills in this repoSkill Release Gate
fancyboi999/ai-engineering-from-scratch-zh
在发布前评估 Agent Skill bundle 的结构完整性、触发质量、产物改进、脚本正确性、安全性、已安装目录树完整性和目标宿主可移植性。
Find Your Level
fancyboi999/ai-engineering-from-scratch-zh
交互式测验，将你的 AI/ML 知识映射到 523 节课、20 个阶段的 AI Engineering from Scratch 课程起点。
Check Understanding
fancyboi999/ai-engineering-from-scratch-zh
AI Engineering from Scratch 的阶段测验。用于“给我测验一下”、“测试阶段”、“检查我的理解”、“我掌握第 3 阶段了吗”，也支持 "quiz me", "test phase", "check my understanding", "do I know phase 3" 或 /check-understanding <phase。
Claude Certification
fancyboi999/ai-engineering-from-scratch-zh
AI Engineering from Scratch 中四条独立 Claude 认证路线的 AI 原生导师与入门流程。适用于学习者 希望选择 Claude 认证、备考 CCAO-F、CCDV-F、CCAR-F 或 CCAR-P、继续认证路径、以交互方式学习 下一课、运行并验证实践实验、构建并评分产物、参加诊断或模拟测评，或通过 GitHub 上的 Claude Code、…
Course Guide
fancyboi999/ai-engineering-from-scratch-zh
AI Engineering from Scratch 课程的主题路由器。给它一个主题、问题或正在处理的 bug， 它会指出精确教授它的课程，以及下一条正确命令。触发短语： “在哪里学习”、“哪节课涵盖”、“课程导航”、“我卡在”、“接下来该做什么”、 “教我 MCP”、“教我 Agent Skills”、“在哪里准备 Claude certification”，或 "where do I…
Learn
fancyboi999/ai-engineering-from-scratch-zh
AI Engineering from Scratch 课程的交互式课程 tutor。读取 LEARNING.md，获取下一课， 在终端按章节教学，结尾测验并记录进度。可在克隆仓库中或完全通过 raw.githubusercontent.com 工作—— 无需设置。触发短语：“下一课”、“教我”、“继续课程”、“我们来学习”、“继续学习”，或 "next lesson", "teach…
Related
Works with
Categories
Questions about Skill Catalog Builder
What does Skill Catalog Builder do?
在明确的发现 scope 中构建受限的 Agent Skill 目录，并在加载指令正文前报告冲突. An agent skill from fancyboi999/ai-engineering-from-scratch-zh. Skill Catalog Builder is an agent skill from fancyboi999/ai-engineering-from-scratch-zh.
When should I use Skill Catalog Builder?
Skill Catalog Builder fits situations like: AI & LLM Engineering work in your project.
How do I install Skill Catalog Builder in Claude Code?
Run `npx skills add fancyboi999/ai-engineering-from-scratch-zh --skill skill-catalog-builder -a claude-code`. Or copy the skill folder (phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder in fancyboi999/ai-engineering-from-scratch-zh) into .claude/skills/skill-catalog-builder in your project. Claude Code loads it when a task matches its description.
How do I install Skill Catalog Builder in Codex?
Run `npx skills add fancyboi999/ai-engineering-from-scratch-zh --skill skill-catalog-builder -a codex`. Or copy the skill folder (phases/13-tools-and-protocols/24-skill-discovery-and-progressive-disclosure/outputs/skill-catalog-builder in fancyboi999/ai-engineering-from-scratch-zh) into .agents/skills/skill-catalog-builder in your project. Codex loads it when a task matches its description.
Can I use Skill Catalog Builder in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add fancyboi999/ai-engineering-from-scratch-zh --skill skill-catalog-builder -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/skill-catalog-builder, .gemini/skills/skill-catalog-builder, .github/skills/skill-catalog-builder and .opencode/skills/skill-catalog-builder in your project.
What does Skill Catalog Builder need to run?
Going by SKILL.md and its folder, Skill Catalog Builder needs Python for the scripts in its folder and the command-line tools its instructions call (python3). Our summary lists: Python 3.
Does Skill Catalog Builder access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Skill Catalog Builder safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
What licence does Skill Catalog Builder use?
Skill Catalog Builder is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Skill Catalog Builder use?
About 133 tokens (SKILL.md is roughly 532 characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 147 tokens, read only when the agent opens those files.
What are the alternatives to Skill Catalog Builder?
Skills that share tags, products or a category with Skill Catalog Builder: LLM Benchmarking with lm-evaluation-harness (Orchestra-Research/AI-Research-SKILLs, 13k stars), Segment Anything Model Guide (Orchestra-Research/AI-Research-SKILLs, 13k stars), Chroma Vector Database (Orchestra-Research/AI-Research-SKILLs, 13k stars) and CLIP Image-Text Matching (Orchestra-Research/AI-Research-SKILLs, 13k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Skill Catalog Builder?
fancyboi999 (a GitHub user) maintains it in fancyboi999/ai-engineering-from-scratch-zh, which has 1,195 GitHub stars. The repository holds 14 skills in this directory. The repository was last updated on October 9, 2026.
Source: fancyboi999/ai-engineering-from-scratch-zh on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.