Official agent skill

Phoenix Evals

Build and run evaluators for AI/LLM applications using Phoenix.

OfficialApache-2.0Auto-check passedAI & LLM Engineering

Install Phoenix Evals

skills CLI

$ npx skills add github/awesome-copilot --skill phoenix-evals -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI

$ gh skill install github/awesome-copilot phoenix-evals --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy

$ git clone --depth 1 https://github.com/github/awesome-copilot.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/phoenix-evals .claude/skills/phoenix-evals && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Paste into Claude Code

Install the "phoenix-evals" agent skill from https://github.com/github/awesome-copilot/tree/main/skills/phoenix-evals into .claude/skills/phoenix-evals/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "phoenix-evals", then confirm the skill loads.

Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

$skill-installer

$skill-installer install https://github.com/github/awesome-copilot/tree/main/skills/phoenix-evals

Type this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.

skills CLI

$ npx skills add github/awesome-copilot --skill phoenix-evals -a codex

Project install goes to .agents/skills/; add -g for ~/.codex/skills/.

GitHub CLI

$ gh skill install github/awesome-copilot phoenix-evals --agent codex

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/github/awesome-copilot.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/phoenix-evals .agents/skills/phoenix-evals && rm -rf skills-src

Use ~/.agents/skills/ instead of .agents/skills for a personal install.

Codex skills documentation · loads skills from .agents/skills/

Paste into Codex

Install the "phoenix-evals" agent skill from https://github.com/github/awesome-copilot/tree/main/skills/phoenix-evals into .agents/skills/phoenix-evals/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "phoenix-evals", then confirm the skill loads.

Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add github/awesome-copilot --skill phoenix-evals -a cursor

Project install goes to .agents/skills/; add -g for ~/.cursor/skills/.

GitHub CLI

$ gh skill install github/awesome-copilot phoenix-evals --agent cursor

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/github/awesome-copilot.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/phoenix-evals .cursor/skills/phoenix-evals && rm -rf skills-src

Use ~/.cursor/skills/ instead of .cursor/skills for a personal install.

Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/

Paste into Cursor

Install the "phoenix-evals" agent skill from https://github.com/github/awesome-copilot/tree/main/skills/phoenix-evals into .cursor/skills/phoenix-evals/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "phoenix-evals", then confirm the skill loads.

Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

gemini skills install

$ gemini skills install https://github.com/github/awesome-copilot.git --path skills/phoenix-evals

--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.

skills CLI

$ npx skills add github/awesome-copilot --skill phoenix-evals -a gemini-cli

Project install goes to .agents/skills/; add -g for ~/.gemini/skills/.

GitHub CLI

$ gh skill install github/awesome-copilot phoenix-evals --agent gemini-cli

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/github/awesome-copilot.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/phoenix-evals .gemini/skills/phoenix-evals && rm -rf skills-src

Use ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.

Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/

Paste into Gemini CLI

Install the "phoenix-evals" agent skill from https://github.com/github/awesome-copilot/tree/main/skills/phoenix-evals into .gemini/skills/phoenix-evals/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "phoenix-evals", then confirm the skill loads.

Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

GitHub CLI

$ gh skill install github/awesome-copilot phoenix-evals

Installs for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).

skills CLI

$ npx skills add github/awesome-copilot --skill phoenix-evals -a github-copilot

Project install goes to .agents/skills/; add -g for ~/.copilot/skills/.

Manual copy

$ git clone --depth 1 https://github.com/github/awesome-copilot.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/phoenix-evals .github/skills/phoenix-evals && rm -rf skills-src

Use ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.

GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/

Paste into GitHub Copilot

Install the "phoenix-evals" agent skill from https://github.com/github/awesome-copilot/tree/main/skills/phoenix-evals into .github/skills/phoenix-evals/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "phoenix-evals", then confirm the skill loads.

GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

skills CLI

$ npx skills add github/awesome-copilot --skill phoenix-evals -a opencode

OpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.

GitHub CLI

$ gh skill install github/awesome-copilot phoenix-evals --agent opencode

Project scope by default (.agents/skills/); add --scope user for a personal install.

Manual copy

$ git clone --depth 1 https://github.com/github/awesome-copilot.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/phoenix-evals .opencode/skills/phoenix-evals && rm -rf skills-src

Use ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.

OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/

Paste into OpenCode

Install the "phoenix-evals" agent skill from https://github.com/github/awesome-copilot/tree/main/skills/phoenix-evals into .opencode/skills/phoenix-evals/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "phoenix-evals", then confirm the skill loads.

OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.

Facts

Skill name: phoenix-evals
GitHub stars: 40k
Used in: 3 other repos
Token cost: ~1.1k tokens
SKILL.md length: 189 words
Files: 35 (incl. references)
Skills in repo: 417
Repo updated: Oct 7, 2026
First seen: Oct 7, 2026
Licence: Apache-2.0

At a glance

Build and run evaluators for AI/LLM applications using Phoenix.

Tasks that involve LLM observability
SKILL.md covers Quick Reference, Workflows, Reference Categories and Key Principles
Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
Tasks that involve LLM evaluation

What it does

Phoenix Evals is an agent skill from github/awesome-copilot, published by the product's own GitHub organization. Build and run evaluators for AI/LLM applications using Phoenix.

Its SKILL.md is about 1.1k tokens, which your agent loads only when the skill is triggered. The skill folder holds 35 other files, including reference files (for example `references/axial-coding.md`, `references/common-mistakes-python.md` and `references/error-analysis-multi-turn.md`). Compatibility notes: Requires Phoenix server. Python skills need phoenix and openai packages; TypeScript skills need @arizeai/phoenix-client.

It sits in AI & LLM Engineering, covering LLM observability and LLM evaluation. It works with Python and TypeScript. The repository describes itself as: Community-contributed instructions, agents, skills, and configurations to help you make the most of GitHub Copilot. The licence is Apache-2.0.

When your agent uses it

Tasks that involve LLM observability
Tasks that involve LLM evaluation

Example prompts

“/phoenix-evals”

Requirements

Python 3
Compatibility (from SKILL.md): Requires Phoenix server. Python skills need phoenix and openai packages; TypeScript skills need @arizeai/phoenix-client.

What it can do on your machine

Read from SKILL.md and the folder at commit 727ff2e. It shows what the files ask for, not the result of running them.

Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
No scripts in the folder and no shell commands in SKILL.md.
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Compatibility
Requires Phoenix server. Python skills need phoenix and openai packages; TypeScript skills need @arizeai/phoenix-client.
From compatibility in the SKILL.md frontmatter.

Context cost

Phoenix Evals loads about 1.1k tokens when it runs, and up to ~23k if it reads all its reference files. Until then it costs about 19 tokens; SKILL.md has 189 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it: ~19
When it runs · the whole SKILL.md, loaded when a task matches: ~1.1k
With references · SKILL.md plus every file in references/, read only if the agent opens them: ~23k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from github/awesome-copilot at commit 727ff2e, republished under its Apache-2.0 licence (© github). 189 words, ~1,148 tokens.

Download SKILL.mdSave it as .claude/skills/phoenix-evals/SKILL.md (or your agent's skills folder). This skill also uses 34 other files; get the full folder from GitHub.

name: phoenix-evals
description: Build and run evaluators for AI/LLM applications using Phoenix.
compatibility: Requires Phoenix server. Python skills need phoenix and openai packages; TypeScript skills need @arizeai/phoenix-client.
license: Apache-2.0
metadata.author: oss@arize.com
metadata.version: 1.0.0
metadata.languages: Python, TypeScript

Phoenix Evals

Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.

Quick Reference

Task	Files
Setup	setup-python, setup-typescript
Decide what to evaluate	evaluators-overview
Choose a judge model	fundamentals-model-selection
Use pre-built evaluators	evaluators-pre-built
Build code evaluator	evaluators-code-python, evaluators-code-typescript
Build LLM evaluator	evaluators-llm-python, evaluators-llm-typescript, evaluators-custom-templates
Batch evaluate DataFrame	evaluate-dataframe-python
Understand experiments	experiments-overview
Run experiment	experiments-running-python, experiments-running-typescript
Create dataset	experiments-datasets-python, experiments-datasets-typescript
Generate synthetic data	experiments-synthetic-python, experiments-synthetic-typescript
Validate evaluator accuracy	validation, validation-evaluators-python, validation-evaluators-typescript
Sample traces for review	observe-sampling-python, observe-sampling-typescript
Analyze errors	error-analysis, error-analysis-multi-turn, axial-coding
RAG evals	evaluators-rag
Avoid common mistakes	common-mistakes-python, fundamentals-anti-patterns
Production	production-overview, production-guardrails, production-continuous

Workflows

Starting Fresh: observe-tracing-setup → error-analysis → axial-coding → evaluators-overview

Building Evaluator: fundamentals → common-mistakes-python → evaluators-{code|llm}-{python|typescript} → validation-evaluators-{python|typescript}

RAG Systems: evaluators-rag → evaluators-code-* (retrieval) → evaluators-llm-* (faithfulness)

Production: production-overview → production-guardrails → production-continuous

Reference Categories

Prefix	Description
`fundamentals-*`	Types, scores, anti-patterns
`observe-*`	Tracing, sampling
`error-analysis-*`	Finding failures
`axial-coding-*`	Categorizing failures
`evaluators-*`	Code, LLM, RAG evaluators
`experiments-*`	Datasets, running experiments
`validation-*`	Validating evaluator accuracy against human labels
`production-*`	CI/CD, monitoring

Key Principles

Principle	Action
Error analysis first	Can't automate what you haven't observed
Custom > generic	Build from your failures
Code first	Deterministic before LLM
Validate judges	>80% TPR/TNR
Binary > Likert	Pass/fail, not 1-5

Files

SKILL.md and 34 other files (references) in skills/phoenix-evals of github/awesome-copilot.

SKILL.md
references/axial-coding.md
references/common-mistakes-python.md
references/error-analysis-multi-turn.md
references/error-analysis.md
references/evaluate-dataframe-python.md
references/evaluators-code-python.md
references/evaluators-code-typescript.md
references/evaluators-custom-templates.md
references/evaluators-llm-python.md
references/evaluators-llm-typescript.md
references/evaluators-overview.md
references/evaluators-pre-built.md
references/evaluators-rag.md
references/experiments-datasets-python.md
references/experiments-datasets-typescript.md
references/experiments-overview.md
references/experiments-running-python.md
references/experiments-running-typescript.md
references/experiments-synthetic-python.md
… and 15 more

Open the folder on GitHubat commit 727ff2e

Used in 3 other repositories

We found 3 copies of this SKILL.md (exact, near-identical or edited) in other folders, from 3 other GitHub owners. This page covers the copy in github/awesome-copilot, which our catalogue first saw on October 7, 2026.

Compare with similar skills

Phoenix Evals next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Phoenix Evals compared with similar skills
Skill	Stars	Used in	Tokens	Auto-check	Licence	Repo updated
Phoenix Evals this skillgithub/awesome-copilot	40k	3 repos	~1.1k	Automated safety check: Pass	Apache-2.0	today
Failproof AI SDK IntegrationFailproofAI/failproofai	5.3k	—	~6k	Automated safety check: Pass	Custom licence	yesterday
Phoenix Skills AuditArize-ai/phoenix	12k	—	~5.1k	Automated safety check: Pass	Custom licence	today
Langchain Dependencieslangchain-ai/langchain-skills	1.3k	1 repos	~3.6k	Automated safety check: Pass	MIT	2 days ago
Opik External Integrationscomet-ml/opik	22k	—	~1.3k	Automated safety check: Pass	Apache-2.0	today
Opik SDK Integrationscomet-ml/opik	22k	—	~1.4k	Automated safety check: Pass	Apache-2.0	today

Similar skills

Failproof AI SDK Integration
FailproofAI/failproofai
Helps instrument a custom Python or TypeScript agent to record events for Failproof AI, verify what gets written, and run an evaluator worker that scores the runs.
5.3k GitHub stars~6k tokensUpdated yesterday
AI & LLM EngineeringAuto-check passed
Phoenix Skills Audit
Arize-ai/phoenix
Audit recent changes to Phoenix's user-facing surfaces (Python clients, TypeScript clients, CLI, REST/GraphQL APIs) and patch the three external-facing agent skills — phoenix-tracing, phoenix-cli…
12k GitHub stars~5.1k tokensUpdated today
AI & LLM EngineeringAuto-check passed
Langchain Dependencies
langchain-ai/langchain-skills
Official
INVOKE THIS SKILL when setting up a new project or when asked about package versions, installation, or dependency management for LangChain, LangGraph, LangSmith, or Deep Agents.
1.3k GitHub starsUsed in 1 repo~3.6k tokens
AI & LLM EngineeringAuto-check passed
Opik External Integrations
comet-ml/opik
Builds Opik tracing integrations that live outside the Opik repository, either as standalone opik-* packages or as contributions into projects such as LiteLLM or Dify.
22k GitHub stars~1.3k tokensUpdated today
AI & LLM EngineeringAuto-check passed
Opik SDK Integrations
comet-ml/opik
Builds, updates and verifies integrations inside the Opik Python and TypeScript SDKs so users can trace a framework or provider with one call.
22k GitHub stars~1.4k tokensUpdated today
AI & LLM EngineeringAuto-check passed
Phoenix Integration Snippets
Arize-ai/phoenix
Generates onboarding code snippets for Phoenix tracing integrations and wires them into the project onboarding UI.
12k GitHub stars~1.4k tokensUpdated today
AI & LLM EngineeringAuto-check passed

Questions about Phoenix Evals

What does Phoenix Evals do?

Build and run evaluators for AI/LLM applications using Phoenix. Phoenix Evals is an agent skill from github/awesome-copilot, published by the product's own GitHub organization. Build and run evaluators for AI/LLM applications using Phoenix.

When should I use Phoenix Evals?

Phoenix Evals fits situations like: tasks that involve LLM observability; tasks that involve LLM evaluation.

How do I install Phoenix Evals in Claude Code?

Run `npx skills add github/awesome-copilot --skill phoenix-evals -a claude-code`. Or copy the skill folder (skills/phoenix-evals in github/awesome-copilot) into .claude/skills/phoenix-evals in your project. Claude Code loads it when a task matches its description.

How do I install Phoenix Evals in Codex?

Run `npx skills add github/awesome-copilot --skill phoenix-evals -a codex`. Or copy the skill folder (skills/phoenix-evals in github/awesome-copilot) into .agents/skills/phoenix-evals in your project. Codex loads it when a task matches its description.

Can I use Phoenix Evals in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add github/awesome-copilot --skill phoenix-evals -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/phoenix-evals, .gemini/skills/phoenix-evals, .github/skills/phoenix-evals and .opencode/skills/phoenix-evals in your project.

What does Phoenix Evals need to run?

SKILL.md names no scripts, command-line tools or credentials: Phoenix Evals is instructions for the agent only. Our summary lists: Python 3. Compatibility (from SKILL.md): Requires Phoenix server. Python skills need phoenix and openai packages; TypeScript skills need @arizeai/phoenix-client..

Does Phoenix Evals access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Phoenix Evals safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Phoenix Evals use?

Phoenix Evals is published under the Apache-2.0 licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Phoenix Evals use?

About 1.1k tokens (SKILL.md is roughly 4.6k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 22k tokens, read only when the agent opens those files.

What are the alternatives to Phoenix Evals?

Skills that share tags, products or a category with Phoenix Evals: Failproof AI SDK Integration (FailproofAI/failproofai, 5.3k stars), Phoenix Skills Audit (Arize-ai/phoenix, 12k stars), Langchain Dependencies (langchain-ai/langchain-skills, 1.3k stars) and Opik External Integrations (comet-ml/opik, 22k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Phoenix Evals?

github (a GitHub organization, an official publisher) maintains it in github/awesome-copilot, which has 39,748 GitHub stars. The repository holds 417 skills in this directory. The repository was last updated on October 7, 2026.

Source: github/awesome-copilot on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.

Phoenix Evals

Install Phoenix Evals

Facts

At a glance

What it does

When your agent uses it

Example prompts

Requirements

What it can do on your machine

Tool permissions

Runs code

Network

Credentials

Compatibility

Context cost

Safety

SKILL.md

Phoenix Evals

Quick Reference

Workflows

Reference Categories

Key Principles

Files

Used in 3 other repositories

Compare with similar skills

Similar skills

Failproof AI SDK Integration

Phoenix Skills Audit

Langchain Dependencies

Opik External Integrations

Opik SDK Integrations

Phoenix Integration Snippets

More from github/awesome-copilot

Acquire Codebase Knowledge

Azure Architecture Autopilot

Draw.io Diagram Generator

Credit Risk Data Cleaning

Daily Focus Board

Python Pypi Package Builder

Questions about Phoenix Evals

Install Phoenix Evals

Facts

At a glance

What it does

When your agent uses it

Example prompts

Requirements

What it can do on your machine

Tool permissions

Runs code

Network

Credentials

Compatibility

Context cost

Safety

SKILL.md

Phoenix Evals

Quick Reference

Workflows

Reference Categories

Key Principles

Files

Used in 3 other repositories

Compare with similar skills

Similar skills

More from github/awesome-copilot

Related

Questions about Phoenix Evals