Triage Agent Eval Failures
novuhq/novu
Triage failing @novu/agent-evals scenarios to decide whether a failure is real or flaky, and whether to fix the playbook/prompt or the test (grader, tape, scenario, or judge).
[omh] Failing run handed to Jev for a next move: Jev failure triage: retry, fix a dependency, ask for access, or change approach on a failing run.
MITAuto-check passed
Install Omh Jev Failure Triage
$ npx skills add rlaope/oh-my-hermes --skill omh-jev-failure-triage -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install rlaope/oh-my-hermes omh-jev-failure-triage --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/rlaope/oh-my-hermes.git skills-src && mkdir -p .claude/skills && cp -r skills-src/skills/omh-jev-failure-triage .claude/skills/omh-jev-failure-triage && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "omh-jev-failure-triage" agent skill from https://github.com/rlaope/oh-my-hermes/tree/main/skills/omh-jev-failure-triage into .claude/skills/omh-jev-failure-triage/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "omh-jev-failure-triage", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/rlaope/oh-my-hermes/tree/main/skills/omh-jev-failure-triageType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add rlaope/oh-my-hermes --skill omh-jev-failure-triage -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install rlaope/oh-my-hermes omh-jev-failure-triage --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/rlaope/oh-my-hermes.git skills-src && mkdir -p .agents/skills && cp -r skills-src/skills/omh-jev-failure-triage .agents/skills/omh-jev-failure-triage && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "omh-jev-failure-triage" agent skill from https://github.com/rlaope/oh-my-hermes/tree/main/skills/omh-jev-failure-triage into .agents/skills/omh-jev-failure-triage/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "omh-jev-failure-triage", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add rlaope/oh-my-hermes --skill omh-jev-failure-triage -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install rlaope/oh-my-hermes omh-jev-failure-triage --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/rlaope/oh-my-hermes.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/skills/omh-jev-failure-triage .cursor/skills/omh-jev-failure-triage && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "omh-jev-failure-triage" agent skill from https://github.com/rlaope/oh-my-hermes/tree/main/skills/omh-jev-failure-triage into .cursor/skills/omh-jev-failure-triage/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "omh-jev-failure-triage", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/rlaope/oh-my-hermes.git --path skills/omh-jev-failure-triage--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add rlaope/oh-my-hermes --skill omh-jev-failure-triage -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install rlaope/oh-my-hermes omh-jev-failure-triage --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/rlaope/oh-my-hermes.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/skills/omh-jev-failure-triage .gemini/skills/omh-jev-failure-triage && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "omh-jev-failure-triage" agent skill from https://github.com/rlaope/oh-my-hermes/tree/main/skills/omh-jev-failure-triage into .gemini/skills/omh-jev-failure-triage/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "omh-jev-failure-triage", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install rlaope/oh-my-hermes omh-jev-failure-triageInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add rlaope/oh-my-hermes --skill omh-jev-failure-triage -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/rlaope/oh-my-hermes.git skills-src && mkdir -p .github/skills && cp -r skills-src/skills/omh-jev-failure-triage .github/skills/omh-jev-failure-triage && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "omh-jev-failure-triage" agent skill from https://github.com/rlaope/oh-my-hermes/tree/main/skills/omh-jev-failure-triage into .github/skills/omh-jev-failure-triage/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "omh-jev-failure-triage", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add rlaope/oh-my-hermes --skill omh-jev-failure-triage -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install rlaope/oh-my-hermes omh-jev-failure-triage --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/rlaope/oh-my-hermes.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/skills/omh-jev-failure-triage .opencode/skills/omh-jev-failure-triage && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "omh-jev-failure-triage" agent skill from https://github.com/rlaope/oh-my-hermes/tree/main/skills/omh-jev-failure-triage into .opencode/skills/omh-jev-failure-triage/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "omh-jev-failure-triage", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
Facts
- Skill name
omh-jev-failure-triage- GitHub stars
- 3.2k
- Token cost
- ~673 tokens
- SKILL.md length
- 289 words
- Files
- 3 (incl. references)
- Skills in repo
- 143
- Repo updated
- First seen
- Licence
- MIT
At a glance
[omh] Failing run handed to Jev for a next move: Jev failure triage: retry, fix a dependency, ask for access, or change approach on a failing run.
- Works in 4 steps: Name what leaves the machine: the… → Call omh_jev_ask with preset:… → Report policy_result.outcome and its… → …
- The user says: jev-failure-triage
- SKILL.md covers Procedure, Completion Checklist, Recovery Notes and Workflow Lane
- Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md
What it does
Omh Jev Failure Triage is an agent skill from rlaope/oh-my-hermes. [omh] Failing run handed to Jev for a next move: Jev failure triage: retry, fix a dependency, ask for access, or change approach on a failing run. Use when the user says: jev-failure-triage, jev failure triage, ask jev if this failure is transient.
Its SKILL.md is about 670 tokens, which your agent loads only when the skill is triggered. The skill folder holds 3 other files, including reference files (for example `references/full-contract.md` and `references/preset.md`).
The repository describes itself as: All in one plugin for Hermes Agent ⚚ the coding intelligence, a long-term memory system and model optimized workflow packages. The licence is MIT.
When your agent uses it
- The user says: jev-failure-triage
- Jev failure triage
- Ask jev if this failure is transient
Example prompts
- “/omh-jev-failure-triage”
Workflow steps
4 steps, taken from the first numbered list in SKILL.md.
- Name what leaves the machine: the command and an error excerpt (plus earlier attempt excerpts when repeating).
- Call omh_jev_ask with preset: failure_triage/v1, state {error_excerpt, last_command, earlier_attempts}, and attempts_so_far.
- Report policy_result.outcome and its rule beside the raw answers.
- Hand the fix to build-failure-triage or agent-debug; a rerun still needs the host's approval.
What it can do on your machine
Read from SKILL.md and the folder at commit 7cd0d02. It shows what the files ask for, not the result of running them.
Tool permissions
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Runs code
No scripts in the folder and no shell commands in SKILL.md.
From the folder's file list and the shell code blocks in SKILL.md.
Network
No URLs in SKILL.md.
From URLs in SKILL.md, links to its own repository left out.
Credentials
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Context cost
Omh Jev Failure Triage loads about 673 tokens when it runs, and up to ~1.9k if it reads all its reference files. Until then it costs about 68 tokens; SKILL.md has 289 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
Safety
Auto-check passed
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.
SKILL.md
The full file from rlaope/oh-my-hermes at commit 7cd0d02, republished under its MIT licence (© rlaope). 289 words, ~673 tokens.
Download SKILL.mdSave it as
.claude/skills/omh-jev-failure-triage/SKILL.md (or your agent's skills folder). This skill also uses 2 other files; get the full folder from GitHub.- name
- omh-jev-failure-triage
- description
- [omh] Failing run handed to Jev for a next move: Jev failure triage: retry, fix a dependency, ask for access, or change approach on a failing run. Use when the user says: jev-failure-triage, jev failure triage, ask jev if this failure is transient.
Jev Failure Triage
Required: the failing command, an error excerpt, and the retry count. Output: failure_triage/v1 next move with its rule.
Read omh-routing/references/jev-rail.md before the first ask: consent, what leaves the machine, limits, and non-answer statuses. Prepared OMH routing is not execution or approval. Shared product, routing, compatibility, and evidence rules: omh-routing/references/skill-common-rail.md.
Contract: references/full-contract.md. Preset: references/preset.md.
Procedure
- Name what leaves the machine: the command and an error excerpt (plus earlier attempt excerpts when repeating).
- Call
omh_jev_askwithpreset: failure_triage/v1,state{error_excerpt, last_command, earlier_attempts}, andattempts_so_far. - Report
policy_result.outcomeand its rule beside the raw answers. - Hand the fix to
build-failure-triageoragent-debug; a rerun still needs the host's approval.
Completion Checklist
- Preserve workflow intent and stop conditions; load the full contract before claiming completion.
- Reply in the user's own words and the host's own voice: its SOUL.md persona owns reply language, tone, speech level, and sentence endings, progress updates included (where it sets no language, use the one the user wrote in), and OMH shapes structure and content only; OMH's record terms (surface, lane, wrapper, handoff, evidence boundary, not_observed) stay in records and tool calls, never in the sentence the user reads unless they ask about one; and when a stop condition or a decision the user owns ends the turn, offer the next action as a question rather than declaring what will not be done.
- Report the tool's status, ask_id, served model, and cost source; a non-answer is
not_observedwithrule: not_answered:<status>, never Jev's answer. - Record observed delegation results; use Hermes-native subagent/delegation features when available: native subagents -> Hermes delegation when available, otherwise sequential lanes.
Recovery Notes
- Tool missing,
consent_not_observed, or refused: follow the rail's recovery section; never simulate an answer.
Workflow Lane
advisory local context
© rlaope, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
Files
SKILL.md and 2 other files (references) in skills/omh-jev-failure-triage of rlaope/oh-my-hermes.
- SKILL.md
- references/full-contract.md
- references/preset.md
Open the folder on GitHubat commit 7cd0d02
Compare with similar skills
Omh Jev Failure Triage next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Omh Jev Failure Triage this skillrlaope/oh-my-hermes | 3.2k | — | ~673 | Automated safety check: Pass | MIT | |
| Triage Agent Eval Failuresnovuhq/novu | 40k | — | ~1.5k | Automated safety check: Pass | Custom licence | |
| Triaging Merge Queue FailuresPostHog/posthog | 40k | — | ~9.5k | Automated safety check: Pass | Custom licence | |
| Triage CI FailureDataDog/datadog-agent | 3.8k | — | ~2.3k | Automated safety check: Pass | Apache-2.0 | |
| PR Status Triagevercel/next.js | 143k | — | ~866 | Automated safety check: Pass | MIT | |
| Triage CI Flakepayloadcms/payload | 45k | — | ~4.4k | Automated safety check: Pass | MIT |
Similar skills
- Official
Triaging Merge Queue Failures
PostHog/posthog
Decision procedure for a PR that failed or was removed from the Trunk merge queue: classify the kick (superseded by a newer commit, not mergeable, a gate that failed on a cancelled run, real…
- Official
Triage CI Failure
DataDog/datadog-agent
Classify a failed CI as either caused by an active incident, flakiness, or a true code regression.
- Official
PR Status Triage
vercel/next.js
Triage CI failures and PR review comments using scripts/pr-status.js.
Triage CI Flake
payloadcms/payload
A skill your agent uses when CI tests fail on main branch after PR merge, when investigating flaky test failures, or when user provides a PR URL/number to aggregate all failing tests
Triaging Issues
pytorch/pytorch
Triages GitHub issues by routing to oncall teams, applying labels, and closing questions.
More from rlaope/oh-my-hermes
All 143 skills in this repoOmh Accessibility Audit
rlaope/oh-my-hermes
[omh] Screen-reader or keyboard accessibility gaps: prepare WCAG, keyboard, focus, screen-reader, target-size, and reflow evidence gates for UI surfaces.
Omh Agent Debug
rlaope/oh-my-hermes
[omh] Agent is stuck, looping, or drifting: capture a stuck, looping, drifting, or repeatedly failing agent run, diagnose the likely failure pattern, and prepare the smallest safe recovery action.
Omh Agent Evaluation
rlaope/oh-my-hermes
[omh] Choosing between coding agents on evidence: compare executor or agent choices on reproducible tasks using quality, cost, time, tool, and evidence metrics.
Omh Agent Instructions
rlaope/oh-my-hermes
[omh] Agent instruction file for a repo -- AGENTS.md, CLAUDE.md, a Cursor rule: write or update what an agent cannot derive from the code, inside a marked region, with every command verified or…
Omh Agent Ops Review
rlaope/oh-my-hermes
[omh] AI agent progress for managers: help managers inspect AI-agent progress, blockers, quality gates, and throughput levers.
Omh AI Slop Cleaner
rlaope/oh-my-hermes
[omh] Messy or AI-generated code to clean up: delete AI-generated slop, dead code, and duplication while observable behavior stays identical.
Questions about Omh Jev Failure Triage
What does Omh Jev Failure Triage do?
[omh] Failing run handed to Jev for a next move: Jev failure triage: retry, fix a dependency, ask for access, or change approach on a failing run. Omh Jev Failure Triage is an agent skill from rlaope/oh-my-hermes. [omh] Failing run handed to Jev for a next move: Jev failure triage: retry, fix a dependency, ask for access, or change approach on a failing run.
When should I use Omh Jev Failure Triage?
Omh Jev Failure Triage fits situations like: the user says: jev-failure-triage; jev failure triage; ask jev if this failure is transient.
How do I install Omh Jev Failure Triage in Claude Code?
Run `npx skills add rlaope/oh-my-hermes --skill omh-jev-failure-triage -a claude-code`. Or copy the skill folder (skills/omh-jev-failure-triage in rlaope/oh-my-hermes) into .claude/skills/omh-jev-failure-triage in your project. Claude Code loads it when a task matches its description.
How do I install Omh Jev Failure Triage in Codex?
Run `npx skills add rlaope/oh-my-hermes --skill omh-jev-failure-triage -a codex`. Or copy the skill folder (skills/omh-jev-failure-triage in rlaope/oh-my-hermes) into .agents/skills/omh-jev-failure-triage in your project. Codex loads it when a task matches its description.
Can I use Omh Jev Failure Triage in Cursor, Gemini CLI or GitHub Copilot?
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add rlaope/oh-my-hermes --skill omh-jev-failure-triage -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/omh-jev-failure-triage, .gemini/skills/omh-jev-failure-triage, .github/skills/omh-jev-failure-triage and .opencode/skills/omh-jev-failure-triage in your project.
What does Omh Jev Failure Triage need to run?
SKILL.md names no scripts, command-line tools or credentials: Omh Jev Failure Triage is instructions for the agent only.
Does Omh Jev Failure Triage access the network?
SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.
Is Omh Jev Failure Triage safe to install?
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.
What licence does Omh Jev Failure Triage use?
Omh Jev Failure Triage is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
How many tokens does Omh Jev Failure Triage use?
About 673 tokens (SKILL.md is roughly 2.7k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 1.2k tokens, read only when the agent opens those files.
What are the alternatives to Omh Jev Failure Triage?
Skills that share tags, products or a category with Omh Jev Failure Triage: Triage Agent Eval Failures (novuhq/novu, 40k stars), Triaging Merge Queue Failures (PostHog/posthog, 40k stars), Triage CI Failure (DataDog/datadog-agent, 3.8k stars) and PR Status Triage (vercel/next.js, 143k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
Who maintains Omh Jev Failure Triage?
rlaope (a GitHub user) maintains it in rlaope/oh-my-hermes, which has 3,243 GitHub stars. The repository holds 143 skills in this directory. The repository was last updated on October 10, 2026.
Source: rlaope/oh-my-hermes on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.