Gaia Debugging

What it does

When your agent uses it

• A GAIA benchmark run reports a failed/incorrect taskid and you need to root-cause it before resubmitting
• Tasks that involve Debugging
• Tasks that involve Root cause analysis

Example prompts

• “/gaia-debugging”

Requirements

• Node.js
• A credential in GOOGLE_AI_API_KEY
• Pre-approved tools (allowed-tools): Bash, Read, mcp__plugin_ruflo-core_ruflo__memory_search, mcp__plugin_ruflo-core_ruflo__memory_store, mcp__plugin_ruflo-core_ruflo__agentdb_pattern_search, mcp__plugin_ruflo-core_ruflo__agentdb_pattern_store

Workflow steps

5 steps, taken from the step headings in SKILL.md.

1. 1Load the question trace
2. 2Classify the failure
3. 3Re-run with extended logging
4. 4Apply targeted fix
5. 5Verify fix and store pattern

What it can do on your machine

Read from SKILL.md and the folder at commit 6051f67. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves these tools, so the agent can use them without asking each time:

  • Bash
  • Read
  • mcp__plugin_ruflo-core_ruflo__memory_search
  • mcp__plugin_ruflo-core_ruflo__memory_store
  • mcp__plugin_ruflo-core_ruflo__agentdb_pattern_search
  • mcp__plugin_ruflo-core_ruflo__agentdb_pattern_store

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • node
  • npx

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md. Its commands use npx, which can reach the network depending on how they are called.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names these keys or tokens, usually read from environment variables:

  • GOOGLE_AI_API_KEY

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Gaia Debugging loads about 1.1k tokens when it runs. Until then it costs about 56 tokens; SKILL.md has 339 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~56

When it runs · the whole SKILL.md, loaded when a task matches

~1.1k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

• NotePre-approves every shell command (allowed-tools: Bash)SKILL.md

  allowed-tools: Bash, Read, mcp__plugin_ruflo-core_ruflo__memory_search, mcp__plugin_ruflo-core_ruflo__memory_store,

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from ruvnet/ruflo at commit 6051f67, republished under its MIT licence (© ruvnet). 339 words, ~1,102 tokens.

name

gaia-debugging

description

Diagnose why a GAIA question failed — extract trace, classify failure mode, and propose a fix. Use when a GAIA benchmark run reports a failed/incorrect task_id and you need to root-cause it before resubmitting.

allowed-tools

Bash, Read, mcp__plugin_ruflo-core_ruflo__memory_search, mcp__plugin_ruflo-core_ruflo__memory_store, mcp__plugin_ruflo-core_ruflo__agentdb_pattern_search, mcp__plugin_ruflo-core_ruflo__agentdb_pattern_store

argument-hint

<task_id> [--results=<path>]

# Find the result for the task_id in the latest run
RESULTS=~/.cache/ruflo/gaia/results-latest.json
node -e "
  const r = JSON.parse(require('fs').readFileSync('$RESULTS'));
  const q = r.results.find(x => x.task_id === '$TASK_ID');
  console.log(JSON.stringify(q, null, 2));
"

node v3/@claude-flow/cli/bin/cli.js gaia-bench run \
  --level 1 --limit 1 \
  --task-id $TASK_ID \
  --models claude-sonnet-4-6 \
  --max-turns 20 \
  --output json

# Re-run the single question
node … gaia-bench run --task-id $TASK_ID --models $MODEL --output json

# If now passing, store the pattern
npx @claude-flow/cli@latest memory store \
  --namespace gaia-debug-patterns \
  --key "fix-$FAILURE_CODE-$(date +%Y%m%d)" \
  --value "task_id=$TASK_ID, mode=$FAILURE_CODE, fix=$FIX_DESCRIPTION"

node -e "
  const { createDefaultToolCatalogue } = require('./v3/@claude-flow/cli/src/benchmarks/gaia-tools/index.js');
  const cat = createDefaultToolCatalogue({});
  console.log('Tools registered:', cat.definitions.map(t => t.name));
"

npx @claude-flow/cli@latest memory store \
  --namespace gaia-debug-patterns \
  --key "session-$(date +%Y%m%d-%H%M)" \
  --value '{"task_id":"$TASK_ID","failure_mode":"$CODE","fix":"$FIX","verified":true}'

npx @claude-flow/cli@latest memory search \
  --namespace gaia-debug-patterns \
  --query "extraction bug final answer regex"

© ruvnet, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in plugins/ruflo-workflows/skills/gaia-debugging of ruvnet/ruflo.

Open the folder on GitHubat commit 6051f67