Skeptical Triage

What it does

When your agent uses it

• Checking a P0 or P1 security review finding before it blocks a release
• Deciding whether a failing test is a real regression or a flaky test
• Settling an architecture decision dispute where both options look reasonable
• Filtering false positives out of a multi-angle code review

Example prompts

• “Run skeptical triage on the SQL injection finding in the review report.”
• “Is this failing checkout test a regression or a flake? Challenge it in three rounds.”
• “Both options in our ADR look reasonable, so put them through skeptical triage.”

Requirements

• The Read, Grep, Bash and Glob tools
• Pre-approved tools (allowed-tools): Read, Grep, Bash, Glob

Workflow steps

6 steps, taken from the first numbered list in SKILL.md.

1. 1Absence of defense → VALID, not UNCERTAIN. If you searched for a defense and did not find one, that is the answer. "Other code probably…
2. 2A constant name is not a verified bound — only its resolved value is. Grep for the #define / const declaration.
3. 3Name the line or it does not exist. Vague references to "assumptions in this codebase" do not count.
4. 4Do not contradict your own conclusion in the same response. If you verified a defense is insufficient, that is the verdict. Stop searching…
5. 5Code quality issue ≠ security vulnerability. Data race on diagnostic state, NULL check on internal-only API, UB only in debug builds →…
6. 6Trust your own reasoning. If you see the crux on first read, don't manufacture a counter-argument.

What it can do on your machine

Read from SKILL.md and the folder at commit 0e9df12. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves these tools, so the agent can use them without asking each time:

  • Read
  • Grep
  • Bash
  • Glob

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Shell commands in SKILL.md call:

  • jq

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Skeptical Triage loads about 2.1k tokens when it runs. Until then it costs about 52 tokens; SKILL.md has 901 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~52

When it runs · the whole SKILL.md, loaded when a task matches

~2.1k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

• NotePre-approves every shell command (allowed-tools: Bash)SKILL.md

  allowed-tools: Read, Grep, Bash, Glob

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from avelikiy/great_cto at commit 0e9df12, republished under its MIT licence (© avelikiy). 901 words, ~2,101 tokens.

name

skeptical-triage

description

Reusable 3-round self-challenge + arbiter pattern for filtering false positives from findings/verdicts. Use when the cost of a false-positive gate block exceeds the cost of ~4 extra LLM turns.

allowed-tools

Read, Grep, Bash, Glob

when_to_use

Apply skeptical triage when: - A finding could block a gate (gate:code, gate:ship, gate:qa, gate:arch) and flipping it wrongly wastes CTO time - A verdict is…

effort

medium

paths

docs/**, src/**, lib/**, app/**

{
  "verdict": "VALID",
  "crux": "memcpy at auth.c:142 copies network-controlled len bytes into 64-byte stack buffer with no bound check",
  "reasoning": "Rounds 1 and 3 verified attacker reach; Round 2 found no size check in 50 LOC radius; arbiter confirms no caller clamps len."
}

confidence = valid_rounds_before_arbiter / 3

{
  "timestamp": "2026-04-19T12:34:56Z",
  "caller": "review|security-officer|qa-engineer|architect",
  "finding_id": "SEC-042",
  "file": "src/auth.c:142",
  "original_severity": "P0",
  "rounds": [
    {"round": 1, "verdict": "VALID",   "crux": "..."},
    {"round": 2, "verdict": "VALID",   "crux": "...", "grep_used": true},
    {"round": 3, "verdict": "INVALID", "crux": "..."}
  ],
  "arbiter": {"verdict": "VALID", "crux": "..."},
  "confidence": 0.67,
  "final_severity": "P0"
}

# False-positive rate: how many findings the arbiter flipped to INVALID
jq 'select(.arbiter.verdict=="INVALID")' .great_cto/triage-log.jsonl | wc -l

# Average rounds-to-consensus (did we need all 3 or did R1+R2 agree?)
jq '[.rounds[].verdict] | unique | length' .great_cto/triage-log.jsonl

© avelikiy, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in skills/skeptical-triage of avelikiy/great_cto.

Open the folder on GitHubat commit 0e9df12