Sherlock Review

What it does

When your agent uses it

• Verifying implementation claims
• Investigating bugs
• Validating fixes
• Conducting root cause analysis

Example prompts

• “/sherlock-review”

Workflow steps

5 steps, taken from the first numbered list in SKILL.md.

1. 1OBSERVE: Gather all evidence (code, tests, history, behavior)
2. 2DEDUCE: What does evidence actually show vs. what was claimed?
3. 3ELIMINATE: Rule out what cannot be true
4. 4CONCLUDE: Does evidence support the claim?
5. 5DOCUMENT: Findings with proof, not assumptions

What it can do on your machine

Read from SKILL.md and the folder at commit 1363bc7. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Ships 1 file in scripts/, which the agent can run.

  Shell commands in SKILL.md call:

  • git
  • npm

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md. Its commands use git and npm, which can reach the network depending on how they are called.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Sherlock Review loads about 1.8k tokens when it runs. Until then it costs about 82 tokens; SKILL.md has 404 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~82

When it runs · the whole SKILL.md, loaded when a task matches

~1.8k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.

SKILL.md

The full file from proffesor-for-testing/agentic-qe at commit 1363bc7, republished under its MIT licence (© proffesor-for-testing). 404 words, ~1,772 tokens.

name

sherlock-review

description

Evidence-based investigative code review using deductive reasoning to determine what actually happened versus what was claimed. Use when verifying implementation claims, investigating bugs, validating fixes, or conducting root cause analysis. Elementary approach to finding truth through systematic observation.

category

quality-review

priority

high

tokenEstimate

1100

agents

qe-code-reviewer, qe-security-auditor, qe-performance-validator

implementation_status

optimized

optimization_version

1

last_optimized

2025-12-03

quick_reference_card

true

tags

investigation, evidence-based, code-review, root-cause, deduction

trust_tier

2

# 1. OBSERVE: Gather evidence
git diff <commit>
npm test -- --coverage

# 2. DEDUCE: Compare claim vs reality
# Does code match description?
# Do tests prove the fix/feature?

# 3. CONCLUDE: Verdict with evidence
# SUPPORTED / PARTIALLY SUPPORTED / NOT SUPPORTED

## Sherlock Investigation: [Claim]

### The Claim
"[What PR/commit claims to do]"

### Evidence Examined
- Code changes: [files, lines]
- Tests added: [count, coverage]
- Behavior observed: [what actually happens]

### Deductive Analysis

**Claim**: [specific assertion]
**Evidence**: [what you found]
**Deduction**: [logical conclusion]
**Verdict**: ✓/⚠/✗

### Findings
- What works: [with evidence]
- What doesn't: [with evidence]
- What's missing: [gaps in implementation/testing]

### Recommendations
1. [Action based on findings]

## Case: PR #123 "Fix race condition in async handler"

### Claims Examined:
1. "Eliminates race condition"
2. "Adds mutex locking"
3. "100% thread safe"

### Evidence:
- File: src/handlers/async-handler.js
- Changes: Added `async/await`, removed callbacks
- Tests: 2 new tests for async flow
- Coverage: 85% (was 75%)

### Analysis:

**Claim 1: "Eliminates race condition"**
Evidence: Added `await` to sequential operations. No actual mutex.
Deduction: Race avoided by removing concurrency, not synchronization.
Verdict: ⚠ PARTIALLY TRUE (solved differently than claimed)

**Claim 2: "Adds mutex locking"**
Evidence: No mutex library, no lock variables, no sync primitives.
Verdict: ✗ FALSE

**Claim 3: "100% thread safe"**
Evidence: JavaScript is single-threaded. No worker threads used.
Verdict: ? NONSENSICAL (meaningless in this context)

### Conclusion:
Fix works but not for reasons claimed. Race condition avoided by
making operations sequential, not by adding synchronization.

### Recommendations:
1. Update PR description to accurately reflect solution
2. Add test for concurrent request handling
3. Remove incorrect technical claims

// Evidence-based code review
await Task("Sherlock Review", {
  prNumber: 123,
  claims: [
    "Fixes memory leak",
    "Improves performance 30%"
  ],
  verifyReproduction: true,
  testEdgeCases: true
}, "qe-code-reviewer");

// Bug fix verification
await Task("Verify Fix", {
  bugCommit: 'abc123',
  fixCommit: 'def456',
  reproductionSteps: steps,
  testBoundaryConditions: true
}, "qe-code-reviewer");

aqe/sherlock/
├── investigations/*   - Investigation reports
├── evidence/*         - Collected evidence
├── verdicts/*         - Claim verdicts
└── patterns/*         - Common deception patterns

const investigationFleet = await FleetManager.coordinate({
  strategy: 'evidence-investigation',
  agents: [
    'qe-code-reviewer',        // Code analysis
    'qe-security-auditor',     // Security claim verification
    'qe-performance-validator' // Performance claim verification
  ],
  topology: 'parallel'
});

© proffesor-for-testing, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 2 other files (scripts) in assets/skills/sherlock-review of proffesor-for-testing/agentic-qe.

• SKILL.md
• schemas/output.json
• scripts/validate-config.json

Open the folder on GitHubat commit 1363bc7