New: 98,000+ skills indexed from GitHub repos with 100+ stars · every skill safety-checkedBrowse top skillsTop skills
Esc to closeBrowse all skills
Submit a skillBrowse skills
Submit a skillBrowse skills

Agent skill

Backtest Diagnosis

by HKUDS in HKUDS/Vibe-Trading

Finds why a trading backtest crashed or gave odd results, sorts the cause into a runtime, logic or data error, fixes the code and checks the rerun metrics.

MITAuto-check passedBusiness, Finance & HR

Install Backtest Diagnosis

skills CLI
$ npx skills add HKUDS/Vibe-Trading --skill backtest-diagnose -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install HKUDS/Vibe-Trading backtest-diagnose --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/HKUDS/Vibe-Trading.git skills-src && mkdir -p .claude/skills && cp -r skills-src/agent/src/skills/backtest-diagnose .claude/skills/backtest-diagnose && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
backtest-diagnose
GitHub stars
35k
Token cost
~1.2k tokens
SKILL.md length
663 words
Files
1
Skills in repo
89
Repo updated
First seen
Licence
MIT

At a glance

Finds why a trading backtest crashed or gave odd results, sorts the cause into a runtime, logic or data error, fixes the code and checks the rerun metrics.

  • Works in 5 steps: Read existing artifacts: use read_file… → Read the code: use read_file to inspect… → Classify the issue: determine the root… → …
  • A backtest exits with an error and you need the cause found and fixed
  • SKILL.md covers Overview, Diagnostic Workflow, Error Taxonomy and Hard-Gate Checklist, plus 4 more sections
  • Instructions only: no scripts, shell commands, URLs or credentials in SKILL.md

What it does

When a backtest fails or performs strangely, this skill has the agent read the saved artifacts (metrics.csv, equity.csv and trades.csv) and the strategy code in signal_engine.py and config.json, classify the cause with a built-in taxonomy, edit the code, rerun, and read the new metrics to confirm the fix.

The taxonomy has three parts. Runtime errors such as ImportError, KeyError, IndexError and TypeError come with common causes and fixes, for example the signal must be a pandas Series. Logic bugs cover zero trades, a first trade more than two years after the start, capital utilization below half, and a position left open at the end. Data errors cover missing or too-short data. An ignore list names provider-side messages, such as rate or daily limits, where the code should not be changed because the data source is at fault. The excerpt is cut off after that list.

When your agent uses it

  • A backtest exits with an error and you need the cause found and fixed
  • A backtest finishes with zero trades or mostly idle capital
  • Telling a real strategy bug apart from a data-provider limit or outage

Example prompts

  • “My backtest ran but made zero trades. Diagnose the signal logic and fix it.”
  • “The run failed with a KeyError on a column name, so check the data map and repair it.”
  • “The first trade only happens years after the start date. Find out why.”

Requirements

  • A backtest run folder with artifacts, signal_engine.py and config.json

Workflow steps

5 steps, taken from the first numbered list in SKILL.md.

  1. Read existing artifacts: use read_file to inspect artifacts/metrics.csv, equity.csv, and trades.csv
  2. Read the code: use read_file to inspect code/signal_engine.py and config.json
  3. Classify the issue: determine the root cause using the error taxonomy below
  4. Apply the fix: use edit_file to modify the code, then rerun the backtest
  5. Verify the fix: use read_file to inspect the new metrics.csv

What it can do on your machine

Read from SKILL.md and the folder at commit 14cabaf. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    No scripts in the folder and no shell commands in SKILL.md.

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    No URLs in SKILL.md.

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Backtest Diagnosis loads about 1.2k tokens when it runs. Until then it costs about 26 tokens; SKILL.md has 663 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~26
When it runs · the whole SKILL.md, loaded when a task matches
~1.2k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from HKUDS/Vibe-Trading at commit 14cabaf, republished under its MIT licence (© HKUDS). 663 words, ~1,245 tokens.

Download SKILL.mdSave it as .claude/skills/backtest-diagnose/SKILL.md (or your agent's skills folder).
name
backtest-diagnose
description
Diagnose failed or underperforming backtests, locate the root cause, and fix the issue
category
tool

Backtest Diagnosis

Overview

Use this skill when a user reports that a backtest failed, raised an error, or produced poor results.

Diagnostic Workflow

  1. Read existing artifacts: use read_file to inspect artifacts/metrics.csv, equity.csv, and trades.csv
  2. Read the code: use read_file to inspect code/signal_engine.py and config.json
  3. Classify the issue: determine the root cause using the error taxonomy below
  4. Apply the fix: use edit_file to modify the code, then rerun the backtest
  5. Verify the fix: use read_file to inspect the new metrics.csv

Error Taxonomy

Runtime Errors (exit_code != 0)
Error TypeCommon CauseFix
ImportErrorMissing dependencybash("pip install xxx")
KeyErrorDataFrame column-name mismatchCheck the actual column names in data_map
IndexErrorEmpty data or insufficient lengthAdd length checks
TypeErrorIncorrect signal typeEnsure the return value is pd.Series
Logic Bugs (Backtest Succeeds but Results Are Abnormal)
  1. Zero trades (trade_count=0): signal-logic bug. Conditions are too strict, so the signal stays at 0. Check whether entry and exit logic is reasonable, and inspect the signal series to confirm it is not all zeros.
  2. Late trades (first trade occurs more than 2 years after the backtest start): data-filtering bug. The lookback window may be too long, or the initial data segment may have been dropped. Shorten the window or check whether dropna is too aggressive.
  3. Capital utilization < 50% (mostly in cash): position-sizing bug. Signal triggers may be too sparse, or the position-sizing logic may be wrong.
  4. Open position at the end (a position still exists when the backtest ends): exit-timing bug. Forced liquidation may be missing, or exit logic does not cover the final segment.
Data Errors
SymptomRoot CauseFix
No data fetchedInvalid API token or code issueCheck config.json
Too little dataDate range too narrowExpand the date range
Data-Source Error Ignore List

If you encounter the following keywords, do not modify the code. The problem is on the data-provider side:

  • a provider-side "no data available" response
  • rate limit
  • API limit
  • daily limit
  • Information (common in Tushare API responses)

These issues require the user to check the API token, switch data sources, or wait for the quota to reset.

Hard-Gate Checklist

  1. artifacts/metrics.csv exists and is non-empty
  2. artifacts/equity.csv exists and is non-empty
  3. trade_count > 0 (0 trades means a signal bug)
  4. The equity series contains no NaN
  5. exit_code == 0
Show full SKILL.md (273 more words)Show less

Evidence hookup

This Hard-Gate Checklist is also the evidence ingestion gate for Strategy Discovery: a run failing any gate produces no evidence rows — never partial rows — and is skipped with a stable machine-readable token (hard-gate:exit-nonzero, hard-gate:metrics-missing, hard-gate:zero-trades, hard-gate:equity-empty, hard-gate:equity-nan). Diagnose and fix the failing gate as usual, rerun the backtest, then repopulate the evidence cache with refresh_strategy_evidence (agent tool / MCP tool, or vibe-trading strategy-evidence refresh --manifest <path>) so the fixed run becomes queryable evidence. See the strategy-discovery skill for the manifest format and the full gate list.

Fixing Principles

  • Use edit_file to make precise code fixes instead of rewriting the entire file with write_file, unless the structure is fundamentally broken
  • Fix the bug only, do not change strategy logic unless the user explicitly asks
  • Fix one issue at a time, and rerun the backtest immediately after each fix
  • Limit yourself to at most 3 repair iterations

Post-Fix Validation Rules

After modifying signal_engine.py, you must confirm:

  1. AST syntax passes: bash("python -c \"import ast; ast.parse(open('code/signal_engine.py').read()); print('OK')\"")
  2. Contains class SignalEngine: the file must define class SignalEngine
  3. Contains def generate: the class must contain a def generate method
  4. Rerun the backtest: after the fix, rerun the backtest and verify the results

action_items Writing Rules

After diagnosis, output actionable improvement suggestions:

  • Format: "Change X from A to B" or "Add X logic in signal_engine.py"
  • Be specific about parameter values, filenames, and function names
  • Provide at least 2 items
  • Examples:
    • "Change RSI threshold from 30 to 25 in signal_engine.py line 42"
    • "Add signals = signals.fillna(0) after signal calculation to prevent NaN propagation"
    • "Add a volume filter: skip buy signals when volume is below the 20-day average"

© HKUDS, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

Just SKILL.md in agent/src/skills/backtest-diagnose of HKUDS/Vibe-Trading.

Open the folder on GitHubat commit 14cabaf

Compare with similar skills

Backtest Diagnosis next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Backtest Diagnosis compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Backtest Diagnosis this skillHKUDS/Vibe-Trading35k—~1.2kAutomated safety check: PassMIT
TqSdk Trading and Datashinnytech/tqsdk-python5.1k—~2kAutomated safety check: PassApache-2.0
The Art of Debuggingstas00/the-art-of-debugging1.7k—~6.1kAutomated safety check: NotesCC-BY-SA-4.0
Problem Solving ProHoangTheQuyen/think-better123—~2.8kAutomated safety check: NotesMIT
Bug DetectiveGalaxy-Dawn/claude-scholar5.7k1 repos~2.1kAutomated safety check: PassMIT
Flowfile Debugging PlaybookEdwardvaneechoud/Flowfile373—~6.3kAutomated safety check: PassMIT

Similar skills

  • TqSdk Trading and Data

    shinnytech/tqsdk-python

    Answers TqSdk Python questions on market data, accounts, orders, margin trials, simulation and backtesting, using the library's own docs and examples.

    5.1k GitHub stars~2k tokensUpdated 1 mo ago
    Business, Finance & HRAuto-check passed

  • The Art of Debugging

    stas00/the-art-of-debugging

    Condensed debugging method and tool recipes for Unix, Python and PyTorch programs: crashes, hangs, segfaults, wrong output, CUDA OOM, NaN values and slowness.

    1.7k GitHub stars~6.1k tokensUpdated 2 days ago
    DevelopmentAuto-check: notes

  • Problem Solving Pro

    HoangTheQuyen/think-better

    Systematic problem-solving toolkit: root cause analysis, hypothesis testing, debugging strategies, critical thinking frameworks.

    123 GitHub stars~2.8k tokensUpdated 6 mo ago
    DevelopmentAuto-check: notes

  • Bug Detective

    Galaxy-Dawn/claude-scholar

    Applies a systematic debugging workflow to errors, exceptions and failures, with error-type tables, localization techniques and reference notes for Python, JavaScript and shell.

    5.7k GitHub starsUsed in 1 repo~2.1k tokens
    DevelopmentAuto-check passed

  • Flowfile Debugging Playbook

    Edwardvaneechoud/Flowfile

    Symptom-to-cause triage playbook for Flowfile (core/worker/kernel/frontend/AI) — covers "no such table" DB cascades (two distinct causes), import-time Alembic migration corruption, silent…

    373 GitHub stars~6.3k tokensUpdated yesterday
    DevelopmentAuto-check passed

  • Debug Like Expert

    glittercowboy/taches-cc-resources

    Deep analysis debugging mode for complex issues. An agent skill from glittercowboy/taches-cc-resources.

    2k GitHub stars~2.8k tokensUpdated 6 mo ago
    DevelopmentAuto-check passed

More from HKUDS/Vibe-Trading

All 89 skills in this repo

  • Eastmoney Market Data

    HKUDS/Vibe-Trading

    Index of Eastmoney's free, no-token market data interfaces for China A-shares and Hong Kong stocks: fund flows, dragon-tiger lists, margin trading, reports and news.

    35k GitHub stars~1k tokensUpdated yesterday
    Auto-check passed

  • OKX Market Data

    HKUDS/Vibe-Trading

    Retrieves public OKX cryptocurrency market data such as spot prices, candlesticks, funding rates and open interest through the OKX V5 REST API, with no authentication.

    35k GitHub stars~1.3k tokensUpdated yesterday
    Auto-check passed

  • SEC EDGAR Filings Fetcher

    HKUDS/Vibe-Trading

    Fetches U.S. SEC EDGAR data: resolves tickers to CIK numbers, lists recent 10-K, 10-Q and 8-K filings with document URLs, and pulls XBRL financial series.

    35k GitHub stars~1.4k tokensUpdated yesterday
    Auto-check passed

  • A-Share ST Risk Screener

    HKUDS/Vibe-Trading

    Predicts whether a mainland China A-share company risks an ST or *ST warning after its next annual report, using financial thresholds and Sina penalty records.

    35k GitHub stars~4.9k tokensUpdated yesterday
    Auto-check passed

  • Supply-Chain Bottleneck Hunter

    HKUDS/Vibe-Trading

    Breaks a structural trend such as AI infrastructure into its physical supply chain and ranks lesser-known listed companies sitting on each bottleneck.

    35k GitHub stars~2.7k tokensUpdated yesterday
    Auto-check passed

  • Eight-Part Company Deep Dive

    HKUDS/Vibe-Trading

    Plans and drafts an eight-part, roughly 120k-word investigative series on one company, built around a strict fact-check pass rather than fast drafting.

    35k GitHub stars~2.4k tokensUpdated yesterday
    Auto-check passed

Works with

Categories

Questions about Backtest Diagnosis

What does Backtest Diagnosis do?

Finds why a trading backtest crashed or gave odd results, sorts the cause into a runtime, logic or data error, fixes the code and checks the rerun metrics. json, classify the cause with a built-in taxonomy, edit the code, rerun, and read the new metrics to confirm the fix.

When should I use Backtest Diagnosis?

Backtest Diagnosis fits situations like: A backtest exits with an error and you need the cause found and fixed; A backtest finishes with zero trades or mostly idle capital; telling a real strategy bug apart from a data-provider limit or outage.

How do I install Backtest Diagnosis in Claude Code?

Run `npx skills add HKUDS/Vibe-Trading --skill backtest-diagnose -a claude-code`. Or copy the skill folder (agent/src/skills/backtest-diagnose in HKUDS/Vibe-Trading) into .claude/skills/backtest-diagnose in your project. Claude Code loads it when a task matches its description.

How do I install Backtest Diagnosis in Codex?

Run `npx skills add HKUDS/Vibe-Trading --skill backtest-diagnose -a codex`. Or copy the skill folder (agent/src/skills/backtest-diagnose in HKUDS/Vibe-Trading) into .agents/skills/backtest-diagnose in your project. Codex loads it when a task matches its description.

Can I use Backtest Diagnosis in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add HKUDS/Vibe-Trading --skill backtest-diagnose -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/backtest-diagnose, .gemini/skills/backtest-diagnose, .github/skills/backtest-diagnose and .opencode/skills/backtest-diagnose in your project.

What does Backtest Diagnosis need to run?

SKILL.md names no scripts, command-line tools or credentials: Backtest Diagnosis is instructions for the agent only. Our summary lists: A backtest run folder with artifacts, signal_engine.py and config.json.

Does Backtest Diagnosis access the network?

SKILL.md contains no URLs. Any network use would come from the scripts or tools the agent runs. This is read from the text; nothing was executed.

Is Backtest Diagnosis safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Backtest Diagnosis use?

Backtest Diagnosis is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Backtest Diagnosis use?

About 1.2k tokens (SKILL.md is roughly 5k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Backtest Diagnosis?

Skills that share tags, products or a category with Backtest Diagnosis: TqSdk Trading and Data (shinnytech/tqsdk-python, 5.1k stars), The Art of Debugging (stas00/the-art-of-debugging, 1.7k stars), Problem Solving Pro (HoangTheQuyen/think-better, 123 stars) and Bug Detective (Galaxy-Dawn/claude-scholar, 5.7k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Backtest Diagnosis?

HKUDS (a GitHub organization) maintains it in HKUDS/Vibe-Trading, which has 34,949 GitHub stars. The repository holds 89 skills in this directory. The repository was last updated on October 8, 2026.

Source: HKUDS/Vibe-Trading on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.