Theme Detector

Theme Detector

Overview

This skill detects and ranks trending market themes by analyzing cross-sector momentum, volume, and breadth signals. It identifies both bullish (upward momentum) and bearish (downward pressure) themes, assesses lifecycle maturity (Emerging/Accelerating/Trending/Mature/Exhausting), and provides a confidence score combining quantitative data with narrative analysis.

3-Dimensional Scoring Model:

1. Theme Heat (0-100): Direction-neutral strength of the theme (momentum, volume, uptrend ratio, breadth)
2. Lifecycle Maturity: Stage classification (Emerging / Accelerating / Trending / Mature / Exhausting) based on duration, extremity clustering, valuation, and ETF proliferation
3. Confidence (Low / Medium / High): Reliability of the detection, combining quantitative breadth with narrative confirmation. Script output is capped at Medium; Claude's WebSearch narrative confirmation step can elevate to High.
4. Stock Leadership: Optional daily scan-hit evidence from 5D+20%, EP9M, range expansion, new highs, and high-RS stocks. When supplied, this is blended into Theme Heat v2; when absent, it lowers confidence coverage but does not force leadership to zero.
5. Theme Match Quality: Specificity of the theme match from industry participation, explicit stock-basket hits, proxy ETF confirmation, and optional offline narrative scores. This is evidence quality, not a trade recommendation.

Key Features:

• Cross-sector theme detection using FINVIZ industry data
• Direction-aware scoring (bullish and bearish themes)
• Lifecycle maturity assessment to identify crowded vs. emerging trades
• ETF proliferation scoring (more ETFs = more mature/crowded theme)
• Integration with uptrend-dashboard for 3-point evaluation
• Stock-level leadership evidence via --scan-hits
• Theme match quality via explicit stock baskets and proxy ETF confirmation
• Leader candidate evidence ranked by abnormal move/volume/range/RS metrics, with market cap shown only as a risk bucket
• Theme history and acceleration metrics via --history-file
• Dual-mode operation: FINVIZ Elite (fast) or public scraping (slower, limited)
• WebSearch-based narrative confirmation for top themes

When to Use This Skill

Explicit Triggers:

• "What market themes are trending right now?"
• "Which sectors are hot/cold?"
• "Detect current market themes"
• "What are the strongest bullish/bearish narratives?"
• "Is AI/clean energy/defense still a strong theme?"
• "Where is sector rotation heading?"
• "Show me thematic investing opportunities"

Implicit Triggers:

• User wants to understand broad market narrative shifts
• User is looking for thematic ETF or sector allocation ideas
• User asks about crowded trades or late-cycle themes
• User wants to know which themes are emerging vs. exhausted

When NOT to Use:

• Individual stock analysis (use us-stock-analysis instead)
• Specific sector deep-dive with chart reading (use sector-analyst instead)
• Portfolio rebalancing (use portfolio-manager instead)
• Dividend/income investing (use value-dividend-screener instead)

Prerequisites

Required:

• Python 3.9+ with core dependencies.

  pip install -r skills/theme-detector/requirements.txt

Cron / mixed-Python fallback: If the active python3 is older than 3.10, or a newer Hermes venv lacks the data-science dependencies, run the detector through uv with an explicit modern interpreter and temporary dependencies instead of editing the environment mid-cron:

uv run --python 3.12 \
  --with requests --with beautifulsoup4 --with lxml \
  --with pandas --with numpy --with yfinance \
  --with finvizfinance --with PyYAML \
  python skills/theme-detector/scripts/theme_detector.py \
  --finviz-api-key "$FINVIZ_API_KEY" \
  --fmp-api-key "$FMP_API_KEY" \
  --output-dir reports/

Use this as a setup workaround, not as evidence that the detector is broken; still report FINVIZ/FMP/API-data caveats separately.

Optional API Keys:

FINVIZ Elite (recommended for full industry coverage and speed):

export FINVIZ_API_KEY=your_finviz_elite_api_key_here

FMP API (optional, for P/E ratio valuation data):

export FMP_API_KEY=your_fmp_api_key_here

The requirements include finvizfinance, PyYAML, pandas/numpy, requests, and yfinance because normal public-mode execution imports or uses each of them.

Without FINVIZ Elite, the skill uses public FINVIZ scraping (limited to ~20 stocks per industry, slower rate limits).

Workflow

Step 1: Verify Environment

Check that API keys are configured (see Prerequisites):

# Verify FINVIZ Elite API key (optional but recommended)
echo $FINVIZ_API_KEY

# Verify FMP API key (optional)
echo $FMP_API_KEY

Step 2: Execute Theme Detection Script

Run the main detection script:

python3 skills/theme-detector/scripts/theme_detector.py \
  --output-dir reports/

Script Options:

# Full run (public FINVIZ mode, no API key required)
python3 skills/theme-detector/scripts/theme_detector.py \
  --output-dir reports/

# With FINVIZ Elite API key
python3 skills/theme-detector/scripts/theme_detector.py \
  --finviz-api-key $FINVIZ_API_KEY \
  --output-dir reports/

# With FMP API key for enhanced stock data
python3 skills/theme-detector/scripts/theme_detector.py \
  --fmp-api-key $FMP_API_KEY \
  --output-dir reports/

# Custom limits
python3 skills/theme-detector/scripts/theme_detector.py \
  --max-themes 5 \
  --max-stocks-per-theme 10 \
  --output-dir reports/

# Explicit FINVIZ mode
python3 skills/theme-detector/scripts/theme_detector.py \
  --finviz-mode public \
  --output-dir reports/

# Add Stockbee/Pradeep-style leadership evidence
python3 skills/theme-detector/scripts/theme_detector.py \
  --scan-hits data/theme_scan_hits_YYYY-MM-DD.json \
  --narrative-scores data/theme_narrative_scores_YYYY-MM-DD.json \
  --history-file reports/theme_detector_history.json \
  --as-of-date YYYY-MM-DD \
  --output-dir reports/

Scan-hit input contract: --scan-hits accepts JSON, JSONL, or CSV. Rows may be pre-labeled with scan_type / scan_types, or raw rows with fields such as symbol, return_5d, change_pct, volume, avg_volume_50d, relative_volume, true_range, atr_20, atr_expansion, close_location, industry, sector, and theme_guess. A raw row can expand into multiple hits when it satisfies multiple rules.

Initial scan rules:

• five_day_20pct: return_5d >= 20
• ep9m: volume >= 9,000,000, relative_volume >= 2.0, and change_pct >= 4
• range_expansion: change_pct >= 4, true_range / atr_20 >= 1.5 or atr_expansion >= 1.5, and close_location >= 0.75
• new_high: explicit new_high / is_new_high, or 52-week high evidence
• high_rs: rs_rating >= 90 or normalized relative_strength >= 0.90

Narrative-score input contract: --narrative-scores is an offline JSON input, not a live WebSearch call. It accepts either {"Theme Name": 82} or {"themes": {"Theme Name": {"narrative_keyword_score": 82}}}. Missing narrative input leaves narrative_keyword_score as null and reduces theme_match_coverage; it does not fail the run.

Expected Execution Time:

• FINVIZ Elite mode: ~2-3 minutes (14+ themes)
• Public FINVIZ mode: ~5-8 minutes (rate-limited scraping)

Step 3: Read and Parse Detection Results

The script generates two output files:

• theme_detector_YYYY-MM-DD_HHMMSS.json - Structured data for programmatic use
• theme_detector_YYYY-MM-DD_HHMMSS.md - Human-readable report

Read the JSON output to understand quantitative results:

# Find the latest report
ls -lt reports/theme_detector_*.json | head -1

# Read the JSON output
cat reports/theme_detector_YYYY-MM-DD_HHMMSS.json

Step 4: Perform Narrative Confirmation via WebSearch

For the top 5 themes (by Theme Heat score), execute WebSearch queries to confirm narrative strength:

Search Pattern:

"[theme name] stocks market [current month] [current year]"
"[theme name] sector momentum [current month] [current year]"

Evaluate narrative signals:

• Strong narrative: Multiple major outlets covering the theme, analyst upgrades, policy catalysts
• Moderate narrative: Some coverage, mixed sentiment, no clear catalyst
• Weak narrative: Little coverage, or predominantly contrarian/skeptical tone

Update Confidence levels based on findings:

• Quantitative High + Narrative Strong = High confidence
• Quantitative High + Narrative Weak = Medium confidence (possible momentum divergence)
• Quantitative Low + Narrative Strong = Medium confidence (narrative may lead price)
• Quantitative Low + Narrative Weak = Low confidence

Step 5: Analyze Results and Provide Recommendations

Cross-reference detection results with knowledge bases:

Reference Documents to Consult:

1. references/cross_sector_themes.md - Theme definitions and constituent industries
2. references/thematic_etf_catalog.md - ETF exposure options by theme
3. references/theme_detection_methodology.md - Scoring model details
4. references/finviz_industry_codes.md - Industry classification reference

Analysis Framework:

For Hot Bullish Themes (Heat >= 70, Direction = Bullish):

• Identify lifecycle stage (Emerging = opportunity, Mature/Exhausting = caution)
• List top-performing industries within the theme
• Recommend proxy ETFs for exposure
• Flag if ETF proliferation is high (crowded trade warning)

For Hot Bearish Themes (Heat >= 70, Direction = Bearish):

• Identify industries under pressure
• Assess if bearish momentum is accelerating or decelerating
• Recommend hedging strategies or sectors to avoid
• Note potential mean-reversion opportunities if lifecycle is Mature/Exhausting

For Emerging Themes (Heat 40-69, Lifecycle = Emerging):

• These may represent early rotation signals
• Recommend monitoring with watchlist
• Identify catalyst events that could accelerate the theme

For Exhausted Themes (Heat >= 60, Lifecycle = Exhausting):

• Warn about crowded trade risk
• High ETF count confirms excessive retail participation
• Consider contrarian positioning or reducing exposure

Step 6: Generate Final Report

Present the final report to the user using the report template structure:

# Theme Detection Report
**Date:** YYYY-MM-DD
**Mode:** FINVIZ Elite / Public
**Themes Analyzed:** N
**Data Quality:** [note any limitations]

## Theme Dashboard
[Top themes table with Heat, Direction, Lifecycle, Confidence]

## What Changed Today
[Newly emerging themes, largest heat acceleration, new EP9M clusters, fading themes]

## Leadership Evidence
[5D+20%, EP9M, range expansion, new highs, high-RS counts and leader symbols]

## Bullish Themes Detail
[Detailed analysis of bullish themes sorted by Heat]

## Bearish Themes Detail
[Detailed analysis of bearish themes sorted by Heat]

## All Themes Summary
[Complete theme ranking table]

## Industry Rankings
[Top performing and worst performing industries]

## Sector Uptrend Ratios
[Sector-level aggregation if uptrend data available]

## Methodology Notes
[Brief explanation of scoring model]

Save the report to reports/ directory.