Mitre Coverage Report

MITRE ATT&CK Coverage Report — Instructions

Purpose

This skill generates a comprehensive MITRE ATT&CK Coverage Report analyzing detection coverage across the ATT&CK Enterprise framework. It inventories all analytic rules and custom detections, maps them to MITRE tactics and techniques, identifies coverage gaps, and provides prioritized recommendations for improving detection posture.

Entity Type: Sentinel workspace (from config.json)

What this report covers: Tactic-level coverage matrix with per-tactic technique counts and percentages, technique-level drill-down with rule-to-technique mapping, coverage gap identification against the full ATT&CK Enterprise framework, SOC Optimization threat scenario alignment (AiTM, ransomware, BEC, etc.), untagged rule remediation with AI-suggested MITRE tags, ICS/OT technique tracking, operational MITRE correlation (which rules actually fire), and a composite MITRE Coverage Score.

Complementary to: This skill pairs with the sentinel-ingestion-report skill — ingestion report covers data volume, tier optimization, and cost; MITRE coverage report covers detection posture against the ATT&CK framework. Run both for a complete workspace assessment.

Architecture

 ┌──────────────────────────────────────────────────────────────────┐
 │  YAML query files        PowerShell script         LLM render   │
 │  queries/phase1-3/  ──→  Invoke-MitreScan.ps1  ──→  Phase 4    │
 │  (6 .yaml files)         (~1030 lines)             (SKILL-      │
 │                          • az rest (Sentinel API)   report.md)  │
 │                          • Invoke-MgGraphRequest                │
 │                          • az monitor (KQL)                     │
 │                          • mitre-attck-enterprise.json          │
 │                          • m365-platform-coverage.json (CTID)   │
 │                          ↓                                      │
 │                     temp/mitre_scratch_<ts>.md                  │
 │                     (~35 KB, 18+ sections)                     │
 └──────────────────────────────────────────────────────────────────┘

Execution model:

• Phases 1-3 (data gathering): Fully automated by Invoke-MitreScan.ps1. Phase 1 uses az rest (Sentinel REST API) and optionally Invoke-MgGraphRequest (Graph API). Phase 2 uses az rest (SOC Optimization API). Phase 3 uses az monitor log-analytics query (KQL).
• Phase 4 (rendering): LLM reads the scratchpad + SKILL-report.md and renders the report. This is the only phase requiring LLM involvement.

Static reference: mitre-attck-enterprise.json contains ATT&CK Enterprise v16.1 with 14 tactics, 216 techniques, and 475 sub-techniques. The PS1 loads this at startup to compute coverage gaps against the full framework. This file is version-controlled and should be updated when MITRE publishes new ATT&CK releases.

Platform coverage reference: m365-platform-coverage.json is a compact CTID (Center for Threat-Informed Defense) mapping of M365 Defender product capabilities to ATT&CK techniques. Contains detect/protect/respond coverage for 81 detect techniques across 38 capabilities (7 SecurityAlert product groups). Used for the 3-tier platform coverage classification:

• Tier 1 (Alert-Proven): SecurityAlert from M6 query has MITRE technique attribution — highest confidence
• Tier 2 (Deployed Capability): Product is active (has alerts) and CTID claims detect coverage for the technique — medium confidence
• Tier 3 (Catalog Capability): CTID maps coverage but no alert evidence for the product in this workspace — lowest confidence

To rebuild from upstream: download the CTID M365 mapping JSON, transform with PowerShell (group by parent technique, map capabilities to SecurityAlert ProductName). See temp/ctid_raw.json for the raw source.

Companion Files — When to Load

📑 TABLE OF CONTENTS

1. Quick Start - 3-step execution pattern
2. Critical Workflow Rules - Prerequisites and prohibitions
3. Execution Workflow - Phases 0-4
4. Query File Reference - All 5 YAML files
5. Output Modes - Inline chat vs. Markdown file
6. Deterministic Rendering Rules - Rules A-D (mandatory for Phase 4)
7. MITRE Coverage Score - 5-dimension scoring methodology
8. Domain Reference - ATT&CK interpretation, tactic priorities, Sentinel-specific mappings
9. SVG Dashboard Generation - Visual dashboard from completed report

Quick Start (TL;DR)

3-step execution pattern:

Step 1:  Run Invoke-MitreScan.ps1 (Phases 1-3 — data gathering)
Step 2:  Read scratchpad + SKILL-report.md (Phase 4 prep)
Step 3:  Render report incrementally (§1 via create_file, then §2–§6 appended via replace_string_in_file)

Step 1: Run Data Gathering

# From workspace root — run all phases (default: 30 days alert/incident lookback):
& ".github/skills/mitre-coverage-report/Invoke-MitreScan.ps1"

# Specify a custom alert/incident lookback:
& ".github/skills/mitre-coverage-report/Invoke-MitreScan.ps1" -Days 7

# Run a specific phase (for re-runs / debugging):
& ".github/skills/mitre-coverage-report/Invoke-MitreScan.ps1" -Phase 1

Output: Scratchpad file at temp/mitre_scratch_<timestamp>.md (~28 KB, 12 sections).

Timing: Full run takes ~60-90 seconds (varying with REST API response times and KQL auth state).

Step 2: Load Rendering Context

1. Read the scratchpad file (path printed by PS1 at completion)
2. Read SKILL-report.md for rendering templates

Step 3: Render Report (Incremental Writes)

Render the report across multiple tool calls — one section per call — to avoid single-call output token limits that truncate large reports:

1. create_file → header + disclaimer + §1 (Executive Summary, Score, Inventory, Top 3 Recs)
2. replace_string_in_file → append §2 (Tactic Coverage Matrix)
3. replace_string_in_file → append §3 (Technique Deep Dive — largest section)
4. replace_string_in_file → append §4 (Coverage Gap Analysis)
5. replace_string_in_file → append §5 (Operational MITRE Correlation)
6. replace_string_in_file → append §6 + Appendix

Apply SKILL-report.md templates to scratchpad data, following Rules A–D. See SKILL-report.md for full section templates and the anchor pattern for each append.

🔴 Verbatim table sections — use the deterministic slicer, never hand-copy. Several report tables (§3 TechniqueTables, §5.1 CombinedTacticCoverage, §5.2 AlertFiring, §5.3 ActiveVsTagged, §5.4 IncidentsByTactic, §5.5 DataReadiness, §5.6 ConnectorHealth) are pre-rendered by the PS1 under ## PRERENDERED in the scratchpad. Copy them with the read-only helper instead of transcribing by hand:

python .github/skills/mitre-coverage-report/slice_scratch.py --scratch temp/mitre_scratch_<ts>.md --list
python .github/skills/mitre-coverage-report/slice_scratch.py --scratch temp/mitre_scratch_<ts>.md --section AlertFiring

The slicer prefers the ## PRERENDERED copy when a section name also exists as a raw data block, strips pipeline scaffolding (<!-- … --> comments, SectionTitle: markers) wherever it appears, preserves #### sub-headings, and collapses blank runs — so the output drops straight into the report as a valid markdown table. Do NOT paste the raw Key | Value | … data blocks (the ones with a <!-- header --> comment and no |---| separator row) — they render as plain text, not tables, and pasting the whole scratchpad tail into one section corrupts the report.

⛔ Do NOT render §1–§6 in a single create_file call. The output will truncate silently. The scratchpad is ~60 KB; the rendered report exceeds the single-call output budget.

🔴 ALL 6 APPENDS ARE MANDATORY. Do NOT stop after §5 — §6 (Recommendations) and the Appendix (Score Methodology, Limitations) are critical and must be appended. After the 6th append, run grep_search for ## 6. Recommendations and ## Appendix on the report file to verify both exist. If either is missing, append the missing content immediately.

⚠️ CRITICAL WORKFLOW RULES - READ FIRST ⚠️

Before starting ANY MITRE coverage report:

1. Run Invoke-MitreScan.ps1 — this single script handles ALL data gathering (Phases 1-3). The LLM does NOT run queries, transcribe output, or write scratchpad sections
2. Read config.json for workspace ID, tenant, subscription, and Azure MCP parameters
3. ALWAYS ask the user for output mode if not specified: inline chat summary, markdown file report, or both (default: both)
4. ALWAYS ask the user for timeframe if not specified: the -Days parameter controls the alert/incident KQL lookback (Phase 3). Default: 30 days. Phases 1-2 (REST API) are not time-bounded
5. ALWAYS use create_file for markdown reports (never use terminal commands)
6. ALWAYS sanitize PII from saved reports — use generic placeholders for real rule names, workspace names, and tenant GUIDs in committed files
7. Read scratchpad + SKILL-report.md before rendering — the scratchpad is the sole data source
8. Custom Detections may be SKIPPED — the Graph API requires CustomDetection.Read.All which needs admin consent. If skipped, the report notes this and shows AR-only analysis. Do NOT treat SKIPPED as an error — it's a graceful degradation

Prerequisites

🔴 PROHIBITED

• ❌ Running REST/KQL queries via MCP tools during data gathering — PS1 handles all queries
• ❌ Writing or modifying scratchpad sections manually — PS1 is the sole writer
• ❌ Fabricating technique counts, rule names, or coverage percentages
• ❌ Inventing ATT&CK technique IDs or names not in the reference JSON
• ❌ Overriding MITRE Coverage Score dimensions — the PS1 computes these deterministically
• ❌ Rendering the report without first reading the scratchpad file
• ❌ Reporting "100% coverage" for any tactic unless the data actually shows every technique covered

Execution Workflow

Phase 0: Initialization

1. Read config.json for sentinel_workspace_id, subscription_id, Azure MCP parameters
2. Confirm output mode and timeframe with user (pass -Days to PS1; default 30)
3. Verify prerequisites: az login session active, correct subscription set

Phases 1-3: Data Gathering (automated by PS1)

Run Invoke-MitreScan.ps1 — it handles all 3 phases automatically:

Post-processing (automated by PS1):

Scratchpad output: PS1 writes all results to temp/mitre_scratch_<timestamp>.md (~28 KB, ~12 named sections). See SKILL-report.md for the Section-to-Scratchpad Mapping.

Phase 4: Render Output (LLM)

🔴 MANDATORY — Load scratchpad + report template before rendering:

1. Read the scratchpad file (path printed by PS1). This single file contains ALL data from Phases 1-3.
2. Read SKILL-report.md for the complete rendering templates and formatting rules.

Pre-render validation:

1. Verify scratchpad has all 3 phase sections (PHASE_1 through PHASE_3)
2. Check SCORE section has all 5 dimensions
3. If Phase 3 shows FAILED for M4/M5 (token expiry), note this in the report — the Operational dimension defaults to 0

Render — Section-by-Section:

Query File Reference

All queries are defined as YAML files in queries/phase1-3/.

YAML Format

id: mitre-m1                                   # Unique identifier
name: Analytic Rule MITRE Extraction            # Human-readable name
description: Fetch rules with tactics/techniques # What it does
phase: 1                                        # Which phase (1-3)
type: rest                                      # rest | graph | kql
url: https://management.azure.com/...           # REST API URL with placeholders
jmespath: value[].{...}                         # JMESPath projection (REST)

Complete Query Inventory

Output Modes

Mode 1: Inline Chat Summary (default for quick requests)

Compact executive summary rendered directly in chat with MITRE Coverage Score and top coverage gaps.

Mode 2: Markdown File Report

Full detailed report saved to reports/sentinel/mitre_coverage_report_<YYYYMMDD_HHMMSS>.md.

Mode 3: Both (default when user says "report" or "generate report")

Inline chat executive summary + full markdown file.

Ask user if not specified:

"How would you like the MITRE coverage report? I can provide:

1. Inline chat summary — MITRE Score + top gaps in chat
2. Markdown file — detailed report saved to reports/sentinel/
3. Both (recommended) — summary in chat + full report file"

Deterministic Rendering Rules

These rules eliminate LLM interpretation variance. Apply them EXACTLY during Phase 4 rendering.

Rule A: Coverage Level Classification

Assign emoji badges to each tactic row in the coverage matrix based on the percentage of techniques covered:

⛔ PROHIBITED: Assigning badges based on "importance" or "this tactic is more relevant." The badge MUST match the percentage threshold table above.

Rule B: Threat Scenario Priority

When rendering SOC Optimization threat scenarios, order by coverage gap (recommended minus active) descending, but assign badges based on completion rate (proportional to scenario size):

Why rate-based? Recommendation counts reflect the full Content Hub template catalogue including templates for vendor products not deployed in the environment (e.g., all firewall vendors). A 609-rule scenario will be permanently 🔴 under absolute-gap thresholds even at 80% coverage. Rate-based badges give proportional, meaningful progress signals.

CompletedBySystem note: CompletedBySystem is a SOC Optimization state, not a rate indicator. Some CompletedBySystem entries have low rates (recommended >> active). Always use the completion rate for badge assignment. The State column is displayed for context but does NOT override the rate-based badge.

Rule C: "Paper Tiger" Detection

When Phase 3 data is available, identify paper tiger rules — rules with MITRE tags that have NEVER produced an alert in the lookback period. These rules are tagged but non-operational, and their coverage is theoretical, not proven.

⛔ PROHIBITED: Reporting coverage percentages as "validated" when Phase 3 data is missing. If M4/M5 failed, state: "Coverage percentages reflect rule tagging only — operational validation unavailable (Phase 3 KQL queries failed)."

Rule D: Recommendation Ranking

Rank recommendations by impact using this priority order: