Monte Carlo Remediation

Monte Carlo Remediation Skill

This skill teaches you to investigate and remediate data quality issues detected by Monte Carlo. You use MC MCP tools to understand the alert context, run root cause analysis, assess blast radius, and then execute the appropriate remediation action using whatever external tools the user has connected.

Monte Carlo tool routing (required): Always call Monte Carlo MCP tools through this plugin's bundled server, whose fully-qualified tool names are mcp__plugin_mc-agent-toolkit_monte-carlo-mcp__<tool> (e.g. mcp__plugin_mc-agent-toolkit_monte-carlo-mcp__get_alerts). Bare tool names used in this skill (get_alerts, search, get_table, …) refer to that bundled server. If the session also has a separately-configured monte-carlo-mcp server, do not route to it — it may point at a different endpoint or credentials.

Reference files live next to this skill file. Use the Read tool (not MCP resources) to access them:

• Common remediation patterns and examples: references/patterns.md (relative to this file)
• How to discover available tools at runtime: references/tool-discovery.md (relative to this file)
• Safety rails and escalation criteria: references/safety.md (relative to this file)

When to activate this skill

Activate when the user:

• Asks to remediate, fix, or respond to a data quality alert or incident
• Mentions a specific alert ID, incident, or data quality issue they want resolved
• Says something like "fix the freshness issue on X", "remediate this alert", "handle this incident"
• Asks to triage AND fix an alert (triage alone without remediation intent → use the prevent skill's Workflow 3 instead)
• Wants to automate a response to a recurring data quality pattern
• Asks "what should I do about this alert?" or "how do I fix this?"

When NOT to activate this skill

Do not activate when the user is:

• Just triaging or investigating an alert without remediation intent (use prevent skill's Workflow 3)
• Creating or configuring monitors (use the monitoring-advisor skill)
• Running a change impact assessment before code changes (use the prevent skill's Workflow 4)
• Asking about general data quality best practices without a specific incident
• Exploring table health or lineage without an active issue to fix

Available tools

Monte Carlo MCP server (investigation + post-remediation)

The Monte Carlo MCP server (monte-carlo-mcp) provides the investigation tools used in the workflows below. The workflows reference key tools by name (e.g., get_alerts, run_troubleshooting_agent, get_asset_lineage), but use any Monte Carlo tool that helps — the server has additional tools beyond what the workflows explicitly call out. Explore what's available.

Note on tool call examples: The code blocks below show key parameters to guide you. Always check the tool's own description for the complete parameter list and exact parameter names — they are authoritative.

External tools (remediation execution)

Remediation actions are executed via whatever tools are available — MCP servers, CLI tools, or APIs. See Workflow 2 (Capability Discovery) and references/tool-discovery.md for how to detect and use them. Use whatever works; don't limit yourself to a prescribed list.

Core workflow

Follow these workflows in order. Each workflow builds on the context gathered by the previous one.

Workflow 1: Investigation

Goal: Understand what happened, why it happened, and what's affected.

Before proposing ANY remediation action, you MUST complete this investigation. Do not skip steps — incomplete context leads to wrong fixes.

Step 1: Get alert context

get_alerts(
  alert_ids=["<alert_id>"],
)

If the user provided a table name instead of an alert ID:

search(query="<table_name>")
→ extract MCON
get_alerts(
  table_mcons=["<mcon>"],
  created_after="<7 days ago>",
  created_before="<now>",
  order_by="-createdTime",
  statuses=["NOT_ACKNOWLEDGED", "WORK_IN_PROGRESS"]
)

Extract from the alert: alert_type (Freshness, Volume, Schema Changes, etc.), severity, affected table MCONs, created_time.

Step 2: Assess triage priority

alert_assessment(
  incident_id="<alert_uuid>"
)

This returns incident_likelihood (HIGH/MEDIUM/LOW), alert_impact (HIGH/MEDIUM/LOW), and a summary. Use this to decide urgency:

• HIGH impact + HIGH incident likelihood → proceed immediately to Troubleshooting Agent (TSA) analysis
• LOW impact or LOW incident likelihood → still run TSA, but note to the user that this may not warrant immediate remediation

Step 3: Root cause analysis (TSA)

Always use async mode. TSA analysis takes 4–8 minutes — sync mode will time out.

run_troubleshooting_agent(
  incident_id="<alert_uuid>",
  async_mode=true
)

While TSA runs, proceed with Steps 4–6 in parallel — gather lineage, table context, and query data while waiting. Then poll for TSA results:

get_troubleshooting_agent_results(
  incident_id="<alert_uuid>"
)

Status values:

• not_found → TSA hasn't been triggered yet
• running → still analyzing (wait 30s initially, then 60s intervals)
• success → results available
• failed → check full_response for error; proceed with manual investigation

When TSA succeeds, read both the tldr and the verifications section. The tldr summarizes the root cause — this is your primary input for choosing a remediation action. The full_response includes a "verifications to confirm the root cause" section with specific checks (queries to run, things to compare, upstream systems to inspect). These verifications are often actionable remediation steps themselves — use them to guide what to do next or present them to the user as concrete next steps.

Step 4: Assess blast radius

get_asset_lineage(
  mcons=["<affected_table_mcon>"],
  direction="DOWNSTREAM"
)

For BI report coverage:

get_downstream_bi_reports(
  mcon="<affected_table_mcon>"
)

Then for upstream investigation:

get_asset_lineage(
  mcons=["<affected_table_mcon>"],
  direction="UPSTREAM"
)

Note: has_relationships=false means no dependencies tracked — do not assume missing relationships.

Step 5: Gather table context

get_table(
  mcon="<affected_table_mcon>",
  include_fields=true,
  include_table_capabilities=true
)

Extract: last activity timestamps, row counts, schema, monitoring status, importance score.

For key downstream tables identified in Step 4, also fetch their details:

get_table(mcon="<downstream_mcon>")

Step 6: Check alert context, monitoring, and recent queries

get_monitors(mcons=["<affected_table_mcon>"])

For Custom SQL or Validation alerts, also fetch the monitor configuration to understand the exact rule that breached:

get_monitors(
  monitor_ids=["<monitor_id_from_alert>"],
  include_fields=["config"]
)

The config contains the SQL query or validation conditions — this tells you exactly what the monitor checks, which is essential for understanding what went wrong and what the fix should be.

get_queries_for_table(
  mcon="<affected_table_mcon>",
  query_type="destination",
  limit=10
)

Use query_type="destination" to find queries that write to this table (pipeline queries). This helps identify which pipeline or job is responsible for the data.

Investigation summary

Wait for TSA to complete before presenting findings. Do not present partial results — the TSA root cause analysis and its verifications section are critical for choosing the right remediation action. If TSA is still running, keep polling; gather Steps 4–6 in the meantime.

After all steps are complete, synthesize your findings into a clear summary:

1. What happened: alert type, when it fired, severity
2. Root cause: TSA findings (or your best assessment if TSA failed)
3. TSA verifications: specific checks from the TSA full_response that can confirm the root cause or serve as remediation steps
4. Blast radius: N downstream consumers, any key assets affected
5. Pipeline context: which queries/jobs write to this table, when they last ran
6. Monitoring: what monitors exist, any gaps. Note recurring patterns (e.g., "16 incidents in 30 days" signals a chronic issue, not a one-off)

Present this summary to the user before proceeding to remediation.

Workflow 2: Capability discovery

Goal: Determine what remediation actions are possible given the tools you have available.

Before attempting any remediation action, you must know what tools you can use. You have three categories to check:

1. MCP servers — scan your tool list for mcp__*__* patterns (e.g., mcp__airflow__trigger_dag_run)
2. CLI tools — you have shell access; check for tools like gh, dbt, airflow, curl via which <tool>
3. APIs — any service with a REST API is reachable via curl if you have the right credentials

Don't assume any particular tool is available. But also don't assume MCP is the only option — a gh pr create via the CLI works just as well as a GitHub MCP tool.

For detailed guidance on discovery across all three categories, read references/tool-discovery.md.