Prd V08 Drift Baseline Compare

Drift: Baseline / Compare / History

Position in workflow: v0.8 Monitoring Setup → v0.8 Drift: Baseline / Compare / History → v0.8 Runbook Creation

Execution Mode

Default is standard. See .claude/rules/08-skill-execution-modes.md for selection logic.

What This Does

Generalizes a pattern AgriciDaniel's claude-seo skill encodes for SEO drift — baseline → snapshot → compare → history — into a reusable monitoring shape that works for any metric, config, or dataset that can change over time and needs to be watched.

This is drift monitoring, distinct from alerting on absolute thresholds. Alerting answers "is X over the line right now?" Drift monitoring answers "is X different from last week's normal?" — which catches slow regressions that absolute thresholds miss.

Examples of things worth drift-monitoring:

• KPI: activation rate week-over-week
• AI search position: ChatGPT/Perplexity ranking for target queries
• Config: feature-flag rollout percentages
• Performance: p95 latency by endpoint
• Cost: per-user infra cost
• Marketing: per-channel CAC trend
• Content: changelog post engagement
• Third-party: vendor pricing pages (price hikes), competitor feature pages (parity loss)

How It Works

1. Pick what to monitor — One thing per MON-DRIFT- entry. Must be:
   • Quantifiable (number, percentage, list, configuration value)
   • Snapshotable (captureable at a point in time, ideally automatically)
   • Causally interpretable (when it changes, you know enough to investigate)
2. Capture the baseline — Take a snapshot. Date it. Store in version control or a known location (status/baselines/, monitoring/snapshots/, etc.).
3. Define drift thresholds:
   • Warn: meaningful change (e.g., 10% drift in a KPI; any change in a config value)
   • Critical: serious change (e.g., 25% KPI drop; breaking config change)
   • Recalibrate: intentional change that should refresh the baseline (e.g., after a feature rollout, the baseline is wrong; refresh it)
4. Set compare cadence — How often does this get re-snapshotted?
   • Hot (hourly/daily): production KPIs, AI search positions during a launch
   • Warm (weekly): standard product KPIs, content engagement
   • Cool (monthly): vendor pricing, competitor feature parity, infra cost
5. Build the compare procedure — A script or runbook that:
   • Takes a new snapshot
   • Diffs against baseline
   • Computes drift % per dimension
   • Emits warn/critical signals at thresholds
   • Appends to history log
6. Plan auto-baselining after intentional change [standard+] — When the team makes a deliberate change (ships a feature that should improve activation), the old baseline becomes wrong. Define what triggers a baseline refresh and who approves it.

Example

Monitoring AI search position for the target query "best CRO tool for SaaS founders" across 3 surfaces. (Generalized from AEO Audit re-test cadence.)

Baseline (2026-04-01):

ChatGPT: not mentioned (rank: --)
Perplexity: rank 4 of 5, miscategorized as "analytics"
AI Overviews: not mentioned
Sources cited: [competitor1.com, competitor2.com, reddit.com/r/SaaS]

Thresholds:

• Warn: rank changes by ≥1, sources list changes, miscategorization persists
• Critical: dropped from any surface where previously mentioned

Compare cadence: weekly during launch month, monthly after.

Re-snapshot (2026-04-15):

ChatGPT: rank 3 of 5 (NEW)         ← improved
Perplexity: rank 3 of 5, category corrected (NEW)   ← improved
AI Overviews: rank 5 of 7 (NEW)    ← improved
Sources cited: [+ ourdomain.com (NEW), ...]

Compare output: 3 surfaces improved. Likely cause: alternatives-pages campaign + new CRO-anchored blog post indexed.

History append: 2026-04-01 → 2026-04-15 row added to history log.

Action: No alert; positive drift. Baseline can stay (next compare in 2 weeks). If the trend reverses, the baseline catches it.

What You Get Back

• MON-DRIFT-* entries (one per monitored thing) — Baseline snapshot, thresholds, cadence, compare procedure, history pointer
• History logs in status/drift-history/<id>.jsonl (or similar) — Append-only record of compare results
• Compare scripts (scripts/drift/<id>.sh or equivalent) when automatable

When to Use It

Do not use for things that are already covered by threshold-based alerting (RED/USE metrics, MON-* alerts). Drift is for trends; alerting is for absolutes.