Creative Testing Framework

/digital-marketing-pro:creative-testing-framework

Purpose

Design a systematic creative testing framework that maximizes learning velocity while maintaining statistical rigor across advertising platforms. Produces a complete testing playbook with variable prioritization, sample size requirements, iteration cadence, and documentation standards for continuous creative optimization.

Input Required

The user must provide (or will be prompted for):

• Ad platform(s): Where ads are running — Google Ads, Meta Ads, LinkedIn Ads, TikTok Ads, programmatic DSPs, Pinterest, X/Twitter, or multi-platform
• Creative types available: What formats can be produced — static image, video (short-form/long-form), carousel, text-only, responsive display, HTML5, playable, or collection ads
• Monthly ad budget allocated to testing: How much budget is available specifically for creative experimentation vs. proven performers
• Current top-performing creative: Description or reference to the best-performing ads currently running, including their key metrics
• Learning goals: Which creative elements need optimization — headlines, imagery, CTA copy, video hooks, color palette, offer framing, social proof, format type, or ad copy length
• Audience segments for testing: The audience groups available for testing — prospecting, retargeting, lookalike, interest-based, demographic, or custom segments
• Campaign objectives: What the ads are optimized for — awareness (impressions/reach), consideration (clicks/video views), or conversion (leads/purchases/ROAS)
• Historical creative performance data: Optional — past test results, creative fatigue patterns, seasonal performance variations, and known winners/losers
• Brand guidelines constraints: Visual identity rules, messaging restrictions, mandatory disclaimers, or approval bottlenecks that affect creative production speed
• Testing timeline: How long the testing program should run — single sprint (2-4 weeks), quarterly roadmap, or ongoing evergreen program

Process

1. Load brand context: Read ~/.claude-marketing/brands/_active-brand.json for the active slug, then load ~/.claude-marketing/brands/{slug}/profile.json. Apply brand voice, compliance rules for target markets (skills/context-engine/compliance-rules.md), and industry context. Also check for guidelines at ~/.claude-marketing/brands/{slug}/guidelines/_manifest.json — if present, load restrictions and relevant category files. Check for custom templates at ~/.claude-marketing/brands/{slug}/templates/. Check for agency SOPs at ~/.claude-marketing/sops/. If no brand exists, ask: "Set up a brand first (/digital-marketing-pro:brand-setup)?" — or proceed with defaults.
2. Define testing variables: Catalog all testable creative elements — headline copy, body copy length, CTA text and color, hero image subject, image style (photo vs. illustration vs. UGC), video hook (first 3 seconds), video length, ad format (static vs. carousel vs. video), color palette, offer framing (discount vs. value vs. urgency), social proof type (testimonial vs. stat vs. badge), and layout composition.
3. Prioritize variables by expected impact and ease: Score each variable on a 2x2 matrix of expected performance impact (high/low) and production effort (high/low). Rank variables so the team tests high-impact, low-effort elements first. Use historical data and platform benchmarks to inform impact estimates where available.
4. Design testing matrix: Build the variable-by-variant grid — for each priority variable, define 2-4 variants to test against the current control. Ensure tests are isolated (one variable per test) unless running deliberate multivariate experiments. Map each test to the appropriate audience segment and platform.
5. Calculate sample size per variant and minimum budget: Compute the required conversions per variant with python "${CLAUDE_PLUGIN_ROOT}/scripts/sample-size-calculator.py" --baseline-rate {rate} --mde {mde} --mde-type relative --significance 0.95 --power 0.80 (a "10-20% relative lift" is --mde 0.10-0.20 with --mde-type relative; use --mde-type absolute if the target is stated in percentage points — the two differ by ~40× at a 5% baseline). Translate sample size into minimum budget per test based on current CPM/CPC rates.
6. Define holdout control structure: Design the control framework — allocate 10-20% of testing budget to an unchanging control creative that serves as a stable benchmark. Define when the control should be refreshed (quarterly or when performance degrades below threshold) and how new winners graduate to become the new control.
7. Set statistical significance thresholds: Define the confidence level required to declare a winner (90% for directional decisions, 95% for major creative shifts). Specify whether to use frequentist (p-value) or Bayesian (probability to be best) methodology. Document the minimum observation period (7+ days to account for day-of-week variation) and anti-peeking protocols.
8. Create iteration cadence: Design the testing rhythm — weekly creative refreshes for high-volume accounts, bi-weekly for mid-volume, monthly for lower-volume. Define the pipeline: brief (day 1), production (days 2-3), review and approval (day 4), launch (day 5), monitor (days 6-14), analyze and iterate (day 15). Align cadence with brand approval workflows.
9. Build winner selection criteria: Define how winners are determined — primary metric (CTR, conversion rate, ROAS, or CPA depending on objective), minimum confidence level, minimum sample size reached, and guardrail metrics that must not degrade (e.g., a headline that lifts CTR but tanks conversion rate is not a winner). Confirm a candidate winner is statistically real with python "${CLAUDE_PLUGIN_ROOT}/scripts/significance-tester.py" --control-visitors {n} --control-conversions {n} --variant-visitors {n} --variant-conversions {n} --confidence 0.95 before declaring it. Include rules for ties and inconclusive results.
10. Create documentation template for results and learnings: Design a standardized test card template capturing: hypothesis, variable tested, variants, audience, platform, date range, sample size, primary metric results, secondary metrics, statistical significance, winner declaration, key learning, and next test recommendation. This builds the creative knowledge base over time.