Cross Browser Testing

<objective>
Chrome-only testing gives false confidence: a layout that works in Chromium can break in WebKit, a clipboard call that succeeds in Chrome silently no-ops in Firefox, and a partitioned-cookie flow can pass everywhere except the one engine your users are on. This skill produces an analytics-driven browser matrix, a Playwright (or cloud-platform) config that runs it, and a committed log of known browser divergences — each verified by a test that asserts the user outcome, not the CSS.
</objective>

Quick Route

Discovery Questions

Check .agents/qa-project-context.md first — if it exists, use it and skip anything already answered there. Then:

1. Target browsers from analytics: What do actual users use? Pull browser/OS data from your analytics tool. Testing browsers nobody uses is waste; missing a browser 15% of users rely on is a bug.
2. Desktop and mobile? Mobile Safari on iOS and Chrome on Android render differently than their desktop counterparts. Treat them as separate matrix entries.
3. Cloud platform: BrowserStack, Sauce Labs, LambdaTest, or local engines only? Cloud platforms provide real branded browsers and OSes; Playwright's bundled engines cover Chromium, Firefox, and WebKit (not Chrome/Safari themselves).
4. Progressive enhancement or pixel-perfect? Progressive enhancement accepts graceful degradation. Pixel-perfect demands identical rendering. The answer determines pass/fail criteria.
5. Existing Playwright config? If the project already uses Playwright, cross-browser testing is a configuration change, not a new tool.

Core Principles

1. Analytics-driven matrix. Test what your users actually use. A browser at 0.3% traffic does not need the same investment as one at 40%. Check analytics quarterly — browser share shifts.

2. Progressive enhancement over pixel-perfect. Identical rendering across all browsers is neither achievable nor necessary. Define what "works" means: core functionality operates, content is accessible, layout is usable. Visual differences in shadows, gradients, or animation timing are acceptable.

3. Safari and Firefox surface the most cross-browser bugs. Chrome-only testing catches Chrome bugs. Safari's WebKit engine and Firefox's Gecko engine have the most behavioral differences from Chromium. Prioritize them.

4. Test functionality, not rendering-engine internals. A cross-browser test should verify that the user can complete a task, not that a CSS property renders identically. Visual comparison tools handle pixel-level differences.

5. Engines are not brands. Playwright's WebKit is not Safari and its Chromium is not Chrome — they share an engine, not the shipped product (codecs, fonts, enterprise policy, update cadence all differ). Report "WebKit coverage," not "Safari coverage," unless you ran real Safari on a cloud grid.

6. One test, multiple browsers. Write tests once. Run them across browser configurations. Never duplicate test logic for different browsers.

Browser Matrix Design

Analytics-Based Methodology

Step 1: Export browser/OS data from analytics (last 90 days)
Step 2: Rank by session share
Step 3: Group into tiers
Step 4: Assign test coverage per tier
Step 5: Review quarterly

Tier System

Example Matrix (derived from analytics)

## Browser Matrix — Q1 2026 (next-review: 2026-04-01)

| Browser | Version | Platform | Traffic % | Tier | Notes |
|---------|---------|----------|-----------|------|-------|
| Chrome | Latest | Windows | 34% | P0 | |
| Chrome | Latest | macOS | 12% | P0 | |
| Safari | Latest | macOS | 11% | P0 | WebKit-specific issues |
| Chrome | Latest | Android | 15% | P0 | Mobile viewport |
| Safari | Latest | iOS | 14% | P0 | Mobile Safari quirks |
| Firefox | Latest | Windows | 5% | P1 | Gecko rendering |
| Edge | Latest | Windows | 4% | P1 | Chromium-based but different UA/policy |
| Samsung Internet | Latest | Android | 3% | P1 | Chromium fork, lagging engine |
| Firefox | Latest | macOS | 1.5% | P2 | |
| Chrome | N-1 | Windows | 1.2% | P2 | Previous major version |

Version Coverage Strategy

• Latest: Always test current stable release.
• Latest - 1: Test previous major version only for P0 browsers where analytics show >1% on older versions.
• Extended Support Release (ESR): Test Firefox ESR only if enterprise users are a significant segment.
• Do not test: Beta/Canary/Nightly releases unless you are a browser vendor or building browser-facing tools.

Playwright Browser Configuration

Playwright ships three browser engines — Chromium, Firefox, WebKit — so no cloud platform is needed for basic engine-level coverage. This is engine coverage, not brand coverage: bundled WebKit ≠ Safari and bundled Chromium ≠ Chrome (see Core Principle 5). Define one project per matrix entry, map mobile devices via devices[...], and drive locally installed branded browsers with the channel option.

See references/playwright-and-cloud-config.md for the full playwright.config.ts project list, branded-channel snippets, and --project run commands.

When to use channels: When you need real branded behavior that differs from the bundled engine — installed Chrome (channel: 'chrome') or Edge (channel: 'msedge') for extension support, enterprise policy, or codecs. WebKit and Firefox have no channel option; they are always Playwright's bundled engines. Note the edge project in the config and the msedge channel snippet are illustrative alternatives, not two projects to merge — a config needs one edge project, not both.

page.screencast() (Playwright 1.59+, current in 1.60) captures annotated video of a cross-browser run — useful when a matrix failure needs human review across engines. For agent-driven re-runs and stepping through a failure, use --ui (UI mode) or --debug (Inspector); PWDEBUG=1 and --headed are the other real entry points. There is no --debug=cli flag.

Cloud Platform Setup

Cloud platforms (BrowserStack, Sauce Labs) provide real branded-browser/OS instances Playwright connects to over a CDP/Playwright WebSocket endpoint. Pass credentials and capabilities via environment variables, and keep the platform's playwrightVersion aligned with the Playwright version in package.json (currently 1.60.x — a client/server mismatch causes socket errors).

BrowserStack now recommends the npx browserstack-node-sdk runner plus a client.playwrightVersion capability (in addition to browserstack.playwrightVersion) to keep the client and grid sockets in lock-step. The raw wsEndpoint/CDP config below still works for direct connections; use the SDK path for new setups.

See references/playwright-and-cloud-config.md for the BrowserStack config (with the client.playwrightVersion cap), the Sauce Labs config, and the GitHub Actions parallel matrix that fans out across cloud browsers.

Common Cross-Browser Issues

Real divergences that surface in cross-browser testing, with detection patterns and fixes. The CSS workarounds and Playwright tests for each are in references/common-browser-issues.md, covering: partitioned cookies / CHIPS in iframes, <input type="date">, the Clipboard API, scroll-behavior, backdrop-filter, the <dialog> element, View Transitions, and Web Animations timing.

Modern Cross-Browser Gotchas (2026)

The classic Safari-laggard list is mostly resolved (flexbox gap, :has() shipping, same-document View Transitions are all Baseline). Today's real divergences:

• Partitioned cookies / partitioned storage: Chrome's CHIPS (Partitioned attribute), Safari's ITP, and Firefox's State Partitioning each behave differently for embedded third-party contexts. Test third-party cookies in an iframe per engine, not just "the browser supports cookies." See the runnable per-engine iframe test in references/common-browser-issues.md.
• :has() selector performance: Universally supported since 2023, but a :has()-heavy page can have very different style-recalc cost across engines. Watch list — profile if a page feels janky in one engine; visual-regression it in visual-testing.
• View Transitions API: Same-document transitions are Baseline (Chrome 111, Safari 18, Firefox 144 — Oct 2025), so they are no longer a divergence. Cross-document transitions are still the gap: Chrome 126+, Safari 18.2+, Firefox behind a flag. Treat cross-document as progressive enhancement and verify the no-transition fallback.
• WebDriver BiDi: Production-ready in Selenium 4, partially supported in Playwright. For new cross-runner projects, BiDi is the convergence point. Watch list.

Known-Issues Log

When a divergence is real and you cannot fix the app immediately, record it in a committed file (docs/browser-issues.md) so it is not re-debugged from scratch. The table is the artifact Done When checks for, and every row's test must assert the user outcome, not the CSS property:

| Affected browser | Repro | Workaround / fallback / ticket | Test asserts (user outcome, not CSS) |
|------------------|-------|--------------------------------|--------------------------------------|
| Safari (WebKit) ≤17 | scroll-behavior: smooth is partial | rely on anchor nav; no JS scroll dependency | anchor link puts heading in viewport (`toBeInViewport`) |
| Firefox ≤102 | backdrop-filter unsupported | -webkit- prefix + rgba background fallback | overlay readable; modal content visible |
| Firefox (current) | cross-document View Transitions flagged off | progressive enhancement; instant nav fallback | navigation completes; target page heading visible |

Keep one row per divergence. A row with no ticket and no fallback is an open bug, not a documented issue.