Writing E2E Tests

/writing-e2e-tests — How desktop E2E tests are written

An E2E test is worth its cost only when it fails for exactly one reason: the user-visible behavior it names is broken. It must not fail because the machine was slow, the theme changed, a sidebar moved, or a retry happened to pass.

Read /writing-tests first for the general rules (observable state, no mocks, migration ledgers). Read apps/desktop/e2e/README.md for the full fixture and project reference. This skill is the checklist that turns those into a test that stays green for the right reasons.

Choose the layer before writing anything

If most assertions in a draft E2E test read editor state after page.evaluate, the behavior probably belongs in TestEditor. Keep the E2E test as a thin proof that the real UI reaches it.

Waiting: never sleep

waitForTimeout is banned. Wait for the condition that makes the next step valid:

When elapsed time is itself the contract (debounce, momentum, idle timeouts), put the rule in a pure function driven by timestamps and unit-test it. If the E2E test must remain, dispatch events inside the page and wait on the page clock (performance.now()) that stamps them. glyph-view.spec.ts zoom momentum is the template.

Negative assertions ("nothing else happened") need a positive anchor: wait for the event that would have triggered the unwanted outcome, then assert the outcome is absent.

Oracles: assert what the editor decided, not what pixels look like

Never count or sample hard-coded colours from a canvas. Colour oracles break under themes, alpha, anti-aliasing, device scale, and renderer changes, and many pass when the feature is broken (an empty canvas has zero "wrong" pixels).

Use published state instead:

• geometry: editor.outline(), pointPosition(), selectionIds(), selectionBounds();
• interaction: toolState(), hoverId(), activeSnapGuides();
• rendering decisions: visibleOutlines(nodeId) and handleStates(node) on the glyph node definition;
• UI state: roles, accessible names, aria-pressed, aria-checked, aria-disabled, data-* state published for tests;
• persistence: savedGlyphNames() for .shift files and exportedGlyphNames() for exported fonts — never existsSync, byte inequality, or size > 0.

If the state you need is not published, publish it from the production code that renders it (same computation, not a parallel reimplementation), add a unit test for it, then assert it. That is how visibleOutlines and handleStates were added.

Assertions must be exact enough to fail on the wrong answer:

• assert the exact set or order, not count() > N or not.toEqual(before);
• derive expectations from the fixture or the UI (for example, read the ordered category buttons and compute the expected range) instead of hard-coding one neighbour;
• run the mental-deletion check: if the feature were a no-op, would this assertion fail?

Goldens: one contract, one focused capture

All goldens go through fixtures/snapshots.ts; scripts/check-e2e-projects.mjs rejects toHaveScreenshot or toMatchSnapshot anywhere else.

• expectCanvasSnapshot(editor, name) — the editor canvas stack, exact comparison, after the canvas has rendered.
• expectPanelSnapshot(locator, name) — one panel, menu, toolbar, or dialog.
• expectPageSnapshot(page, name) — full window; use sparingly, only for overall chrome.

Before every golden:

1. Assert the semantic precondition that makes the image meaningful: the point count, selection, tool state, outline targets, handle states, zoom, active theme (data-color-theme), or devicePixelRatio.
2. Park the pointer off the captured element unless hover is the contract.
3. Capture the smallest element that shows the contract. Do not use a full-page golden to protect a toolbar button.

Rules:

• One golden per distinct visual contract. Do not add near-duplicate images; if two captures are byte-identical, one of them is redundant.
• Canvas goldens use maxDiffPixels: 0 with threshold: 0.02 on every host: enough for antialiasing differences between hosts (≤ 0.009), far below a real colour-token change (~0.07). Never raise either to make a mismatch pass; the default threshold of 0.2 silently accepts token changes.
• Interface goldens (expectPanelSnapshot, expectPageSnapshot) are exact but compared on CI only, with baselines generated on the runner via the ci: update visual snapshots label. Never commit a locally generated interface baseline. Prefer semantic assertions and keep interface goldens few.
• Goldens never pass on retry. The helpers throw on a retry attempt, so explain the first failure.
• Use canvas-local positions (dragCanvas(), canvasPagePoint()) so layout changes cannot redraw geometry.
• Theme goldens select the theme through the product and assert data-color-theme first. HiDPI goldens use test.use({ deviceScaleFactor: 2 }) and assert devicePixelRatio first. Add one golden per palette branch or rendering path, not one per theme.

Updating baselines: only for an intentional appearance change. Run the focused spec with --update-snapshots -g "<title>", open every changed PNG and describe what changed, then rerun without update mode and with --retries=0 --repeat-each=3. Canvas baselines may be generated locally. Interface baselines are generated only by the ci: update visual snapshots label; inspect the runner-generated images before merging.