Meticulous Simulate And Diff

Simulate a session and analyze diffs

This skill covers running a single simulation and interpreting the results. For the simulate command's full option reference see the meticulous-cli skill's simulate reference.

Before starting, run the meticulous-cli-update skill to ensure the Meticulous CLI and skills are up to date — unless it has already run earlier in this conversation, in which case skip it.

Prerequisites

• A sessionId to replay
• An appUrl (local dev server, or leave blank to use the original recorded URL)
• Optionally: a baseReplayId — the ID of a prior replay to diff screenshots against. Without this, screenshots are stored but not compared.

If you don't have a baseReplayId, you can find one from a downloaded test run:

meticulous download test-run
# Then inspect ~/.meticulous/test-runs/<testRunId>/coverage.json
# or check the testCases[].replayId fields

Step 1 — Run the simulation

With a base replay (diff mode)

meticulous simulate \
  --sessionId=<sessionId> \
  --appUrl=<url> \
  --baseReplayId=<baseReplayId> \
  --headless

Capture the full stdout. Key things to look for:

# Per-screenshot diff outcomes (one line each):
0.412% pixel mismatch for screenshot screenshot-1234.png (threshold is 0.100%) => FAIL!
0.000% pixel mismatch for screenshot screenshot-5678.png (threshold is 0.100%) => PASS

# Final summary block:
=======
View simulation at: https://app.meticulous.ai/projects/<org>/<project>/simulations/<headReplayId>
View comparison with base: https://app.meticulous.ai/projects/<org>/<project>/simulations/<baseReplayId>/compare-to/<headReplayId>
=======

If there are no FAIL! lines: the session is visually identical to the base — report no regressions, then proceed to Step 6.

Proceed to Steps 2–6 to locate and analyse any diffs, then submit feedback.

Without a base replay (quick-check mode)

If no baseReplayId is available, omit it. Screenshots are still stored locally for direct visual inspection:

meticulous simulate \
  --sessionId=<sessionId> \
  --appUrl=<url> \
  --headless

Then locate the replay directory (Step 2) and open the screenshots in <replayDir>/screenshots/ to verify the UI looks correct. There are no diff images in this mode — inspection is purely visual. Steps 3–5 do not apply; still complete Step 6 after inspection.

Step 2 — Extract the head replay ID and locate the replay directory

From the View simulation at: URL, extract the <headReplayId> (the last path segment).

To find the local replay directory created by this run:

ls -lt ~/.meticulous/replays/ | head -5

The most recently created entry will be the head replay's directory (named with a timestamp, e.g. 2024-01-15T12-30-45.123Z-abc123/). Note this path — it's referred to below as <replayDir>.

Step 3 — Identify which screenshots diffed

ls ~/.meticulous/replays/<replayDir>/diffs/<baseReplayId>/

Each .png file here corresponds to a screenshot where a visual difference was detected. The pixel diff image highlights changed pixels in color. There are also thumb_ prefixed thumbnail versions.

Note the filenames — they match the screenshot identifiers (e.g. screenshot-after-event-42.png).

Step 4 — Analyze the HTML diff for each diffed screenshot

Each screenshot has a corresponding metadata file containing a full HTML snapshot of the page taken just before the screenshot was captured. These files are already on disk:

• Head metadata: ~/.meticulous/replays/<replayDir>/screenshots/<screenshotFilename>.metadata.json
• Base metadata: ~/.meticulous/replays/<baseReplayId>/screenshots/<screenshotFilename>.metadata.json

The base metadata is permanently cached when the simulation downloads the base replay, so no additional download is needed.

Read both .metadata.json files. The relevant fields are:

• before.dom — full HTML of the page at screenshot time; diff these two strings to understand what changed
• before.routeData.url — which page/route the screenshot was taken on

When diffing the HTML, focus on tag additions/removals, class attribute changes, and text content changes.

The per-screenshot stdout lines also report mismatchFraction (proportion of pixels that changed). If there is a pixel diff but the before.dom strings are identical, the change is purely visual (e.g. a color shift) rather than structural.