E2E Matrix Runner

E2E Matrix Runner

Run every cell of the matrix below on real hardware and report a row for each. .claude/commands/e2e-tests.md is the authority on what a row means and how to verify one; read it before starting and follow it. This file adds only what a delegated agent needs: the matrix, the devices, and the rules for reporting.

For a parallel run, delegate $e2e-matrix-runner-google (Play, Amazon, Horizon, VegaOS) and $e2e-matrix-runner-apple (iOS) instead; both follow this file's techniques and reporting contract on their narrowed scope.

The matrix (pinned scope — do not renegotiate per run)

Six frameworks: react-native-iap, expo-iap, flutter_inapp_purchase, kmp-iap, maui-iap, godot-iap, plus the native packages/google (Android) and packages/apple (iOS) rows.

That is 7 iOS + 21 Android/Horizon + 2 VegaOS cells. Do not silently drop a cell.

Standing approvals for E2E runs

• Sandbox/test purchases on Play, Amazon, and iOS are pre-approved: tap the purchase sheet, type the pinned sandbox password below, and finish the transaction without asking.
• Horizon checkout is real money: take the Confirm tap only when its UI is visibly marked test or sandbox. Otherwise report build + launch coverage.
• VegaOS: run at least one purchase attempt when device input and tester UI are available.

Devices attached to this machine

Discover them rather than trusting this list, with adb devices -l and xcrun devicectl list devices. At the time of writing:

The iPhone UDID has changed mid-session before (re-enumeration); re-check it after any device was not found error instead of retrying the stale id. The Mac's LAN address (the iPhone's IAPKit/Metro origin) also changes between networks — confirm with ipconfig getifaddr en1 (fallback en0) each run.

Every example shares the application id dev.hyo.martie, so only one framework can be installed at a time per device. Uninstall before installing the next, and expect INSTALL_FAILED_UPDATE_INCOMPATIBLE when signing keys differ. This applies to iOS too: all six example apps share the bundle id, so install, purchase, and move to the next framework strictly one at a time.

Driving the hardware

Android. adb -s <serial> install -r <apk>, adb -s <serial> shell input tap X Y, and adb -s <serial> exec-out screencap -p > shot.png. Read the screenshot before every tap; do not tap coordinates from memory. With several devices attached, a debug build follows only the device ANDROID_SERIAL names and otherwise links Play, so build each cell with it set to that cell's device and check the store line before installing, as the ## Android Store Selection section of .claude/commands/e2e-tests.md shows.

Quest. screencap returns black because Quest blocks capture of the VR compositor. Do not conclude the device is undriveable. Drive the display-0 VR panel directly: adb -s $QUEST shell input tap X Y reaches the panel, and adb -s $QUEST shell uiautomator dump exposes the accessibility tree with text and bounds — dump, tap the dumped coordinates, dump again. tap, swipe, and keyevent (BACK dismisses the Horizon checkout dialog) are all verified working on display 0 (Quest 3, Horizon OS, 2026-09-25: RN RedBox dismissed, Kepler home → Purchase Flow navigated, MAUI scrolled to products, Purchase tapped into com.oculus.store checkout and BACKed out cleanly).

adb -s "$QUEST" shell am start -n <launcher-activity>  # resolve per framework
adb -s "$QUEST" shell uiautomator dump /sdcard/ui.xml
adb -s "$QUEST" pull /sdcard/ui.xml .
adb -s "$QUEST" shell input tap X Y  # coordinates from the dump

Resolve the launcher activity per framework (cmd package resolve-activity --brief ..., or monkey -p <pkg> -c android.intent.category.LAUNCHER 1): Flutter uses io.flutter.embedding.android.FlutterActivity, MAUI a crc...MainActivity. RN/Expo debug builds need their Metro (adb reverse tcp:8081, one packager at a time) and a cold start if the first bundle load stalls. If Metro serves a stale graph (same RedBox after an entry change, or a 500 Got unexpected undefined), restart it with --reset-cache. When the panel is empty right after launch, wait for the JS bundle (RN shows 6 bare nodes until loaded); a transient null root node from the dump usually clears on retry.

Verified limitation (Quest 3, Horizon OS, scrcpy 3.x): the above applies to display 0 only. On a scrcpy virtual display (--new-display), touch injection is silently ignored — adb shell input -d N tap, explicit input touchscreen -d N tap, and monkey-script tap(x,y) all leave the frame bit-identical (compare md5 before/after). Key events (input -d N keyevent) do reach the app. Prefer display 0 + dumps over the virtual-display + screenshot path; one md5-compare per OS upgrade is enough, do not burn the run re-proving it.

Consequences: drive each Horizon app as install + launch + navigate + tap to the purchase gate. The com.oculus.store checkout dialog renders its full text into the dump (product, total, payment method, Confirm), so the test/sandbox gate stays enforceable without screenshots. A build that linked Play instead of Horizon fails earlier with Play's own error (RN-IAP: initConnection failed ... responseCode -1, no Play Store on Horizon OS); its store line says store=play, so rebuild for the Quest. Report any other store error exactly, not as a generic input block. Horizon purchase cells are BLOCKED by default (real-money Confirm, or no store connection), never guessed.

iOS. Build with xcodebuild -destination "id=$UDID", install with xcrun devicectl device install app, launch with xcrun devicectl device process launch. Never use iPhone Mirroring for debugging or purchases: the phone stays in the user's hand, and Mirror refuses to connect while it is in use. Drive the physical iPhone directly instead.

A physical iPhone can be driven, through XCUITest. Build a UI-test bundle once and point it at any installed app with XCUIApplication(bundleIdentifier:), then run it with xcodebuild test-without-building -xctestrun, passing the flow in environment variables so one signed runner serves every framework. Without an Xcode account, build with CODE_SIGNING_ALLOWED=NO and hand-sign the runner and its nested .xctest with a wildcard development profile.

Shortcut when the custom runner is not at hand: maestro-runner drives a physical iPhone over its bundled WebDriverAgent with Maestro YAML flows as-is. Verified working on this machine (Korea's iPhone, iOS 27, team PRDQGB267K):

export PATH="$HOME/.maestro-runner/bin:$PATH"
maestro-runner --platform ios --device "$IOS_UDID" --team-id PRDQGB267K \
  test flow.yaml

Rules for this path, all verified the hard way:

• Do NOT pass --wda-bundle-id: a custom bundle forces a rebuild that fails signing (No Accounts, stale wildcard profile). The default bundle reuses the good cache under ~/.maestro-runner/cache/wda-builds/.
• On first use with a current Xcode, the bundled WDA fails to build because the project pins IPHONEOS_DEPLOYMENT_TARGET = 12.0 (below Xcode's 15.0 floor). Patch once in the user-local checkout and rerun: sed -i '' 's/IPHONEOS_DEPLOYMENT_TARGET = 12\.0/IPHONEOS_DEPLOYMENT_TARGET = 15.0/g' ~/.maestro-runner/drivers/ios/WebDriverAgent/WebDriverAgent.xcodeproj/project.pbxproj
• takeScreenshot only saves plain filenames (shot.png lands under the run's assets/ dir). Absolute paths like /tmp/x.png fetch fine over WDA but fail to save (no such file or directory) while the step still passes — a silent evidence loss. Always confirm the PNG exists before claiming a visual check.
• A flow test/... writes reports/<timestamp>/ with report.json (status: passed), junit-report.xml, and per-command assets. The suite exit code is unreliable alone; read report.json for the verdict.