Simdeck

SimDeck Agent Guide

SimDeck automates iOS Simulators and Android emulators. Use the CLI for automation and the browser UI for live human visibility. iOS works with NativeScript, UIKit, SwiftUI, React Native, Expo, and Flutter apps; Android live viewing uses the emulator -share-vid shared display surface plus SimDeck's native host H.264 encoder, with lifecycle, screenshots, logs, and UIAutomator hierarchy dumps also handled through ADB.

SimDeck uses one long-running local service. Run simdeck first; it starts or reuses the background service and prints the browser URL plus pairing code. Pass --open only when you want SimDeck to open the default browser itself. Pass -p <port> or --port <port> when the service should use a non-default port. Pass -a or --autostart when the service should also be registered as a macOS LaunchAgent.

Example response from simdeck:

{
  "ok": true,
  "pairingCode": "401974",
  "pid": 91285,
  "projectRoot": "-",
  "started": true,
  "url": "http://127.0.0.1:4310"
}

simdeck
simdeck --open
simdeck -p 4311
simdeck -a
simdeck pair # prints LAN/Tailscale pairing URLs, code, and iOS QR
simdeck service stop
simdeck service kill # stops all SimDeck services from any binary
simdeck service restart

Usually http://127.0.0.1:4310. Pass a simulator name or UDID to simdeck when the browser URL should open with ?device=<UDID>. Use simdeck -p 4311 when the default port is not the right fit. Use simdeck pair when a native iOS client needs to pair. It starts or refreshes the LaunchAgent-backed service, detects LAN and Tailscale IPv4 addresses, and prints a QR with a simdeck://pair URL. Normal service restarts preserve the token and pairing code; use simdeck service reset only when you need to rotate them. Use simdeck service kill or simdeck service killall when you need a clean slate across installed and source-checkout binaries.

Optionally, when a user might want to drive the current simulator from their phone, run simdeck link [sim] to print an https://app.simdeck.sh/open universal link that opens SimDeck Studio on iOS (or the launchpad in a browser) pre-targeted at the resolved simulator.

Always first run simdeck and open the reported URL in the in-app browser using the Browser tool if available.

If Browser Use is not available, only then use simdeck --open; it opens the default browser and may take focus away from the app.

Device and app

Start by choosing a project default device. simdeck use <UDID> stores the selection for the current workspace/CWD so later commands can omit the UDID. Explicit UDIDs, --device, SIMDECK_DEVICE, and SIMDECK_UDID still work for one-off overrides. Prefer short forms in agent loops, such as simdeck tap "Continue" and simdeck snapshot --format agent --max-depth 2 -i.

simdeck list
simdeck list --format json
simdeck use <UDID>
simdeck boot <UDID>
simdeck boot android:<AVD_NAME> --android-emulator-arg=-no-snapshot
simdeck shutdown
simdeck erase
simdeck core-simulator restart
simdeck install /path/to/App.app
simdeck install /path/to/App.ipa
simdeck install android:<AVD_NAME> /path/to/app.apk
simdeck launch com.example.App
simdeck uninstall com.example.App
simdeck open-url myapp://route
simdeck open-url https://example.com
simdeck toggle-appearance
simdeck camera sources
simdeck camera start com.example.App --file /absolute/path/to/camera.mov
simdeck camera start com.example.App --webcam
simdeck camera switch --placeholder
simdeck camera status
simdeck camera stop

simdeck list defaults to compact JSON for token-efficient agent selection. Use simdeck list --format json only when you need full paths and display metadata.

Build apps with project tooling.

Android devices use IDs like android:Pixel_8_API_36. simdeck list discovers AVDs from the Android SDK. SimDeck-owned Android boots use emulator gRPC screenshot streaming for live video, keep the emulator shared-video surface as fallback, and use --android-gpu host by default. Android browser streams default to software H.264 at 60 fps with a 960px long-edge cap so emulator gRPC frame production stays smooth. On macOS, managed boots use -qt-hide-window so the Qt render loop stays active without showing the native emulator window. Use simdeck service restart --android-gpu auto or --android-gpu swiftshader_indirect only when host GPU rendering is unstable. Optional user defaults live in ~/.simdeck/config.json. Supported keys include service.port, android.emulatorArgs, and android.disableAudio; explicit CLI/API boot arguments still win for one-off runs.

Fast agent inspection

Use targeted checks for test loops. describe is a diagnostic snapshot of the whole hierarchy. For verification, prefer the service APIs exposed by simdeck/test: action, query, waitFor, assert, selector tap, and batch.

simdeck describe
simdeck describe --format agent --max-depth 4
simdeck describe --format agent --max-depth 4 --interactive
simdeck snapshot --format agent --max-depth 4 -i
simdeck describe --format compact-json
simdeck describe --point 120,240
simdeck describe --source auto
simdeck describe --source nativescript|react-native|flutter|swiftui|uikit
simdeck describe --source native-ax
simdeck describe --source android-uiautomator
simdeck describe --direct
simdeck wait-for --label "Welcome" --timeout-ms 5000
simdeck wait --label "Welcome" --timeout-ms 5000
simdeck assert --id login.button --source auto --max-depth 8

The default source is native-ax, which is the fastest and most universal path for agents. Use --source auto when you want richer NativeScript, React Native, Flutter, SwiftUI, or UIKit inspector data before native accessibility fallback. Use --direct or --source native-ax for the private CoreSimulator accessibility bridge. Use --source android-uiautomator for Android emulator UIAutomator hierarchies. For Android IDs, describe uses uiautomator dump; use --format agent or --format compact-json the same way as iOS. Use --interactive or -i when an agent only needs controls and actionable framework nodes; SimDeck keeps ancestor context so the output is still navigable. Agent output labels nodes with refs such as @e3; reuse them with simdeck press @e3. snapshot, press, and wait are aliases for describe, tap, and wait-for.

Prefer selectors, coordinates only when needed. Selector taps go through the service and wait for the element server-side. Use --expect-id, --expect-label, or another --expect-* selector when the tap should also wait for the next screen before returning.

simdeck tap --id LoginButton --wait-timeout-ms 5000
simdeck tap --id com.apple.settings.screenTime --expect-id BackButton
simdeck tap --label "Continue" --element-type Button
simdeck tap 120 240
simdeck tap "Continue"
simdeck press @e3

For persistent app integration tests, use simdeck/test instead of shelling out repeatedly:

import { connect } from "simdeck/test";

const udid = "your-device-udid";
const simdeck = await connect({ udid });

try {
  await simdeck.launch("com.example.App");
  await simdeck.waitFor({ id: "login.button" }, { maxDepth: 8 });
  await simdeck.tap(0.5, 0.5);
  await simdeck.assert({ label: "Welcome" }, { maxDepth: 8 });
  const matches = await simdeck.query({ id: "account.name" });
  console.log(matches);
} finally {
  simdeck.close();
}

simdeck/test state-query helpers default to source: "native-ax" for fast agent control. Pass source: "auto" only when you intentionally want richer framework inspector trees before native accessibility fallback.

Use tree()/describe only when a test needs to print the whole UI for debugging. In a normal agent loop, do not fetch the full tree after every action; verify the specific element or text that proves the step succeeded.