Android

WhatsApp — send a message to a contact

1. android_open_app("com.whatsapp")
2. android_wait(text="Chats", timeout_ms=5000) — landing tab.
3. android_tap_text("Search") or tap the magnifying glass icon.
4. android_type("<contact name>") — narrows the chat list.
5. android_tap_text("<contact name>") — opens the chat.
6. android_tap_text("Message") — focus the input field. (Label varies by locale; fall back to read_screen and find the EditText node.)
7. android_type("<message body>")
8. STOP and report. Do not auto-tap Send — it is on the destructive-verb list. Either let the user tap Send, or invoke the Tier 5 confirmation modal by calling android_tap_text("Send") and letting the phone prompt.

Pitfalls:

• Multiple chats with the same contact name will show a picker — if read_screen shows >1 match, stop and ask which one.
• End-to-end encryption banners on first-chat-of-the-day intercept the input field. Dismiss with android_tap_text("OK") if present before typing.

Spotify — play a track or playlist

1. android_open_app("com.spotify.music")
2. android_wait(text="Search", timeout_ms=5000)
3. android_tap_text("Search")
4. android_type("<query>")
5. android_wait(text="<first word of query>", timeout_ms=3000)
6. android_tap_text("<first matching result>") — opens the track or playlist.
7. android_tap_text("Play") — start playback.
8. android_read_screen() — verify the now-playing bar shows the expected track. Report and STOP.

Pitfalls:

• Free-tier accounts sometimes play a different track than the one tapped (shuffle mode override). Always verify from the now-playing bar, not from what you tapped.
• The Play button is sometimes labeled by icon only. If tap_text("Play") fails, re-read and use the node with className containing PlayButton.

Google Maps — navigate to a place

1. android_open_app("com.google.android.apps.maps")
2. android_wait(text="Search here", timeout_ms=5000)
3. android_tap_text("Search here")
4. android_type("<place>")
5. android_wait(text="<place-first-word>", timeout_ms=3000)
6. android_tap_text("<first result>")
7. android_tap_text("Directions")
8. android_read_screen() — report ETA + distance + mode, then STOP. Do not auto-start turn-by-turn. "Start" is on the destructive-verb list because it changes the phone's mode of operation.

Pitfalls:

• Maps opens straight into "navigation mode" if the user had a previous route cached. Always read_screen first and check whether you're on home or already navigating.
• Location permission prompts block everything. If you see a permission modal, stop and hand it to the user.

Settings — change a specific setting

1. android_open_app("com.android.settings")
2. android_tap_text("Search settings") — Settings has a global search that saves 80% of navigation effort.
3. android_type("<setting name>") — e.g. "wifi", "notifications", "display".
4. android_wait(text="<setting name>", timeout_ms=3000)
5. android_tap_text("<matching result>")
6. android_read_screen() — you're now on the target settings page. Make the toggle change (tap_text(<label>)) and verify.
7. Report the before/after state, STOP.

Pitfalls:

• OEM skins (Samsung One UI, Xiaomi HyperOS) ship with different Settings package names on some devices. If com.android.settings fails, try com.samsung.android.lool or fall back to android_get_apps and filter on "Settings".
• The Search bar is at the top on stock Android but mid-screen on Samsung. Use read_screen to find it, not coordinates.

Tinder — swipe through the card deck

1. android_open_app("com.tinder")
2. android_wait(text="Discovery", timeout_ms=5000) or landmark of your choice.
3. android_read_screen() — confirm a profile card is visible.
4. android_swipe("right") to like, android_swipe("left") to pass. (Do NOT use coordinate taps — Tinder's card stack is canvas-rendered and coordinates are unstable.)
5. Repeat step 4 up to 3 times per invocation.
6. android_read_screen() — verify the deck advanced. Report and STOP.

Pitfalls:

• A "It's a Match!" modal interrupts the deck. Read the screen and tap Keep Swiping before continuing.
• Do not auto-send messages. Match-message composition is a destructive action the user should drive.

9. Safety rails awareness

The phone enforces user-configurable Tier 5 safety rails on every bridge.command. You will see these as 403 responses. Do not retry on a 403 — report it.

• Blocklist. A DataStore-backed list of package names (banking, payments, password managers, 2FA apps) blocked by default. If the current app is on the blocklist, every tool except android_ping and android_current_app returns 403 blocked package <name>. Tell the user to either switch out of the blocked app or remove the app from the blocklist in Hermes-Relay → Bridge → Safety.
• Destructive-verb confirmation. android_tap_text and android_type are intercepted when the payload contains any of the default destructive verbs (send, pay, delete, transfer, confirm, submit, post, publish, buy, purchase, charge, withdraw). The phone pops a confirmation modal with the full payload — the user has a configurable timeout (default 30 s) to allow or deny. A denied or timed-out confirmation returns 403 user denied destructive action.
• Idle auto-disable. After a configurable number of minutes of inactivity (default 30, range 5–120), the phone auto-disables device control and posts a notification. The next bridge.command will return 403. Tell the user to re-enable the bridge master toggle.

Agent discipline: expect 403s, report them, do not retry. The user's safety config is not something you should try to work around.

10. Sideload-only features

Hermes-Relay ships in two build flavors: googlePlay (Play Store, conservative accessibility config) and sideload (full agent control). The sideload build unlocks:

• Voice-to-bridge intents. Transcribed voice utterances like "text mom saying on my way" or "open Spotify" are classified on-device and routed through the bridge channel instead of becoming chat messages. Destructive intents (SMS) speak a confirmation and start a 5-second cancellable countdown before dispatching.
• Contact search, direct SMS, direct dial. Not exposed as android_* tools — they're user-facing voice intents only. You will not see them in your tool list.
• Full accessibility capability set. typeAllMask, gesture dispatch, interactive windows, view IDs — used by the direct tools under the hood. The Play flavor uses a narrower subset.
• Location queries. Phone-side, voice-triggered only.

Agent rule: you do not need to know which flavor is running to use the android_* tools. Every tool in section 3 works in both flavors. The flavor only matters if you're building a feature that requires sideload capabilities, in which case check BuildFlavor.current on the phone side, not from the tool layer.

11. Extending this skill

This file is read by the LLM before it touches the android_* tools. Keep it tight and keep it current:

1. When new android_* tools land, add them to the section 3 table with a one-line purpose.
2. When new apps are commonly requested, add them to section 7 and write a 6–12 step playbook in section 8.
3. When the Tier 5 safety rails change (new blocklist defaults, new destructive verbs, new gates), update section 9.
4. Do not add marketing prose, screenshots, or "Why Hermes-Relay is great" sections. This is an agent-readable spec, not a README.

When adding a new playbook, follow the section 8 template: numbered steps using tool-call syntax, a "Pitfalls" subsection with 2–4 bullets, and an explicit STOP-and-report step before any destructive action.