Scripting Adb For CI

Gradle wiring — androidTestUtil, NOT androidTestImplementation

android {
  defaultConfig {
    testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    testInstrumentationRunnerArguments clearPackageData: 'true'
  }
  testOptions {
    execution 'ANDROIDX_TEST_ORCHESTRATOR'
  }
}

dependencies {
  androidTestImplementation 'androidx.test:runner:1.7.0'
  androidTestUtil           'androidx.test:orchestrator:1.6.1'   // NOT androidTestImplementation
}

clearPackageData: 'true' makes the Orchestrator run pm clear <pkg> between tests — the strongest isolation knob.

Direct ADB invocation (non-Gradle)

adb install --force-queryable -r path/to/orchestrator-1.6.1.apk
adb install --force-queryable -r path/to/test-services-1.6.0.apk

adb shell 'CLASSPATH=$(pm path androidx.test.services) app_process / \
 androidx.test.services.shellexecutor.ShellMain am instrument -w -e \
 targetInstrumentation com.example.test/androidx.test.runner.AndroidJUnitRunner \
 androidx.test.orchestrator/.AndroidTestOrchestrator'

--force-queryable is needed on API 30+ so the Orchestrator can bindService into the target despite package-visibility rules.

Tradeoffs

• Pros: each test starts from a known-good Application; one crash doesn't kill the rest of the run; pm clear between tests is real, not "I cleaned up in @After".
• Cons: per-test process spin-up adds ~1–2 s. A 500-test suite goes from ~5 minutes to ~15+ minutes. Reserve for integration suites where state leakage actually bites.

Patterns

Pattern: WRONG — gating CI on am instrument's exit code

# WRONG
adb shell am instrument -w -r com.example.test/androidx.test.runner.AndroidJUnitRunner
RC=$?
[ "$RC" -eq 0 ] && echo "tests passed" || echo "tests failed (rc=$RC)"
# WRONG because: am instrument calls System.exit(0) (AOSP Instrument.java) and AndroidJUnitRunner's
# finish() reports RESULT_OK regardless. $? is 0 even when tests fail — with OR without -w.

# RIGHT — keep -w (so stdout is complete) and -r (raw, parseable), then parse the stream
adb shell am instrument -w -r com.example.test/androidx.test.runner.AndroidJUnitRunner \
  | tee instrument.log
# -1 = error, -2 = failure; -3 (ignored) and -4 (assumption failure) are skips, not failures.
if grep -qE '^INSTRUMENTATION_STATUS_CODE: -[12]$' instrument.log; then
  echo "tests failed"; exit 1
fi
echo "tests passed"

Pattern: WRONG — confusing adb forward and adb reverse argument order

# WRONG — copy/pasted the forward syntax for reverse
adb reverse tcp:8080 tcp:8080      # ambiguous (works because both are tcp:8080)
adb reverse tcp:9000 tcp:8080      # this means: device port 9000 → host port 8080
# Developer expected: host service on 8080 reachable as 9000 on device.
# WRONG because: reverse argument order is REMOTE LOCAL. The device tries to dial 8080 on
# the host, but the dev meant to expose the local 8080 to the device on port 9000.

# RIGHT — adb reverse REMOTE LOCAL
# Expose host service on 8080 to the device as 9000:
adb reverse tcp:9000 tcp:8080
# In the app code: open a connection to localhost:9000 — it terminates at the host's :8080.

# Or, simpler, identical port:
adb reverse tcp:8080 tcp:8080      # device 8080 ⇆ host 8080

Pattern: WRONG — androidTestImplementation for the orchestrator APK

// WRONG
dependencies {
  androidTestImplementation 'androidx.test:orchestrator:1.6.1'
}
// WRONG because: the orchestrator is a SEPARATE APK, not a library. androidTestImplementation
// merges it into the test APK's classpath, which AGP rejects (or silently ignores). The
// correct configuration is androidTestUtil, which AGP installs as an additional APK.

// RIGHT
dependencies {
  androidTestImplementation 'androidx.test:runner:1.7.0'
  androidTestUtil           'androidx.test:orchestrator:1.6.1'
}

Pattern: timeout an instrumentation invocation

timeout 600s adb shell am instrument -w -r \
  com.example.test/androidx.test.runner.AndroidJUnitRunner | tee instrument.log
RC=${PIPESTATUS[0]}   # exit of `timeout`, not `tee`
case "$RC" in
  124) echo "timed out at 600s"; exit 124 ;;
  137) echo "timed out and had to be SIGKILLed"; exit 137 ;;
esac
# RC is 0 whether tests passed or failed (am instrument always exits 0); pass/fail comes from the stream:
if grep -qE '^INSTRUMENTATION_STATUS_CODE: -[12]$' instrument.log; then
  echo "failed"; exit 1
fi
echo "passed"

Pattern: retry-with-server-bounce

retry adb -s "$SERIAL" install -r app-debug.apk
retry adb -s "$SERIAL" shell am instrument -w -r \
  -e clearPackageData true \
  com.example.test/androidx.test.runner.AndroidJUnitRunner

The retry helper bounces the adb server on transient failure (see "Retries on transient errors" above).

Pattern: parallel sharded fan-out with per-device artefacts

SERIALS=( $(adb devices | awk '$2=="device"{print $1}') )
N=${#SERIALS[@]}
mkdir -p artifacts

pids=()
for i in "${!SERIALS[@]}"; do
  S="${SERIALS[$i]}"
  D="artifacts/${S//[:_]/-}"
  mkdir -p "$D"
  (
    adb -s "$S" logcat -c
    adb -s "$S" shell pm clear com.example.app
    adb -s "$S" shell am instrument -w -r \
         -e numShards "$N" -e shardIndex "$i" \
         com.example.test/androidx.test.runner.AndroidJUnitRunner \
         > "$D/results.txt"
    # am instrument exits 0 regardless — pass/fail is in the captured stream.
    if grep -qE '^INSTRUMENTATION_STATUS_CODE: -[12]$' "$D/results.txt"; then
      adb -s "$S" shell screencap -p /sdcard/fail.png
      adb -s "$S" pull /sdcard/fail.png "$D/fail.png"
      adb -s "$S" logcat -d -v threadtime > "$D/logcat.txt"
      exit 1
    fi
  ) &
  pids+=($!)
done

rc=0
for p in "${pids[@]}"; do wait "$p" || rc=1; done   # fail the job if any shard failed
exit $rc

Mandatory rules

• MUST pass -w -r to am instrument so the shell waits for the runner and emits raw, parseable output. Without -w the shell returns before any output is produced.
• MUST NOT gate CI on $? from am instrument — it calls System.exit(0) regardless of test outcome (with or without -w). Detect failures by grepping stdout for INSTRUMENTATION_STATUS_CODE: -1 (error) and -2 (failure).
• MUST distinguish -3 IGNORED and -4 ASSUMPTION_FAILURE from real failures when parsing INSTRUMENTATION_STATUS_CODE. -1 and -2 are the failure codes.
• MUST remember that adb forward is LOCAL REMOTE (host first) and adb reverse is REMOTE LOCAL (device first). The argument orders are opposite.
• MUST wrap long-running adb commands with timeout <N>s (or gtimeout on macOS). adb has no built-in timeout flag — adb -t is transport ID, not timeout.
• MUST use androidTestUtil("androidx.test:orchestrator:1.6.1"), not androidTestImplementation. The orchestrator is a separate APK, not a library.
• MUST retry transient adb errors (device not found, closed, protocol fault) with a bounded retry helper that calls adb kill-server; adb start-server between attempts.
• MUST install a trap cleanup EXIT that removes port forwards (adb forward --remove-all, adb reverse --remove-all), restores animation scales, and pm clears the app under test. Each step || true so cleanup never masks the test exit code.
• MUST pm clear <pkg> (not just am force-stop) before each test run. See ../../control/injecting-input-and-state/SKILL.md.
• MUST NOT rely on adb shell exit-code propagation on devices below API 24. Use a sentinel __ADB_RC=$? in stdout for older targets, or pin minSdk for the test infra.
• MUST NOT confuse adb -t <transport-id> with timeout. They share zero semantics.
• MUST NOT parallelise without serialising the install / setup phase — concurrent adb install to the same device race.
• PREFERRED: Test Orchestrator + clearPackageData: 'true' for hermetic isolation; pair with useTestStorageService: 'true' for routed artefacts (see ../../transfer/extracting-test-artifacts/SKILL.md).
• PREFERRED: namespace artefacts by serial (artifacts/${SERIAL//[:_]/-}/) so archive artifacts/**/* in CI captures everything cleanly.

Verification

• Every am instrument invocation in scripts/Gradle has -w -r, and no script branches on its $? as a pass/fail signal — failures are detected by parsing INSTRUMENTATION_STATUS_CODE: -1/-2 from stdout.
• adb forward and adb reverse appear with arguments in the documented order; CI passes a smoke test that the host can reach the device service and vice versa.
• androidTestUtil('androidx.test:orchestrator:...') (not androidTestImplementation) — ./gradlew :app:dependencies --configuration androidTestUtil shows the orchestrator.
• trap cleanup EXIT is installed in the top-level CI script.
• All long-running adb commands are wrapped with timeout.
• CI run completes 50 iterations without a "leaked port forward" or "previous run's animation scale" flake.
• Status-code parsing treats -3 and -4 as skips, not failures.
• Per-device artefact directories are namespaced by serial; CI artefact archive shows them in the expected layout.
• Retry helper bounces adb kill-server; adb start-server between attempts on transient errors.

References

• developer.android.com/tools/adb — forward / reverse syntax, transport, server.
• developer.android.com/studio/test/command-line — am instrument -w -r, -e numShards / -e shardIndex, AndroidJUnitRunner arguments, output dirs.
• developer.android.com/training/testing/instrumented-tests — Test Orchestrator overview.
• developer.android.com/reference/android/app/Instrumentation — REPORT_VALUE_RESULT_* constants (OK = 0, START = 1, ERROR = -1, FAILURE = -2).
• firebase.google.com/docs/test-lab/android/command-line — Firebase Test Lab CI exit codes (0 pass, 10 test failure, 15 indeterminate, 18 incompatible, 20 infra error).
• Research note tasks/research/A3-adb-observability-automation.md — full bash idioms, exit-code propagation matrix, port-forward token grammar, parallel fan-out, retries, capture-on-failure, Test Orchestrator wiring.
• Research note tasks/research/A2-adb-shell-commands.md — am instrument arguments, status codes, AndroidJUnitRunner -3/-4 extensions.
• Sibling skill: ../../architecture/understanding-adb-architecture/SKILL.md — server / daemon / adb kill-server.
• Sibling skill: ../../devices/connecting-to-devices/SKILL.md — device states, wait-for-device.
• Sibling skill: ../../devices/connecting-over-wifi/SKILL.md — adb pair / adb connect for headless CI.
• Sibling skill: ../../apps/installing-and-managing-apps/SKILL.md — pm install / pm uninstall / pm clear.
• Sibling skill: ../../tests/running-instrumented-tests-via-adb/SKILL.md — am instrument -w -r deep dive.
• Sibling skill: ../../control/injecting-input-and-state/SKILL.md — hermetic preamble (animations to 0, pm clear, am force-stop).
• Sibling skill: ../../capture/capturing-screenshots-and-screenrecord/SKILL.md — screencap / screenrecord capture-on-failure.
• Sibling skill: ../../observability/extracting-logs-with-logcat/SKILL.md — logcat -d capture-on-failure.
• Sibling skill: ../../transfer/extracting-test-artifacts/SKILL.md — pull / push / run-as / TestStorage.
• Cross-set: ../../../instrumentation/runner/running-instrumented-tests-with-androidjunit4/SKILL.md — runner internals.
• Cross-set: ../../../instrumentation/scenarios/launching-activities-with-activityscenario/SKILL.md — Activity scenarios under instrumentation.
• Cross-set: ../../../fundamentals/strategies/applying-testing-strategies/SKILL.md — when to invest in CI orchestration vs simpler tooling.