android-emulator

Android emulator (Flutter)

A bash helper that wraps adb and the qemu emulator console so an AI agent can see (screenshots, accessibility tree) and act on (tap, long-press, swipe, pinch) a Flutter app running on an Android emulator. Computer-use APIs cannot reach the emulator window (qemu has no macOS app bundle), and adb shell input is single-touch only — this script bridges both gaps.

When to use this

The user wants to launch, debug, smoke-test, QA, or screenshot a Flutter app on Android. Reach for this skill before resorting to coordinate-guessing, manual screenshotting via DevTools, or asking the user to run flutter run themselves.

Setup

The script assumes a working Android SDK install (with adb and emulator on $PATH or in ~/Library/Android/sdk/ / ~/Android/Sdk/), at least one AVD created, and Flutter (or fvm) installed. macOS-only utility: sips (used to resample screenshots — ships with macOS). On Linux, swap in convert from ImageMagick if porting.

The script is at scripts/emu.sh relative to this skill. Once the skill is wired into your project (see README.md), agents call it with that relative path.

Quick start (cold boot to interactive)

scripts/emu.sh boot        # start an AVD (cold boot), idempotent
scripts/emu.sh run         # flutter build & install in background (~30–60s first time)
scripts/emu.sh wait-run    # blocks until attached or errors (180s timeout)
scripts/emu.sh health      # sanity check
scripts/emu.sh ui-list     # see what's on screen — start here, not screenshot

run auto-detects the project root (walks up to find pubspec.yaml) and uses fvm flutter if the project pins it. Always pair run with wait-run instead of sleep — the daemon's stdin is gone, so timing is the only signal that the build finished.

Commands

Emulator lifecycle

Command	What it does
boot	Start the AVD in cold boot (-no-snapshot-load) and block until sys.boot_completed. Idempotent — if the device is already online, returns immediately. AVD defaults to the first one listed by emulator -list-avds; pin with ANDROID_EMU_AVD.
exit	Shut down the running emulator via the qemu console (kill).
health	One-shot status: connection, AVD name, resolution, foreground app, detected project root, detected package, flutter log size. Run this first to orient. Exits non-zero if the device isn't connected.
devices / size / foreground	Raw adb devices / wm size / focused activity.
app-running	Exit 0 if the target package is the focused app, 1 otherwise. Use in scripts.

App control

Command	What it does
launch	monkey-launch the already-installed APK (no rebuild).
stop	am force-stop the app.
run	Start flutter run -d <device> in background. Log: /tmp/android-emu-flutter-<id>.log (per-invocation, keyed on ANDROID_EMU_TMP_ID). Writes the daemon pid to <log>.pid. Uses fvm flutter when .fvm/ is present.
wait-run	Block until the log shows Flutter run key commands. or a build failure. 180s timeout (ANDROID_EMU_WAIT_SECS). Always pair run with wait-run — not sleep.
kill-run	Kill the flutter daemon (via <log>.pid, falls back to pkill -f flutter_tools.snapshot) and force-stop the app.
log [-f] [N]	Tail last N lines (default 50) of the per-invocation log; with -f, follow live. Filter via grep, e.g. log 500 | grep -E 'I/flutter.*\[(W|S)\]' for warnings + severe (logcat wraps every print() with I/flutter ( PID): , so the level tag isn't at line start).

Input — coordinate-based (all coords in screenshot pixels, 360-wide space)

Command	What it does
screenshot	Capture screen → /tmp/android-emu-shot-<id>.jpg (360px wide JPEG q85). The exact path is printed on stdout — Read that path to see the current state. Per-invocation paths keep concurrent callers from clobbering each other.
tap X Y	Single tap.
hold X Y [MS]	Long-press (default 800ms).
swipe X1 Y1 X2 Y2 [MS]	Swipe (default 300ms). Shorter ms = fling.
pinch out|in [CX CY SG EG]	Two-finger pinch. out zooms in. Defaults: center (180,367), start gap 67, end gap 333. Pinch goes through the qemu console — adb input cannot do multi-touch.

Input — label-based (preferred — no coordinate guessing)

These read Android's accessibility tree via uiautomator dump and act on the node whose text, content-desc, resource-id, or hint matches the given LABEL. Exact match wins over substring fallback. hint covers empty TextFields (Flutter's InputDecoration.labelText surfaces there).

Command	What it does
ui-list	Human-readable list of on-screen labelled nodes: screenshot-space center, tap/hold/scroll flags, label. Start here to discover what's addressable.
ui-find LABEL	Print device-px bounds and screenshot-px center for the first match. Debugging aid.
ui-dump	Raw uiautomator XML. Useful when ui-list hides the node you want (e.g. an unlabelled parent).
tap-label LABEL	Tap the center of the first matching node.
hold-label LABEL [MS]	Long-press the center of the first matching node.

Coords are in screenshot space. The cx cy columns in ui-list use the same 360-wide frame as the screenshot JPEG and tap X Y, so you can cross-reference the two without rescaling.

Example ui-list output:

 cx   cy  flags          label
 27   70  tap            'Settings'
137  141  tap            'Get started'
 47  254  tap            'New project'
 45  760  tap            'Home\nTab 1 of 4'
135  760  tap            'Search\nTab 2 of 4'
225  760  tap            'Library\nTab 3 of 4'
315  760  tap            'Profile\nTab 4 of 4'

Then tap-label "Search" — substring match is enough; you do not have to include the \nTab 2 of 4 suffix.

Filtering logs by severity

If the app uses package:logging with [F]/[I]/[W]/[S] prefixes, pipe log through grep. Logcat wraps every print() with I/flutter ( PID): , so the level tag isn't at line start — match it after the wrapper:

scripts/emu.sh log 500 | grep -E 'I/flutter.*\[(W|S)\]'    # warnings + severe
scripts/emu.sh log -f  | grep --line-buffered -E 'I/flutter.*\[S\]'   # follow severe

Swap the regex for whatever scheme the app emits (e.g. \[(WARNING|SEVERE)\]).

Choosing screenshot vs ui-list

Screenshots are ~30–60 KB each and add up fast in the conversation context, while ui-list output is ~2 KB. Default to ui-list. If the labels on screen changed, you're on a new screen — that's what most navigation steps need to confirm.

Reach for screenshot only when:

1. Content is inherently visual — a CustomPainter, image/camera preview, color swatches, thumbnails — anything rendered as pixels rather than widgets.
2. Labels don't differentiate — same screen, state change that isn't reflected in the accessibility tree (e.g. a slider dragged to a new value, a toggled chip that re-uses the same text).
3. Debugging a tap-label failure — when a label match fails or taps the wrong thing, a screenshot is faster than reading ui-dump to figure out what's on screen.

Heuristic: after a tap, check ui-list first. Screenshot only if it doesn't answer your question.

Making widgets addressable (Flutter Semantics)

uiautomator dump reads Android's accessibility tree. In a Flutter app, that tree is empty unless the app's semantics tree is exposed. Two ways:

1. Enable semantics in debug builds — add this near the top of main():

   import 'package:flutter/foundation.dart';
   import 'package:flutter/semantics.dart';
   
   void main() {
     if (kDebugMode) {
       SemanticsBinding.instance.ensureSemantics();
     }
     runApp(const MyApp());
   }
   

   Material widgets (BottomNavigationBar, IconButton with tooltip:, TextButton, TextField, etc.) emit Semantics automatically — no per-widget wrapping needed for the nav bar, app-bar buttons, form fields, etc.

2. Enable an accessibility service on the AVD — TalkBack or a similar service forces the platform to materialize the tree. Heavier-handed; option (1) is cleaner.

If ui-list prints (no labelled nodes found — is Flutter semantics enabled?), neither is in effect.

For an icon-only custom widget that needs to be addressable:

Semantics(
  identifier: 'some_stable_id',   // optional, locale-invariant
  label: 'Human readable name',
  child: ...,
)

identifier is preferred for tests because it doesn't change with locale; label is what assistive tech (and tap-label) reads.

Gotchas

• adb shell input swipe X Y X Y MS is not a long-press. It sends DOWN/UP so close together that Flutter's LongPressGestureRecognizer treats it as a cancelled tap and never fires onLongPress. Use hold (or hold-label) — they send a real motionevent DOWN, sleep, motionevent UP pair.
• Pinch does not go through adb. adb shell input is single-touch only. Multi-touch via sendevent /dev/input/event1 requires either root (production AVDs deny adb root) or SELinux permissions the shell user lacks. The pinch command falls back to qemu's host-side console (localhost:5554, auth token at ~/.emulator_console_auth_token), which writes events directly into the virtual input device.
• Without semantics, the app is one opaque rectangle. ui-list on a release build (or a debug build that didn't call ensureSemantics()) returns nothing useful. Fix it in the app, not by guessing coordinates — see the section above.
• ui-list only shows nodes with a label. Unlabelled parents/wrappers won't appear; use ui-dump to see the raw XML if a node you expect is missing.
• Hot-restart vs full rebuild. run always launches a fresh flutter run. If you want hot-reload after a code change, send r\n to the daemon's stdin — but the script backgrounds it (so stdin is gone) and assumes you'll kill-run and run again. For tighter inner loops, run flutter run in your own terminal and use only the input/screenshot commands here.
• First-launch dialogs. Many apps show splash, onboarding, or upsell screens on first launch. Use ui-list to see what's blocking, then tap-label to dismiss (e.g. tap-label "Close", tap-label "Skip"). Don't rely on a fixed sleep — wait until ui-list shows the screen you expect.

Typical workflows

Cold start from nothing

scripts/emu.sh boot
scripts/emu.sh run
scripts/emu.sh wait-run
scripts/emu.sh health
scripts/emu.sh ui-list

Debug something the user is seeing

1. scripts/emu.sh ui-list — what's on screen now?
2. Reproduce the user's path: tap-label "…" for each step.
3. After the suspect action, ui-list again. If it looks right, screenshot only if the bug is visual.
4. scripts/emu.sh log 100 — tail the flutter log to catch exceptions/asserts.

Stop a debug session cleanly

scripts/emu.sh kill-run

Don't try TaskStop from inside the agent — the background task may already be reaped, and either way the app keeps running on the device until force-stopped.

Full teardown

scripts/emu.sh kill-run    # stop flutter + app
scripts/emu.sh exit        # shut down the emulator

Why screenshots are 360px-wide JPEGs

Full-resolution PNGs (1080×2424 on a Pixel-class AVD) are 600 KB–1.5 MB and balloon the conversation context. PNG resize alone barely helps — the encoder isn't aggressive. JPEG q85 at 360px wide gets to ~25–60 KB while staying legible for UI labels and small text. The 360-wide image is the canonical input space for tap/hold/swipe/pinch — the script handles device-side scaling itself, so the resolution is invariant from the caller's point of view. Override the JPEG quality with ANDROID_EMU_SHOT_QUALITY if labels are illegible (try 92) or you need smaller files (try 70).

Concurrent use

Per-invocation scratch paths (suffixed with ANDROID_EMU_TMP_ID, default $$) cover screenshots, UI dumps, and the flutter daemon log + pidfile, so two agents driving different emulator devices don't clobber each other. Pin ANDROID_EMU_DEVICE and ANDROID_EMU_TMP_ID across the full run → wait-run → log → kill-run chain. screenshot echoes its output path — read that, don't hardcode it.

One emulator still hosts only one flutter daemon, so two agents on the same device must coordinate (typically: one owns run/kill-run, both freely log/screenshot/ui-list). Shared state: the emulator input stream (simultaneous taps interleave) and the device-size cache.

Auto-detection

What	How
Project root	Walks up from $PWD looking for pubspec.yaml. Override with ANDROID_EMU_PROJECT_ROOT.
applicationId	Parsed from android/app/build.gradle or android/app/build.gradle.kts (first applicationId "…" line). Override with ANDROID_EMU_PKG.
AVD	First entry of emulator -list-avds if ANDROID_EMU_AVD isn't set. With multiple AVDs, pin one explicitly.
flutter CLI	fvm flutter when the project has a .fvm/ directory and fvm is on PATH; otherwise flutter. Override with ANDROID_EMU_FLUTTER_CMD.

scripts/emu.sh health prints the resolved values — run it first if anything seems off.

Environment overrides

Var	Default	Purpose
ANDROID_EMU_DEVICE	emulator-5554	adb device serial
ANDROID_EMU_AVD	first listed	AVD to boot
ANDROID_EMU_PKG	parsed from gradle	applicationId
ANDROID_EMU_PROJECT_ROOT	walked up from $PWD	Flutter project root
ANDROID_EMU_FLUTTER_CMD	flutter or fvm flutter	flutter command override
ANDROID_EMU_CONSOLE_PORT	5554	qemu console port
ANDROID_EMU_SHOT_QUALITY	85	JPEG quality (1–100)
ANDROID_EMU_WAIT_SECS	180	wait-run timeout
ANDROID_EMU_BOOT_SECS	180	boot timeout
ANDROID_EMU_TMP_ID	$$ (script PID)	scratch-file suffix; pin across calls when multiple agents share one emulator