Run automation
Observe and act
Two calls give an agent its loop: look at the screen, then act on it.
The loop
GET /v1/devices/{name}/observe?w=540
# → {"seq": 41, "image": {"w": 540, "format": "jpeg", "data": "…"}, "package": "…", "activity": "…",
# "ui": [{"text": "Checkout", "id": "…", "b": [60, 1560, 1020, 1680], "click": true}, …]}
POST /v1/devices/{name}/act {"op":"tap_text","text":"Checkout","seq":41,"settle":true}
# → {"ok": true, "seq": 42, "settled": true, "waited_ms": 830}Observe
observe always takes a fresh screenshot and returns up to 500 elements on the screen with their text, id and bounds. The agent sees exactly what is on the phone at that moment.
Act
act taps, swipes, types, presses keys and opens apps. It refuses with 409 stale if another observe or act happened after the seq you pass, so an agent never acts on an old picture. With settle it waits until the screen stops changing, up to 5 seconds.
Watch it live
Open the phone in the dashboard to see its screen live while the agent works, or put several phones side by side in Multi-view. Each step can also return a screenshot, and a webhook tells your server when a job is done.