Skip to content

Commit 4cfa34b

Browse files
authored
docs: reposition README around mobile app automation for AI agents (#1780)
* docs: reposition README around mobile app automation for AI agents Lead the README with the category (mobile app automation, testing, and verification for AI coding agents) and the three surfaces (CLI, MCP server, Node.js API) so humans and search engines can classify it, then keep verification and evidence as the differentiator. - Move per-platform transport/caveat sentences (HarmonyOS HDC/uitest, Vega VVD-only) out of the intro into "How it works" and add an inline guard comment plus an AGENTS.md rule so new platforms only add a name to the intro list. - Add MCP and Node.js quick starts next to the CLI walkthrough. - Add a works-with/proof line, a "What to ask your agent" prompt list, a product-ladder sentence, and two AEO-shaped FAQ entries. - Point the cloud/remote row at the remote proxy and device clouds docs. - Align npm, MCP registry, and docs-site descriptions with the same category phrasing. - Remove em dashes and evaluative filler per the humanizer skill. * docs: tighten README hero, link proof points, move install above capabilities * docs: add mobile-MCP FAQ distinction, sharpen hero support line, drop coming-soon * docs: make the build-on-top audience explicit in README * docs: trim README repetition and signposting * docs: close the session in finally in the README Node.js snippet
1 parent 856ff38 commit 4cfa34b

5 files changed

Lines changed: 101 additions & 34 deletions

File tree

AGENTS.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,5 +251,9 @@ connect errors, retry policy, or command typing, start in
251251
help/metadata, README or `website/docs/**` when user-facing, and a help-conformance bench case
252252
(`scripts/help-conformance-*.mjs`) when command-planning guidance changes.
253253
- State in the final summary whether docs/skills were updated, and why not if they weren't.
254+
- README altitude: the intro is a category line, the job pitch with the platform list, and the
255+
works-with/proof line — nothing else. Per-platform transports, tool names, vendor lists, and
256+
support caveats go in "How it works" and `website/docs/**`. A new platform adds its name to the
257+
intro list and its backend to "How it works", nothing more.
254258

255259
When guidance conflicts, Hard Rules win, then scope, then testing, then style.

README.md

Lines changed: 94 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<a href="https://www.callstack.com/open-source?utm_campaign=generic&utm_source=github&utm_medium=referral&utm_content=agent-device" align="center">
22
<picture>
3-
<img alt="agent-device: device automation CLI for AI agents" src="website/docs/public/agent-device-banner.jpg">
3+
<img alt="agent-device: mobile app automation and verification for AI coding agents" src="website/docs/public/agent-device-banner.jpg">
44
</picture>
55
</a>
66

@@ -13,13 +13,13 @@
1313
[![License: MIT](https://img.shields.io/badge/license-MIT-black.svg)](LICENSE)
1414
[![Glama MCP server](https://glama.ai/mcp/servers/callstack/agent-device/badges/score.svg)](https://glama.ai/mcp/servers/callstack/agent-device)
1515

16-
Let your coding agent verify its changes in the running app.
16+
**Mobile app automation and verification for AI coding agents.** Give coding agents a live app feedback loop through a CLI, built-in MCP server, or typed Node.js API.
1717

18-
`agent-device` lets coding agents inspect, control, and verify apps on iOS, Android, HarmonyOS, tvOS, Android TV, Amazon Vega OS TV through the Vega Virtual Device (VVD), web, macOS, and Linux. Agents can read token-efficient accessibility snapshots where supported, find elements by ref or selector, run device actions, and save evidence for review. HarmonyOS support uses HDC and ArkUI `uitest` on connected devices and DevEco emulators; use `capabilities --platform harmonyos` to inspect its evidence-backed command subset. Initial Vega OS support is VVD-only and covers discovery, app lifecycle, and complete TV-remote control; physical Fire TV, capture, and selector backends remain unsupported.
18+
<!-- Intro rule: category line, job pitch with the platform list, works-with/proof line, nothing else. Per-platform transports, tool names, vendor lists, and support caveats belong in "How it works" and website/docs, never here. Every name in the proof line must link to public evidence. -->
1919

20-
Your coding agent or QA tool reads each result and chooses the next command. `agent-device` runs the command and saves evidence when asked.
20+
Let your coding agent verify its changes in the running app. `agent-device` lets agents inspect, control, debug, and verify apps on iOS, Android, and HarmonyOS (simulators, emulators, and physical devices), plus tvOS, Android TV, Amazon Vega OS TV (Vega Virtual Device), web, macOS, and Linux. Agents read token-efficient accessibility snapshots instead of reasoning over screenshots alone, act through refs and selectors, and save evidence for review. It also coordinates device access across parallel agent worktrees and connects to remote device clouds.
2121

22-
`agent-device` uses the inspect-act-verify process from Vercel's [agent-browser](https://github.com/vercel-labs/agent-browser) for mobile, TV, and desktop apps. Basic `--platform web` support runs `agent-browser` in the same session and replay system.
22+
Works with Claude Code, Codex, Cursor, Windsurf, Cline, Goose, and any agent that can run a CLI or connect over MCP, or as the runtime under agents you build with the [AI SDK](https://oss.callstack.com/agent-device/docs/ai-sdk) or [Eve](https://oss.callstack.com/agent-device/docs/eve). Developers at [Expensify](https://www.callstack.com/blog/how-expensify-uses-agent-device-for-mobile-bug-evidence-and-profiling), [Shopify](https://x.com/mustafa01ali/status/2036577353178943826), and [others](#who-uses-agent-device) use it to verify their apps.
2323

2424
## Quick start
2525

@@ -31,7 +31,9 @@ agent-device doctor
3131
agent-device help workflow
3232
```
3333

34-
Run `agent-device doctor` yourself before handing the CLI to an agent. The installed CLI help defines current behavior. `agent-device help workflow` links to guides for debugging, replay, React Native profiling, and other tasks.
34+
Run `doctor` yourself before handing the CLI to an agent; `help workflow` links to the guides for debugging, replay, and profiling, and the installed help always matches the installed version.
35+
36+
### Drive an app from the CLI
3537

3638
Add a contact in the built-in iOS Contacts app:
3739

@@ -59,13 +61,46 @@ agent-device screenshot ./contact-form.png
5961
agent-device close
6062
```
6163

62-
Use refs only from the latest output. Do not assume an earlier `@eN` still identifies the same element. After a command with `--settle`, use the refs in its diff. Take another snapshot only if the diff omits what you need.
64+
Refs are only valid from the latest output: after a `--settle` command, use the refs in its diff, and take a new snapshot only if the diff omits what you need. Snapshots come from the app's accessibility tree, so clear labels, roles, and test IDs make agent runs more reliable; use screenshots and video as evidence or when accessibility data is poor.
6365

64-
`--settle` works the same way on `scroll` and `back`, so scroll-then-observe and back-then-observe are one call too.
66+
![agent-device demo showing Codex using agent-device to create a new contact in the iOS Contacts app from a simple prompt](./website/docs/public/agent-device-contacts.gif)
6567

66-
Snapshots use the app's accessibility tree. Clear labels, roles, and test IDs make agent runs more reliable. Use screenshots and videos as evidence or when accessibility data is poor. Use refs and selectors for actions and assertions when you can.
68+
### Add MCP tools to your agent
6769

68-
![agent-device demo showing Codex using agent-device to create a new contact in the iOS Contacts app from a simple prompt](./website/docs/public/agent-device-contacts.gif)
70+
`agent-device mcp` starts the official stdio MCP server, exposing the installed commands as structured tools over the same execution path as the CLI:
71+
72+
```json
73+
{
74+
"mcpServers": {
75+
"agent-device": {
76+
"command": "agent-device",
77+
"args": ["mcp"]
78+
}
79+
}
80+
}
81+
```
82+
83+
See [AI Agent Setup](https://oss.callstack.com/agent-device/docs/agent-setup) for per-client setup and when to prefer plain CLI over MCP.
84+
85+
### Script it from Node.js
86+
87+
`createAgentDeviceClient()` gives Node.js code typed access to the same commands, as model tools in your own agent or from orchestration code:
88+
89+
```ts
90+
import { createAgentDeviceClient } from 'agent-device';
91+
92+
const client = createAgentDeviceClient({ session: 'qa-run' });
93+
try {
94+
await client.apps.open({ app: 'com.apple.Preferences', platform: 'ios' });
95+
const snapshot = await client.capture.snapshot({ interactiveOnly: true });
96+
const button = snapshot.nodes.find((node) => node.role === 'button');
97+
if (button) await client.interactions.press({ ref: button.ref });
98+
} finally {
99+
await client.sessions.close();
100+
}
101+
```
102+
103+
See the [Node.js API](https://oss.callstack.com/agent-device/docs/client-api), the [runnable examples](https://github.com/callstack/agent-device/tree/main/examples/sdk), and the [AI SDK](https://oss.callstack.com/agent-device/docs/ai-sdk) and [Eve](https://oss.callstack.com/agent-device/docs/eve) integration guides.
69104

70105
## What agents can do
71106

@@ -78,58 +113,86 @@ See [Commands](https://oss.callstack.com/agent-device/docs/commands) for the com
78113

79114
![Diagram of the agentic development loop: humans assign tasks, agents write and review code, agent-device verifies mobile apps, pull requests receive evidence, and bugs or performance issues lead to fixes](./website/docs/public/agentic-development-loop.svg)
80115

81-
## Next steps
82-
83-
- **Set up your agent**: run the CLI from Cursor, Codex, Claude Code, Windsurf, or another agent terminal. See [AI Agent Setup](https://oss.callstack.com/agent-device/docs/agent-setup) for skills, rules, MCP tools, and setup for each client.
84-
- **Try the sample app**: clone the repo and run the bundled Expo test app. [Quick Start](https://oss.callstack.com/agent-device/docs/quick-start) covers a guided run with screenshots, replay, and performance data.
85-
- **Build repeatable tests**: use [Replay & E2E](https://oss.callstack.com/agent-device/docs/replay-e2e) to repeat tests. Use [Debugging & Profiling](https://oss.callstack.com/agent-device/docs/debugging-profiling) to find bugs.
116+
## What to ask your agent
86117

87-
## Articles and videos
118+
With the CLI installed, prompts like these work end to end:
88119

89-
### Articles
120+
- "Implement the onboarding screen, run it on the iOS simulator and Android emulator, and attach screenshots."
121+
- "Reproduce this crash and capture the logs that lead up to it."
122+
- "Check whether this change causes unnecessary React Native re-renders."
123+
- "Explore the checkout flow once, save it as a replay script, and run it in CI."
124+
- "Verify this pull request on a physical device and attach reviewable evidence."
90125

91-
- [Build an AI QA agent for Expo apps with EAS Workflows](https://expo.dev/blog/build-an-ai-qa-agent-for-expo-apps-with-eas-workflows-in-minutes-today)
92-
- [Agent Device: iOS & Android automation for AI agents](https://www.callstack.com/blog/agent-device-ai-native-mobile-automation-for-ios-android)
93-
- [Building mobile QA agents with Vercel Eve](https://www.callstack.com/blog/building-reviewable-mobile-qa-agents-with-vercel-eve)
94-
- [How we optimized Agent Device for mobile app automation](https://www.callstack.com/blog/how-we-optimized-agent-device-for-mobile-app-automation)
95-
96-
### Videos
126+
## Next steps
97127

98-
- [Verifying mobile apps with agent-device](https://youtu.be/kZDU-k5r9kE)
99-
- [Using agent-device in an AI coding workflow](https://youtu.be/dfVG_aNPkW4)
100-
- [Cloud agents that test mobile apps on real devices](https://youtu.be/r5P0detC4bs?is=_KB6SZbLFRB1au_z)
128+
- [AI Agent Setup](https://oss.callstack.com/agent-device/docs/agent-setup): skills, project rules, and per-client setup for Cursor, Codex, Claude Code, Windsurf, and others.
129+
- [Quick Start](https://oss.callstack.com/agent-device/docs/quick-start): a guided run on the bundled Expo test app with screenshots, replay, and performance data.
130+
- [Replay & E2E](https://oss.callstack.com/agent-device/docs/replay-e2e) and [Debugging & Profiling](https://oss.callstack.com/agent-device/docs/debugging-profiling): repeatable tests and bug hunting.
101131

102132
## Where to run agent-device
103133

134+
The same session and evidence model works at every step: the agent explores the app, captures evidence, saves a replay, runs it in CI, and moves onto remote devices.
135+
104136
| Path | Best for | Start with |
105137
| --- | --- | --- |
106138
| Local | Trying commands and debugging apps on simulators, emulators, physical devices, macOS, and Linux. | Follow the Quick Start. |
107-
| CI/CD | Automated pull request and merge validation with replay scripts and captured artifacts. | Try the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml). GitHub Actions template coming soon. |
108-
| Cloud / remote | Linux runners, managed devices, and remote jobs. | Use [Agent Device Cloud](https://agent-device.dev/cloud), set a remote profile with [Commands](https://oss.callstack.com/agent-device/docs/commands), or [contact Callstack](mailto:hello@callstack.com) for team QA. |
139+
| CI/CD | Automated pull request and merge validation with replay scripts and captured artifacts. | Try the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml). |
140+
| Cloud / remote | Linux runners, managed devices, and remote jobs. | Set up a [remote proxy](https://oss.callstack.com/agent-device/docs/remote-proxy), connect a [device cloud](https://oss.callstack.com/agent-device/docs/device-clouds) (BrowserStack, AWS Device Farm, Limrun), or [contact Callstack](mailto:hello@callstack.com) for team QA. |
109141

110142
## How it works
111143

112144
`agent-device` keeps device state in sessions. It sends commands to XCTest on iOS and tvOS, ADB and the snapshot helper on Android, HDC and ArkUI `uitest` on HarmonyOS, Vega CLI/VDA on the Vega Virtual Device, a local helper on macOS, and AT-SPI on Linux.
113145

114-
Node.js apps can use the typed client or public subpaths. `agent-device/android-adb` provides the Android ADB provider interface, helpers for logcat, the clipboard, the keyboard, and apps, and port reverse management.
146+
Support depth varies by target. Newer backends such as HarmonyOS and Vega OS cover a subset of commands; run `agent-device capabilities --platform <platform>` to see what a target supports.
147+
148+
Sessions are scoped to the caller's git worktree, and host-local device claims stop parallel agents from taking over each other's simulators and emulators. The same commands drive hosted devices on [BrowserStack, AWS Device Farm, and Limrun](https://oss.callstack.com/agent-device/docs/device-clouds).
149+
150+
`agent-device` uses the inspect-act-verify process from Vercel's [agent-browser](https://github.com/vercel-labs/agent-browser) for mobile, TV, and desktop apps. Basic `--platform web` support runs `agent-browser` in the same session and replay system.
115151

116152
## FAQ
117153

118154
### What is agent-device?
119155

120-
`agent-device` is a command-line tool that lets coding agents inspect, control, and verify apps and save evidence for review. It supports iOS, Android, HarmonyOS, TV, web, macOS, and Linux.
156+
`agent-device` is a command-line tool and MCP server that lets AI coding agents inspect, control, and verify mobile apps and save evidence for review. It supports iOS, Android, HarmonyOS, TV, web, macOS, and Linux.
157+
158+
### Is there an MCP server for mobile app automation?
159+
160+
Yes. `agent-device mcp` starts the official stdio MCP server. The Quick start above has the client config, and [AI Agent Setup](https://oss.callstack.com/agent-device/docs/agent-setup) covers per-client details.
121161

122162
### Does it work with React Native, Expo, Flutter, and native apps?
123163

124164
Yes. `agent-device` supports native iOS and Android apps, plus React Native, Expo, and Flutter apps on supported targets. The commands and evidence vary by target.
125165

166+
### How is it different from mobile MCP servers?
167+
168+
The MCP server is one entry point to the same runtime used by the CLI and typed Node.js API. Sessions, device ownership, selectors, evidence, replay, CI workflows, and cloud routing stay consistent across all three.
169+
170+
### Can I build my own agent or QA product on agent-device?
171+
172+
Yes. The typed Node.js client is a public surface over that same runtime, so an agent you build inherits everything above. Start from the [Node.js API](https://oss.callstack.com/agent-device/docs/client-api), [AI SDK](https://oss.callstack.com/agent-device/docs/ai-sdk), or [Eve](https://oss.callstack.com/agent-device/docs/eve) guides.
173+
126174
### How is it different from Appium, Detox, or Maestro?
127175

128176
With `agent-device`, an agent reads app state and chooses each command at run time. Teams use Appium, Detox, and Maestro to write and maintain test suites. `agent-device` can complement them by saving its runs as `.ad` scripts or exporting them as strict Maestro YAML.
129177

130178
### Can agent-device run in CI?
131179

132-
Yes. Record a run as an `.ad` script, replay it locally or in CI, and save screenshots, logs, and other artifacts for review. See [Replay & E2E](https://oss.callstack.com/agent-device/docs/replay-e2e) or start with the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml).
180+
Yes. Record a run as an `.ad` script, replay it in CI, and keep the screenshots and logs as artifacts; the [EAS workflow template](https://github.com/callstackincubator/eas-agent-device/blob/main/.eas/workflows/agent-qa-mobile.yml) is a working example.
181+
182+
## Articles and videos
183+
184+
### Articles
185+
186+
- [Build an AI QA agent for Expo apps with EAS Workflows](https://expo.dev/blog/build-an-ai-qa-agent-for-expo-apps-with-eas-workflows-in-minutes-today)
187+
- [Agent Device: iOS & Android automation for AI agents](https://www.callstack.com/blog/agent-device-ai-native-mobile-automation-for-ios-android)
188+
- [Building mobile QA agents with Vercel Eve](https://www.callstack.com/blog/building-reviewable-mobile-qa-agents-with-vercel-eve)
189+
- [How we optimized Agent Device for mobile app automation](https://www.callstack.com/blog/how-we-optimized-agent-device-for-mobile-app-automation)
190+
191+
### Videos
192+
193+
- [Verifying mobile apps with agent-device](https://youtu.be/kZDU-k5r9kE)
194+
- [Using agent-device in an AI coding workflow](https://youtu.be/dfVG_aNPkW4)
195+
- [Cloud agents that test mobile apps on real devices](https://youtu.be/r5P0detC4bs?is=_KB6SZbLFRB1au_z)
133196

134197
## Who uses agent-device?
135198

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "agent-device",
33
"version": "0.20.8",
4-
"description": "Agent-native CLI for AI app automation across iOS, Android, tvOS, Android TV, macOS, Linux, and web.",
4+
"description": "Mobile app automation and verification for AI coding agents. CLI, MCP server, and typed Node.js API for iOS, Android, HarmonyOS, TV, web, macOS, and Linux.",
55
"mcpName": "io.github.callstack/agent-device",
66
"license": "MIT",
77
"author": "Callstack",

server.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
33
"name": "io.github.callstack/agent-device",
44
"title": "agent-device",
5-
"description": "Let AI agents inspect, control, and debug real iOS, Android, desktop, and TV apps",
5+
"description": "MCP server for mobile app automation: verify, control, and debug iOS, Android, TV, and desktop apps",
66
"repository": {
77
"url": "https://github.com/callstack/agent-device",
88
"source": "github"

website/rspress.config.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ export default withCallstackPreset(
77
docs: {
88
title: 'agent-device',
99
description:
10-
'Agent-native CLI for AI mobile testing, React Native QA, Expo app verification, simulator and emulator automation, app observability, and replayable device workflows.',
10+
'Mobile app automation and verification for AI coding agents through a CLI, MCP server, and typed Node.js API.',
1111
editUrl: 'https://github.com/callstack/agent-device/edit/main/website',
1212
rootUrl: 'https://oss.callstack.com/agent-device',
1313
rootDir: 'docs',

0 commit comments

Comments
 (0)