---

title: "agent-device Mobile Device Control Pipeline for AI Agents" slug: "agent-device-mobile-testing-pipeline-2026" workflow_id: "agent-device-mobile-testing-pipeline-2026" primary_keyword: "agent-device mobile testing" category: "Developer Tools" difficulty: "Intermediate" tools_required: ["agent-device (callstack, MIT, July 2026)", "Claude Code", "Codex CLI", "iOS Simulator", "Android Emulator"] setup_time: 20 hours_saved_weekly: "10-15" meta_description: "agent-device CLI gives AI agents direct control over iOS and Android devices for automated mobile testing. Complete guide: real device vs simulator, Claude Code integration, CI/CD setup, and honest limitations." author_name: "Deepak Bagada" author_title: "CEO at SaaSNext" author_bio: "Deepak Bagada is the CEO of SaaSNext and leads AI agent architecture at dailyaiworld.com. He has built mobile testing automation pipelines using AI agents across iOS and Android platforms." author_credentials: "Built AI-powered mobile testing pipelines for production apps" author_url: "https://www.linkedin.com/in/deepakbagada" author_image: "https://dailyaiworld.com/authors/deepak-bagada.jpg"

agent-device Mobile Device Control Pipeline for AI Agents

TL;DR: agent-device (by Callstack, MIT license, July 2026) is a CLI tool that gives AI coding agents — Claude Code, Codex CLI, Gemini CLI — direct programmatic control over iOS simulators, Android emulators, and real physical devices. Screenshot, tap, scroll, type, assert UI elements. This guide walks through a complete pipeline from installation to CI/CD, with honest trade-offs for each approach.

---

1. What is agent-device?

agent-device is an open-source CLI tool released by Callstack under the MIT license in July 2026. It exposes a set of shell-commandable operations that let AI agents interact with mobile devices the same way they interact with files and terminals — through stdin/stdout.

The core idea is simple: bridge the gap between an LLM's text-in/text-out interface and the visual-tactile world of mobile UIs. Instead of Screenpipe-style screen recording or Appium's Selenium-like server model, agent-device operates as a lightweight command-line utility that an AI agent invokes directly. Each command returns structured output (JSON or plaintext) that the agent can parse and act on.

Why this matters for AI agents: LLMs are great at reading text and generating code, but they are blind to visual UI state on mobile screens. agent-device gives them synthetic vision (screenshots) and synthetic fingers (tap/scroll/type) so they can test and interact with mobile apps autonomously.

---

2. Why Mobile Device Control Matters for AI Agents

The mobile testing landscape in 2026 is fractured. Traditional frameworks like Appium, Detox, and XCTest/Espresso require engineers to write test scripts in advance. AI coding agents can write those scripts, but they cannot see what happens when the script runs. This creates a feedback loop problem:

• Agent writes test → test fails → agent sees only log output → agent cannot visually verify UI state → blind guesswork.

agent-device closes this loop. The agent can:

1. Launch the app on a simulator/emulator.
2. Take a screenshot.
3. Analyze the screenshot (via vision-capable LLM or pixel comparison).
4. Tap a button identified by coordinates or label.
5. Screenshot again to confirm the navigation happened.
6. Assert that an element is present or absent.
7. Scroll and repeat.

This visual feedback loop transforms mobile testing from "write and pray" into an interactive, agent-driven process.

---

3. Real Device vs Simulator/Emulator – When to Use What

| Criterion | Simulator (iOS) | Emulator (Android) | Real Device | |---|---|---|---| | Speed | Fastest | Fast | Slow (USB/network) | | Hardware sensors | Limited | Limited | Full (camera, GPS, biometrics) | | CI/CD friendliness | Excellent | Excellent | Poor (needs device farm) | | Push notifications | Simulated | Simulated | Real | | App Store / Play Store | Not available | Limited | Full | | agent-device support | Full | Full | Supported (USB/ADB) | | Cost | Free (macOS) | Free | Device cost |

Recommendation: Use simulators/emulators for 90% of testing. Reserve real devices for camera, biometrics (Face ID / fingerprint), push notification flows, and production-only SDK behaviors.

---

4. Installation & Setup

Prerequisites

• macOS (required for iOS simulator — Xcode 16+)
• Node.js 20+ or Bun 1.2+
• Xcode 16+ (for iOS)
• Android Studio / command-line tools (for Android)
• Ruby 3.3+ (for Fastlane, optional)

Install agent-device

npm install -g @callstack/agent-device
# or
bun install -g @callstack/agent-device

Verify installation

agent-device --version
# => 1.2.0 (or later)

agent-device doctor
# Checks Xcode, Android SDK, ADB, and simulator availability

Quick start — boot a simulator and take a screenshot

# List available devices
agent-device devices

# Boot an iOS simulator
agent-device boot --platform ios --device "iPhone 16 Pro"

# Take screenshot
agent-device screenshot --path ./screenshots/initial.png

# Install and launch an app
agent-device install --path ./build/MyApp.app
agent-device launch --bundle-id com.example.myapp

That's it. Five commands and your AI agent has a working mobile device under its control.

---

5. Core Commands: Screenshot, Tap, Scroll, Type

These four commands form the foundation of any agent-driven mobile interaction.

Screenshot

agent-device screenshot --format png --path output.png
# Returns: path, width, height, timestamp

The screenshot command captures the current device display and saves it to disk. When integrated with Claude Code or Codex CLI, the agent can read the image path and pass it to a vision model for analysis.

Tap

agent-device tap --x 200 --y 400
# Returns: { success: true, x: 200, y: 400 }

# Tap by element label (iOS accessibility identifier)
agent-device tap --label "LoginButton"

The tap command supports both absolute coordinates and accessibility-label-based targeting. Label-based tapping is more resilient to layout changes and should be preferred when accessibility identifiers are available.

Scroll

agent-device scroll --direction down --distance 300
# Returns: { success: true, contentOffset: { x: 0, y: 300 } }

# Scroll until element visible
agent-device scroll --until-visible --label "SubmitButton"

The scroll command supports directional scrolling and can auto-scroll until a specific element comes into view. This is critical for infinite-scroll lists and long forms.

Type

agent-device type --text "hello@example.com"
# Returns: { success: true, text: "hello@example.com" }

# Type with key events (for password fields)
agent-device type --text "mypassword" --key-event true

The type command simulates keyboard input. The --key-event flag uses hardware key events instead of pasting text, which is necessary for secure fields that block paste.

---

6. Assertion Commands: Verifying UI State

Assertions are what make agent-driven testing reliable. Without them, the agent is just blindly poking at the screen.

# Assert an element exists (by label, text, or accessibility ID)
agent-device assert --exists --label "WelcomeScreen"
# => { passed: true } or { passed: false, message: "Element not found" }

# Assert an element is visible (on-screen and not obscured)
agent-device assert --visible --label "ContinueButton"

# Assert text content
agent-device assert --text-contains "Welcome back" --element-id "titleLabel"

# Assert count of elements
agent-device assert --count --label "ListItem" --equals 5

# Visual assertion (compare screenshot to baseline)
agent-device assert --visual --baseline ./baselines/home_screen.png --threshold 0.02

The visual assertion feature compares the current screenshot against a baseline using pixel-diff analysis with configurable thresholds. This catches visual regression bugs that text-based assertions miss.

---

7. iOS Simulator: Deep Dive

iOS simulator support is agent-device's strongest platform. The tool leverages Xcode's simctl under the hood but adds a consistent JSON interface.

Key iOS-specific capabilities

# Boot with specific iOS version
agent-device boot --platform ios --device "iPhone 16 Pro" --os "18.3"

# Trigger biometric authentication
agent-device biometric --type face-id --match true

# Simulate location
agent-device location --lat 37.7749 --lng -122.4194

# Simulate push notification
agent-device notification --payload ./push.apns

# Record video of the session
agent-device record --output session.mov --duration 30

Performance note

iOS simulators are significantly faster than Android emulators on Apple Silicon Macs. They run as native macOS processes using the same kernel, so boot times are under 10 seconds and screenshot capture takes <100ms. This makes iOS simulator the ideal target for agent-driven testing loops.

---

8. Android Emulator: Deep Dive

Android emulator support requires the Android SDK and a system image. agent-device wraps adb and emulator commands.

Key Android-specific capabilities

# Boot with hardware acceleration
agent-device boot --platform android --device "Pixel 9 Pro" --avd "Pixel_9_API_35"

# Simulate incoming call (for testing interruption flows)
agent-device telecom --call --number "+15551234567"

# Simulate SMS
agent-device telecom --sms --number "+15551234567" --message "Your code is 123456"

# Rotate device
agent-device rotate --orientation landscape

# Monkey testing (random UI events)
agent-device monkey --event-count 1000 --seed 42

Performance note

Android emulators on macOS require a Hypervisor.framework or QEMU-based system image. Cold boot takes 30-60 seconds. Screenshot capture takes ~300-500ms via ADB. For agent-driven workflows, keep the emulator snapshot alive between sessions or use Quickboot snapshots to reduce boot time to ~5 seconds.

---

9. Integrating with Claude Code (and Other AI Coding Agents)

This is where agent-device becomes truly powerful. The CLI is designed to be invoked by AI agents that can read stdout, parse JSON, and make decisions.

Claude Code integration (example prompt)

You are a mobile QA agent. You have access to agent-device CLI.

Task: Verify that the login flow works on iOS.

Steps:
1. Run `agent-device boot --platform ios --device "iPhone 16 Pro"`
2. Run `agent-device install --path ./build/MyApp.app`
3. Run `agent-device launch --bundle-id com.example.myapp`
4. Run `agent-device screenshot --path step1.png` and describe what you see
5. If you see a Login button, tap it with `agent-device tap --label "LoginButton"`
6. Type email with `agent-device type --text "test@example.com"`
7. ... and so on

Codex CLI integration

Codex CLI can run agent-device commands in its sandboxed environment and pipe results back for decision-making. The key advantage is that Codex's agent loop can iterate rapidly: screenshot → analyze → decide → act → repeat.

A practical agent loop (pseudocode)

import subprocess, json

def agent_device(cmd):
    result = subprocess.run(f"agent-device {cmd}", shell=True, capture_output=True, text=True)
    return json.loads(result.stdout)

# Agent loop
state = agent_device("screenshot --path /tmp/screen.png")
# LLM analyzes the image
decision = llm.analyze_screenshot("/tmp/screen.png")
if decision.action == "tap":
    agent_device(f"tap --x {decision.x} --y {decision.y}")
elif decision.action == "scroll":
    agent_device(f"scroll --direction {decision.direction}")

---

10. Building a Mobile Testing Pipeline

Let's assemble a complete pipeline that an AI agent can execute autonomously.

Pipeline: End-to-end login flow test

#!/bin/bash
# agent-device login flow pipeline

PLATFORM=${1:-ios}
DEVICE=${2:-"iPhone 16 Pro"}
APP_PATH=${3:-"./build/MyApp.app"}
BUNDLE_ID="com.example.myapp"

# Phase 1: Setup
echo "{\"phase\": \"setup\", \"platform\": \"$PLATFORM\"}"
agent-device boot --platform $PLATFORM --device "$DEVICE" || exit 1
agent-device install --path "$APP_PATH" || exit 1
agent-device launch --bundle-id "$BUNDLE_ID" || exit 1

# Phase 2: Navigate to login
agent-device tap --label "GetStartedButton"
agent-device screenshot --path /tmp/phase2.png
agent-device assert --visible --label "EmailTextField" || exit 2

# Phase 3: Enter credentials
agent-device type --text "user@example.com" --target-label "EmailTextField"
agent-device type --text "securePass123" --target-label "PasswordTextField"
agent-device tap --label "SignInButton"

# Phase 4: Verify success
sleep 2
agent-device screenshot --path /tmp/phase4.png
agent-device assert --visible --label "DashboardScreen" || exit 3

# Phase 5: Cleanup
agent-device assert --visual --baseline ./baselines/dashboard.png --threshold 0.05 || exit 4
agent-device shutdown

echo "{\"phase\": \"complete\", \"status\": \"passed\"}"

An AI agent can run this script, catch non-zero exit codes, retry on failure, and report results — all without human intervention.

---

11. CI/CD Integration

agent-device runs in CI/CD environments, but with important caveats.

GitHub Actions (macOS runner)

name: Mobile E2E Tests
on: [push]
jobs:
  test-ios:
    runs-on: macos-15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
      - run: npm install -g @callstack/agent-device
      - run: agent-device doctor
      - run: agent-device boot --platform ios --device "iPhone 16 Pro"
      - run: ./test-pipeline.sh ios
      - run: agent-device shutdown

Android on CI/CD

Android emulators in CI require KVM (Linux runners) or a macOS runner with Hypervisor.framework. GitHub's macos-15 runners support both iOS simulators and Android emulators, but boot time for Android can exceed 60 seconds. Consider using -no-window -no-audio -accel on flags.

# Android CI boot with snapshot
agent-device boot --platform android --avd "Pixel_9_API_35" --args "-no-window -no-audio -snapshot default-boot"

Docker considerations

agent-device does not run inside Docker containers on macOS because Docker for Mac does not support nested virtualization. For containerized CI, use Linux-based runners (e.g., self-hosted Linux with KVM) for Android, and macOS VM runners for iOS.

---

12. Real-World Use Cases

1. Visual regression testing

A Claude Code agent takes screenshots of every screen after each commit, compares them against baselines, and flags visual diffs >2%. This catches UI regressions before they reach QA.

2. Form validation testing

The agent fills out forms with edge-case data (too-long inputs, special characters, empty fields) and asserts correct validation error messages appear. This replaces dozens of manual test cases.

3. Deep-link and deeplink routing

The agent launches the app with a deep link URL, takes a screenshot, and asserts the correct screen is displayed. Testing all deeplinks for a marketing campaign takes minutes instead of hours.

4. Push notification flows

The agent simulates a push notification, taps on it, and verifies the app navigates to the correct destination. This flow is notoriously hard to automate with traditional frameworks.

5. Accessibility audit

The agent scrolls through every screen and asserts that all interactive elements have accessibility labels. Combined with a screenshot analysis by a vision LLM, this provides a basic accessibility audit.

---

13. Honest Limitations & Pain Points

agent-device is powerful but not a silver bullet. Here are the real limitations you'll encounter.

Screenshot-based analysis is slow

Every screenshot→analyze→decide loop takes 2-5 seconds per cycle. A 50-step test takes 2-4 minutes. This is fine for pipeline runs but too slow for real-time interactive use.

Vision model costs

If you're using GPT-4o or Claude Sonnet to analyze screenshots, vision token costs add up. A single test run with 30 screenshots can cost $0.50-$2.00 in API fees alone.

No native element tree access

Unlike Espresso or XCTest, agent-device does not expose the full view hierarchy. It uses accessibility labels for targeting, which means apps without proper accessibility identifiers are much harder to test. You'll fall back to coordinate-based tapping, which is fragile across device sizes.

Real device support is limited

agent-device supports real iOS devices via USB (using idb under the hood) and real Android devices via ADB, but device provisioning, trust certificates, and multi-device orchestration are not well-documented. Expect friction.

No built-in recording or playback

agent-device is intentionally low-level. There is no test recorder that generates commands as you tap. You or your AI agent must construct the command sequence programmatically.

Flakiness from animations

Animations and async operations cause race conditions. The agent takes a screenshot, but the button hasn't finished animating in yet. You need explicit waits (sleep or polling) between steps.

No WebView support

agent-device cannot inspect or interact with WebView content. If your app uses WebViews (e.g., OAuth flows, web checkout), you'll need a separate strategy for those screens.

---

14. The Future of AI-Driven Mobile Testing

agent-device is part of a broader shift: AI agents are becoming the primary test authors and executors. The trajectory is clear:

• 2025: AI agents generate unit tests and simple integration tests.
• 2026 (now): AI agents control devices directly via tools like agent-device, enabling visual end-to-end testing without pre-written test scripts.
• 2027: Expect agent-device or its successors to add real-time screen streaming to the LLM, element tree access, WebView inspection, and self-healing selectors that adjust to UI changes automatically.

Callstack's agent-device is MIT-licensed and community-driven. If you need a feature, the GitHub Issues page is active, and the maintainers have been responsive about adding Android emulator improvements, new biometric simulation modes, and CI/CD documentation.

Final verdict: agent-device is ready for production use today if you accept its limitations. Use it for visual regression pipelines, login/form flow automation, and deeplink testing on simulators/emulators. Avoid it for complex multi-device orchestration, WebView-heavy flows, or real-device farms until the ecosystem matures further.

---

JSON-LD Structured Data

{
  "@context": "https://schema.org",
  "@type": "TechArticle",
  "headline": "agent-device Mobile Device Control Pipeline for AI Agents",
  "description": "agent-device CLI gives AI agents direct control over iOS and Android devices for automated mobile testing. Complete guide: real device vs simulator, Claude Code integration, CI/CD setup, and honest limitations.",
  "author": {
    "@type": "Person",
    "name": "Deepak Bagada",
    "jobTitle": "CEO at SaaSNext",
    "url": "https://www.linkedin.com/in/deepakbagada",
    "image": "https://dailyaiworld.com/authors/deepak-bagada.jpg"
  },
  "datePublished": "2026-07-17",
  "dateModified": "2026-07-17",
  "publisher": {
    "@type": "Organization",
    "name": "DailyAIWorld",
    "url": "https://dailyaiworld.com"
  },
  "image": "https://dailyaiworld.com/og/agent-device-mobile-testing-pipeline-2026.png",
  "keywords": ["agent-device", "mobile testing", "AI agents", "Claude Code", "iOS simulator", "Android emulator", "mobile automation", "agent-device CLI"],
  "mainEntityOfPage": {
    "@type": "WebPage",
    "@id": "https://dailyaiworld.com/workflows/agent-device-mobile-testing-pipeline-2026"
  }
}

---

WORKFLOWS_DATA

{
  "workflows": [
    {
      "id": "agent-device-mobile-testing-pipeline-2026",
      "title": "agent-device Mobile Device Control Pipeline for AI Agents",
      "slug": "agent-device-mobile-testing-pipeline-2026",
      "emoji": "📱",
      "description": "Give AI agents direct CLI control over iOS simulators, Android emulators, and real devices for automated mobile testing. Screenshot, tap, scroll, type, and assert UI state — all from the command line.",
      "category": "Developer Tools",
      "difficulty": "Intermediate",
      "setup_time": 20,
      "hours_saved_weekly": "10-15",
      "tools_required": ["agent-device", "Claude Code", "Codex CLI", "iOS Simulator", "Android Emulator"],
      "primary_keyword": "agent-device mobile testing",
      "keywords": ["mobile testing", "iOS automation", "Android automation", "visual regression", "AI agent testing", "CLI testing", "device control"],
      "last_updated": "2026-07-17",
      "is_new": true,
      "popularity": 75,
      "workflow_steps": [
        {
          "step": 1,
          "title": "Install agent-device CLI",
          "description": "Install globally via npm or bun and verify with `agent-device doctor`",
          "command": "npm install -g @callstack/agent-device && agent-device doctor"
        },
        {
          "step": 2,
          "title": "Boot a device",
          "description": "Boot an iOS simulator or Android emulator with specified device type",
          "command": "agent-device boot --platform ios --device \"iPhone 16 Pro\""
        },
        {
          "step": 3,
          "title": "Install and launch app",
          "description": "Install the app binary and launch it on the booted device",
          "command": "agent-device install --path ./build/MyApp.app && agent-device launch --bundle-id com.example.myapp"
        },
        {
          "step": 4,
          "title": "Take screenshot for analysis",
          "description": "Capture the current device screen for visual analysis by the AI agent",
          "command": "agent-device screenshot --path /tmp/step4.png"
        },
        {
          "step": 5,
          "title": "Interact with UI elements",
          "description": "Tap, scroll, and type using accessibility labels or coordinates",
          "command": "agent-device tap --label \"LoginButton\" && agent-device type --text \"user@example.com\""
        },
        {
          "step": 6,
          "title": "Assert UI state",
          "description": "Verify elements exist, are visible, or match visual baselines",
          "command": "agent-device assert --visible --label \"DashboardScreen\""
        },
        {
          "step": 7,
          "title": "Clean up",
          "description": "Shut down the device after the test completes or on failure",
          "command": "agent-device shutdown"
        }
      ]
    }
  ]
}

---

BLOGS_DATA

{
  "blogs": [
    {
      "title": "agent-device Mobile Device Control Pipeline for AI Agents",
      "description": "agent-device CLI gives AI agents direct control over iOS and Android devices for automated mobile testing. Complete guide: real device vs simulator, Claude Code integration, CI/CD setup, and honest limitations.",
      "primary_keyword": "agent-device mobile testing",
      "slug": "agent-device-mobile-testing-pipeline-2026",
      "category": "Developer Tools",
      "difficulty": "Intermediate",
      "tools_required": ["agent-device (callstack, MIT, July 2026)", "Claude Code", "Codex CLI", "iOS Simulator", "Android Emulator"],
      "setup_time": 20,
      "hours_saved_weekly": "10-15",
      "meta_description": "agent-device CLI gives AI agents direct control over iOS and Android devices for automated mobile testing. Complete guide: real device vs simulator, Claude Code integration, CI/CD setup, and honest limitations.",
      "author_name": "Deepak Bagada",
      "author_title": "CEO at SaaSNext",
      "author_bio": "Deepak Bagada is the CEO of SaaSNext and leads AI agent architecture at dailyaiworld.com. He has built mobile testing automation pipelines using AI agents across iOS and Android platforms.",
      "author_credentials": "Built AI-powered mobile testing pipelines for production apps",
      "author_url": "https://www.linkedin.com/in/deepakbagada",
      "author_image": "https://dailyaiworld.com/authors/deepak-bagada.jpg",
      "content_sections": [
        "agent-device",
        "Why Mobile Device Control Matters for AI Agents",
        "Real Device vs Simulator/Emulator – When to Use What",
        "Installation & Setup",
        "Core Commands: Screenshot, Tap, Scroll, Type",
        "Assertion Commands: Verifying UI State",
        "iOS Simulator: Deep Dive",
        "Android Emulator: Deep Dive",
        "Integrating with Claude Code (and Other AI Coding Agents)",
        "Building a Mobile Testing Pipeline",
        "CI/CD Integration",
        "Real-World Use Cases",
        "Honest Limitations & Pain Points",
        "The Future of AI-Driven Mobile Testing"
      ]
    }
  ]
}