esp32-iphone-agent
Let an AI model use a real iPhone with a $7 ESP32: AirPlay eyes, USB HID hands, any OpenAI-compatible vision model. No Mac, no developer account, no jailbreak.
What it does
Core capabilities at a glance
- AI Agent
- Airplay
- Automation
- Computer USE
- Esp32
- Esp32 S3
- Iphone
- USB HID
Deep dive
The full breakdown - performance, comparisons, and setup
esp32-iphone-agent
esp32-iphone-agent is an agent framework - Let an AI model use a real iPhone with a $7 ESP32: AirPlay eyes, USB HID hands, any OpenAI-compatible vision model. No Mac, no developer account, no jailbreak.
Overview
Let an AI model use a real iPhone with a $7 ESP32. Just the phone and the board: no computer, no jailbreak.
"Set an alarm for 8:00 AM that repeats every Monday and Tuesday, then save it": the agent leaves its own app, opens Clock, types the time into the wheel, ticks the days and saves. (Photo from v0.1.0; the task is the same.)
Left: the real phone. Right: what the agent sees through the board's AirPlay mirror. The second shot is the pointer calibration.
esp32-iphone-agent is open-source, written primarily in C, with 4,132 GitHub stars under the MIT license. The latest release is v0.1.0 (2026-10-14).
Key capabilities
From the project's documentation:
- Eyes — AirPlay to the board. The board opens its own Wi-Fi hotspot and appears in the phone's Screen Mirroring
- Hands — USB HID. The board is a standard USB mouse and keyboard. With AssistiveTouch on, the mouse moves an
- Brain — the app. The app decodes the stream, sends screenshots to any OpenAI-compatible vision model, and runs
- Closing the loop. The pointer is relative, so the app tracks it and re-homes it in a corner; before precise taps
- Review. Every run is kept on the phone: replay it step by step with the taps and swipes drawn on top.
- Any OpenAI-compatible vision endpoint works (vLLM, SGLang, llama.cpp, Ollama, cloud APIs). The app lists the
How it fits a local-AI stack
esp32-iphone-agent runs on your own hardware, so pair it with a model and a GPU sized to your needs. Use the VRAM calculator to pick a model that fits your card, and see what you can run for hardware guidance. Related agent frameworks in the directory:
Sources
- Source code & docs: onehumanbeing/esp32-iphone-agent
- Official website: https://handsy.tech?ref=github
Stats from GitHub, 2026-10-26.
Frequently asked
Quick answers to common questions
What is esp32-iphone-agent?
esp32-iphone-agent is a agent-framework tool for local AI workloads. Let an AI model use a real iPhone with a $7 ESP32: AirPlay eyes, USB HID hands, any OpenAI-compatible vision model. No Mac, no developer account, no jailbreak.
Is esp32-iphone-agent free and open source?
Yes, esp32-iphone-agent has 4,132 GitHub stars and is licensed under MIT. You can self-host it for free on macos, linux.
What platforms does esp32-iphone-agent support?
esp32-iphone-agent runs on macos, linux.
What hardware do I need for esp32-iphone-agent?
The hardware requirements depend on which models you run. Check our hardware directory for compatible GPUs and systems. esp32-iphone-agent has 4,132 GitHub stars and an active community.
Does esp32-iphone-agent support GPU acceleration?
esp32-iphone-agent supports GPU acceleration via CUDA, Metal, or Vulkan depending on your platform. For the best performance, pair it with an NVIDIA RTX 4090 or 5090.
What are the best alternatives to esp32-iphone-agent?
Popular alternatives include other agent-framework tools in our directory. Browse our full collection at /tool for comparisons, community reviews, and benchmark data to find the right fit for your workflow.
How much does esp32-iphone-agent cost?
esp32-iphone-agent is free-open-source. It is completely free and open source to self-host.
Pairs well with
Complementary tools, models, and hardware