extend Agentation for use while developing mobile apps

[andresdefi]

I want to start this by saying I really love Agentation and how easy it makes making changes to specific components.

I have been trying to fix issues for mobile apps I am trying to build, but its difficult to specify the exact element to an AI agent, so I thought, can't we build a version of Agentation for this?

that's when i asked Claude Code to help me figure out if we could build it, and we started doing it. I have successfully made it work testing on a React Native app I am trying to build, but there's surely more left to be improved.
I would love for this to someday become a part of Agentation

What I built

agentation-mobile brings the same click-to-annotate workflow to mobile apps. You tap a UI element on your device or
simulator, and it identifies the component name, source file path, and line number, then syncs that as a structured
annotation to an MCP server so AI agents can act on it precisely.

It currently supports:

• React Native — via Hermes debugger / Chrome DevTools Protocol
• Flutter — via Dart VM Service
• Native iOS — via xcrun simctl + accessibility APIs
• Native Android — via ADB + UIAutomator

The architecture is a monorepo with packages for: core schemas, HTTP/WebSocket server, MCP server (18+ tools), CLI, web UI,
platform bridges, and in-app SDKs for each framework.

How it relates to Agentation

I tried to follow Agentation's philosophy wherever I could:

• Annotations use the same intent types (fix, change, question, approve)
• Output is structured markdown, just like Agentation
• MCP integration follows the same pattern as agentation-mcp
• The web UI for reviewing annotations has a similar feel
• It's local-first, no cloud, no accounts, everything on the developer's machine

The key addition is the platform bridge abstraction, a universal IPlatformBridge interface that all mobile platforms
implement, producing a normalized MobileElement schema with component name, source location, bounding box, and
accessibility info regardless of the underlying platform.

The repo

The full project is here: https://github.com/andresdefi/agentation-mobile

It's early and there's a lot to improve, but the core workflow works: annotate on device → agent receives structured
context → agent fixes the right file.

What I'd love feedback on

• Whether this is something that could live under the Agentation umbrella (as a companion package or separate tool within the ecosystem)
• If the annotation schema should align more closely with Agentation's current format
• Any architectural decisions that would make future integration easier
• General thoughts on the approach

All contributions are encouraged and welcome as I am just a simple guy trying to contribute wherever I can

Thanks for building Agentation, it genuinely changed how I work with AI agents, and I hope this extension to phone screens can be useful to others too.