|
| 1 | +# Agents Desktop Plan |
| 2 | + |
| 3 | +## Goal |
| 4 | + |
| 5 | +Build a desktop app package at `packages/agents-desktop` that reuses the existing |
| 6 | +`packages/agents-server-ui` React app while adding local desktop functionality. |
| 7 | + |
| 8 | +The first version should not bundle an Agents server, Postgres, or Electric. It |
| 9 | +should bundle and manage the local builtin agents runtime from `@electric-ax/agents` |
| 10 | +so Horton and related background agents can run locally while connecting to an |
| 11 | +external Agents server. |
| 12 | + |
| 13 | +## Product Shape |
| 14 | + |
| 15 | +The desktop app is both: |
| 16 | + |
| 17 | +- A desktop windowed version of the Agents UI. |
| 18 | +- A background local agents runtime indicator/controller. |
| 19 | + |
| 20 | +On macOS this should include a menu bar icon next to the clock. On Windows and |
| 21 | +Linux the equivalent should be a tray/status area icon. |
| 22 | + |
| 23 | +The tray/menu bar app should make it clear that the local agents runtime is |
| 24 | +running even when all UI windows are closed. This matters because a CLI version |
| 25 | +of the interface may also use the same background runtime for agents. |
| 26 | + |
| 27 | +The app should support multiple windows. Closing the last window should not |
| 28 | +necessarily stop the local runtime; quitting the app explicitly should. |
| 29 | + |
| 30 | +## Version 1 Scope |
| 31 | + |
| 32 | +In scope: |
| 33 | + |
| 34 | +- Create `packages/agents-desktop`. |
| 35 | +- Package the shared `agents-server-ui` renderer inside Electron. |
| 36 | +- Start and stop a local `BuiltinAgentsServer` from `@electric-ax/agents`. |
| 37 | +- Register Horton and worker agent types with the selected external Agents server. |
| 38 | +- Persist desktop settings using Electron-side storage rather than only browser |
| 39 | + `localStorage`. |
| 40 | +- Show runtime status in both the app UI and tray/menu bar. |
| 41 | +- Support multiple app windows. |
| 42 | +- Allow the app to continue running in the background after windows are closed. |
| 43 | + |
| 44 | +Out of scope for v1: |
| 45 | + |
| 46 | +- Bundling `@electric-ax/agents-server`. |
| 47 | +- Bundling or managing Postgres. |
| 48 | +- Bundling or managing Electric. |
| 49 | +- Solving remote callback tunnelling for non-local Agents servers. |
| 50 | +- Full auto-update/signing/release polish, unless needed for internal testing. |
| 51 | + |
| 52 | +## Important Constraint |
| 53 | + |
| 54 | +The selected Agents server must be able to call the local builtin agents runtime |
| 55 | +webhook. |
| 56 | + |
| 57 | +For local development, this is straightforward: |
| 58 | + |
| 59 | +```text |
| 60 | +Desktop app starts BuiltinAgentsServer on 127.0.0.1:<port> |
| 61 | +Agents UI connects to http://127.0.0.1:4437 |
| 62 | +Agents server calls back to http://127.0.0.1:<port>/_electric/builtin-agent-handler |
| 63 | +``` |
| 64 | + |
| 65 | +For a remote Agents server, `127.0.0.1` would refer to the remote machine, not |
| 66 | +the user's desktop. Remote support needs a later tunnel/relay/public callback |
| 67 | +design. V1 should clearly communicate that the connected Agents server must be |
| 68 | +able to reach the local runtime URL. |
| 69 | + |
| 70 | +## Package Structure |
| 71 | + |
| 72 | +Proposed package: |
| 73 | + |
| 74 | +```text |
| 75 | +packages/agents-desktop/ |
| 76 | + package.json |
| 77 | + tsconfig.json |
| 78 | + vite.config.ts |
| 79 | + electron/ |
| 80 | + main.ts |
| 81 | + preload.ts |
| 82 | + src/ |
| 83 | + renderer-entry.tsx, if needed |
| 84 | + assets/ |
| 85 | + tray icons |
| 86 | +``` |
| 87 | + |
| 88 | +The package should depend on: |
| 89 | + |
| 90 | +- `@electric-ax/agents-server-ui` |
| 91 | +- `@electric-ax/agents` |
| 92 | +- `electron` |
| 93 | +- an Electron packaging/build tool |
| 94 | + |
| 95 | +The exact packaging tool can be chosen during implementation. Prefer the |
| 96 | +simplest setup that works cleanly in the pnpm monorepo and supports macOS first, |
| 97 | +with a path to Windows/Linux packaging. |
| 98 | + |
| 99 | +## Renderer Strategy |
| 100 | + |
| 101 | +Keep `packages/agents-server-ui` as the single shared renderer implementation. |
| 102 | + |
| 103 | +The web server build currently uses: |
| 104 | + |
| 105 | +```ts |
| 106 | +base: `/__agent_ui/` |
| 107 | +``` |
| 108 | + |
| 109 | +Electron should use a renderer build with a file-friendly asset base, likely: |
| 110 | + |
| 111 | +```ts |
| 112 | +base: `./` |
| 113 | +``` |
| 114 | + |
| 115 | +Implementation options: |
| 116 | + |
| 117 | +1. Add an environment-controlled base to `agents-server-ui/vite.config.ts`. |
| 118 | +2. Add a second build command in `agents-server-ui`, for example |
| 119 | + `build:desktop`. |
| 120 | +3. Let `agents-desktop` invoke or reference that desktop build. |
| 121 | + |
| 122 | +Prefer keeping the app code shared and changing only the build base. |
| 123 | + |
| 124 | +## Electron Main Process |
| 125 | + |
| 126 | +The Electron main process should own: |
| 127 | + |
| 128 | +- app lifecycle |
| 129 | +- tray/menu bar icon |
| 130 | +- window creation |
| 131 | +- multi-window tracking |
| 132 | +- local builtin agents runtime lifecycle |
| 133 | +- desktop settings persistence |
| 134 | +- IPC handlers exposed through preload |
| 135 | + |
| 136 | +Runtime startup should use the existing exported API: |
| 137 | + |
| 138 | +```ts |
| 139 | +import { BuiltinAgentsServer } from '@electric-ax/agents' |
| 140 | + |
| 141 | +const runtime = new BuiltinAgentsServer({ |
| 142 | + agentServerUrl, |
| 143 | + host: `127.0.0.1`, |
| 144 | + port: 0, |
| 145 | + workingDirectory, |
| 146 | +}) |
| 147 | + |
| 148 | +await runtime.start() |
| 149 | +``` |
| 150 | + |
| 151 | +Use `port: 0` so the OS selects a free port. The returned URL can be shown in |
| 152 | +debug/status UI and used for health checks. |
| 153 | + |
| 154 | +When the active Agents server changes, the desktop app should stop the current |
| 155 | +runtime and start a new one registered against the new `agentServerUrl`. |
| 156 | + |
| 157 | +## Tray/Menu Bar Behavior |
| 158 | + |
| 159 | +The tray/menu bar icon should indicate the runtime state: |
| 160 | + |
| 161 | +- Starting |
| 162 | +- Running |
| 163 | +- Error |
| 164 | +- Stopped |
| 165 | + |
| 166 | +Suggested menu items: |
| 167 | + |
| 168 | +- Open Agents |
| 169 | +- New Window |
| 170 | +- Runtime status |
| 171 | +- Connected server |
| 172 | +- Restart local runtime |
| 173 | +- Stop local runtime |
| 174 | +- Settings |
| 175 | +- Quit |
| 176 | + |
| 177 | +On macOS: |
| 178 | + |
| 179 | +- Closing a window should close that window but keep the app and runtime alive. |
| 180 | +- Cmd+Q or the tray Quit action should stop the runtime and quit the app. |
| 181 | +- Clicking the dock icon should reopen or create a window. |
| 182 | + |
| 183 | +On Windows/Linux: |
| 184 | + |
| 185 | +- Closing the last window should minimize-to-tray behavior unless the user chose |
| 186 | + Quit. |
| 187 | +- Tray Quit should stop the runtime and exit. |
| 188 | + |
| 189 | +## Multiple Windows |
| 190 | + |
| 191 | +The desktop app should support multiple independent renderer windows. |
| 192 | + |
| 193 | +Each window can load the same renderer build and share the same Electron main |
| 194 | +process state: |
| 195 | + |
| 196 | +- saved server list |
| 197 | +- active server |
| 198 | +- runtime status |
| 199 | +- working directory |
| 200 | +- API key availability/status |
| 201 | + |
| 202 | +The active server should probably be global for v1 because there is one local |
| 203 | +builtin agents runtime process. Per-window active servers would imply multiple |
| 204 | +runtime registrations and more complex lifecycle semantics. |
| 205 | + |
| 206 | +## Preload API |
| 207 | + |
| 208 | +Extend the existing `window.electronAPI` shape rather than exposing Node APIs to |
| 209 | +the renderer. |
| 210 | + |
| 211 | +Initial API: |
| 212 | + |
| 213 | +```ts |
| 214 | +window.electronAPI = { |
| 215 | + getServers(): Promise<Array<ServerConfig>> |
| 216 | + saveServers(servers: Array<ServerConfig>): Promise<void> |
| 217 | + |
| 218 | + getDesktopState(): Promise<DesktopState> |
| 219 | + setActiveServer(server: ServerConfig | null): Promise<void> |
| 220 | + restartRuntime(): Promise<void> |
| 221 | + stopRuntime(): Promise<void> |
| 222 | + |
| 223 | + getWorkingDirectory(): Promise<string | null> |
| 224 | + chooseWorkingDirectory(): Promise<string | null> |
| 225 | + |
| 226 | + onDesktopStateChanged(callback: (state: DesktopState) => void): () => void |
| 227 | +} |
| 228 | +``` |
| 229 | + |
| 230 | +Example state: |
| 231 | + |
| 232 | +```ts |
| 233 | +type DesktopRuntimeStatus = `stopped` | `starting` | `running` | `error` |
| 234 | + |
| 235 | +interface DesktopState { |
| 236 | + runtimeStatus: DesktopRuntimeStatus |
| 237 | + runtimeUrl: string | null |
| 238 | + activeServer: ServerConfig | null |
| 239 | + workingDirectory: string | null |
| 240 | + error: string | null |
| 241 | +} |
| 242 | +``` |
| 243 | + |
| 244 | +The renderer should use this for desktop-only behavior while continuing to work |
| 245 | +in a normal browser without `window.electronAPI`. |
| 246 | + |
| 247 | +## UI Changes |
| 248 | + |
| 249 | +Keep UI changes modest: |
| 250 | + |
| 251 | +- Show a local runtime status surface near the existing server picker or settings |
| 252 | + menu. |
| 253 | +- Distinguish between "connected to Agents server" and "local Horton runtime is |
| 254 | + registered/running". |
| 255 | +- If the runtime fails because no model provider key is configured, show a clear |
| 256 | + message for `ANTHROPIC_API_KEY` / `OPENAI_API_KEY`. |
| 257 | +- If the selected server is remote, warn that the server may not be able to call |
| 258 | + back to the local runtime unless a public callback URL is configured. |
| 259 | + |
| 260 | +## Configuration |
| 261 | + |
| 262 | +Desktop settings should include: |
| 263 | + |
| 264 | +- saved Agents servers |
| 265 | +- active Agents server |
| 266 | +- working directory |
| 267 | +- start runtime on launch |
| 268 | +- keep running after windows close |
| 269 | +- optional public callback/base URL override |
| 270 | + |
| 271 | +API keys need a product decision: |
| 272 | + |
| 273 | +- For v1 internal/dev use, reading from environment may be sufficient. |
| 274 | +- For packaged app users, store credentials in the OS keychain or equivalent. |
| 275 | + |
| 276 | +Do not store model provider API keys in plaintext JSON settings. |
| 277 | + |
| 278 | +## Development Workflow |
| 279 | + |
| 280 | +Suggested scripts: |
| 281 | + |
| 282 | +```json |
| 283 | +{ |
| 284 | + "dev": "run Electron with Vite renderer", |
| 285 | + "build": "build renderer, main, and preload", |
| 286 | + "package": "create unpacked app", |
| 287 | + "dist": "create distributable app" |
| 288 | +} |
| 289 | +``` |
| 290 | + |
| 291 | +During development, the Electron app can load the Vite dev server. In packaged |
| 292 | +builds, it should load the built renderer from disk. |
| 293 | + |
| 294 | +The external Agents server still needs to be started separately, for example: |
| 295 | + |
| 296 | +```sh |
| 297 | +DATABASE_URL=postgresql://... \ |
| 298 | +ELECTRIC_AGENTS_ELECTRIC_URL=http://localhost:3060 \ |
| 299 | +ELECTRIC_INSECURE=true \ |
| 300 | +node packages/agents-server/dist/entrypoint.js |
| 301 | +``` |
| 302 | + |
| 303 | +Then the desktop app connects to that server and starts its local builtin agents |
| 304 | +runtime against it. |
| 305 | + |
| 306 | +## Implementation Phases |
| 307 | + |
| 308 | +### Phase 1: Package Skeleton |
| 309 | + |
| 310 | +- Add `packages/agents-desktop`. |
| 311 | +- Add Electron main/preload TypeScript build. |
| 312 | +- Add a basic window loading the shared UI. |
| 313 | +- Add macOS tray/menu bar icon with Open, New Window, and Quit. |
| 314 | + |
| 315 | +### Phase 2: Shared Renderer Build |
| 316 | + |
| 317 | +- Add a desktop renderer build path for `agents-server-ui`. |
| 318 | +- Ensure hash routing and static assets work from Electron's file URL. |
| 319 | +- Keep the web `/__agent_ui/` build unchanged. |
| 320 | + |
| 321 | +### Phase 3: Settings and IPC |
| 322 | + |
| 323 | +- Move saved server persistence through Electron for desktop. |
| 324 | +- Add desktop state IPC. |
| 325 | +- Keep browser fallback behavior intact. |
| 326 | + |
| 327 | +### Phase 4: Builtin Runtime Lifecycle |
| 328 | + |
| 329 | +- Start `BuiltinAgentsServer` when an active server is selected. |
| 330 | +- Restart it when the active server changes. |
| 331 | +- Stop it on explicit app quit. |
| 332 | +- Surface status and errors in tray/menu bar and UI. |
| 333 | + |
| 334 | +### Phase 5: Multi-Window Polish |
| 335 | + |
| 336 | +- Add New Window support. |
| 337 | +- Share desktop state updates across windows. |
| 338 | +- Decide and implement close/minimize-to-tray behavior per platform. |
| 339 | + |
| 340 | +### Phase 6: Local Runtime UX |
| 341 | + |
| 342 | +- Add runtime status component to the UI. |
| 343 | +- Add restart/stop actions. |
| 344 | +- Add missing-key and unreachable-server guidance. |
| 345 | +- Add optional working directory picker. |
| 346 | + |
| 347 | +### Phase 7: Packaging |
| 348 | + |
| 349 | +- Produce a macOS build for internal testing. |
| 350 | +- Add icons and app metadata. |
| 351 | +- Document Windows/Linux packaging gaps and test the tray behavior there. |
| 352 | + |
| 353 | +## Open Questions |
| 354 | + |
| 355 | +- Should the desktop app own the active server globally, or should each window be |
| 356 | + able to choose independently? V1 should probably keep it global. |
| 357 | +- Should the runtime start immediately on launch or only after the first window |
| 358 | + chooses a server? |
| 359 | +- Where should working directory default to: app data, user's home directory, or |
| 360 | + the last selected project directory? |
| 361 | +- How should packaged users configure `ANTHROPIC_API_KEY` / `OPENAI_API_KEY`? |
| 362 | +- Is remote Agents server support required before public release, or can v1 |
| 363 | + explicitly target local Agents servers? |
| 364 | +- Should the CLI and desktop app coordinate over a shared local runtime lock/API? |
| 365 | + |
| 366 | +## Success Criteria |
| 367 | + |
| 368 | +- The Electron app opens the existing Agents UI. |
| 369 | +- Multiple windows can be opened. |
| 370 | +- The tray/menu bar icon remains present after all windows are closed. |
| 371 | +- The local builtin agents runtime starts and registers Horton with the selected |
| 372 | + Agents server. |
| 373 | +- The UI can start a Horton session through the external Agents server. |
| 374 | +- Quitting the app cleanly stops the local runtime. |
0 commit comments