diff --git a/README.md b/README.md index 28e2e44..f51dd7a 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ MCP server for [Vibe AI Browser](https://vibebrowser.app) — drive your **real, logged-in Chrome** from any MCP client, including agents running on a **different machine** with no inbound port open. -> ⚠️ **Security — treat your relay URL/UUID like a password.** A relay URL or extension UUID (`wss://relay.api.vibebrowser.app/`) grants **live control of your browser session** (read your tabs, take screenshots, read page content). Never share it, paste it into a chat with untrusted parties, or commit it to a repo. Every example UUID in this documentation is a non-routable placeholder (`YOUR-EXTENSION-UUID` or `00000000-0000-0000-0000-000000000000`) — substitute your own secret value locally and keep it out of version control. +> ⚠️ **Security — prefer hosted OAuth.** The canonical hosted MCP endpoint uses OAuth 2.1 and scoped access. Legacy relay URLs and extension UUIDs (`wss://relay.api.vibebrowser.app/`) are bearer credentials that grant full browser control; never share or commit them. Every UUID below is the non-routable placeholder `00000000-0000-0000-0000-000000000000` unless a variable is shown. ## Install prompt for AI agents (OpenClaw / Hermes) @@ -48,7 +48,7 @@ The browser-automation MCP space moved fast. Most of what used to differentiate Two rows are ours alone: -- **Outbound, cross-machine control.** The Vibe extension opens an outbound WebSocket to a relay (`wss://relay.api.vibebrowser.app/`). Nothing listens on the user's machine, nothing is port-forwarded, and the agent can live in a pod, a cron job, or a chat bot. No competitor documents an extension-initiated outbound connection to a remote MCP server. +- **Outbound, cross-machine control.** The Vibe extension opens an outbound WebSocket to a relay (`wss://relay.api.vibebrowser.app/`). Nothing listens on the user's machine, nothing is port-forwarded, and the agent can live in a pod, a cron job, or a chat bot. No competitor documents an extension-initiated outbound connection to a remote relay. - **Real profile without the dialog tax.** Chrome DevTools MCP needs approval for *each* WebSocket connection to Chrome ([#1794][cdp-1794], open; persistence was [closed won't-fix][cdp-825]). The maintainer's workaround — `--remote-debugging-port` with a dedicated `--user-data-dir` — avoids the dialog by giving up your logged-in profile. So: **skip the dialog, or use your real profile — not both.** Also worth knowing: @@ -70,7 +70,7 @@ Also worth knowing: ### Multi-Agent Architecture -Run Claude Desktop, Cursor, VS Code Copilot, and OpenCode at once — they share control of one browser through the relay, which multiplexes requests and routes each response back to the agent that asked. +Run Claude Desktop, Cursor, VS Code Copilot, and OpenCode at once. Each client starts a local `@vibebrowser/mcp` process over stdio; those processes share the browser connection through the relay. ``` Claude Desktop Cursor VS Code OpenCode @@ -81,7 +81,7 @@ Claude Desktop Cursor VS Code OpenCode +------------------+----------------+---------------+ | v - [Relay Daemon] <-- Auto-spawned, handles multiplexing + [Local Relay] | v [Vibe Extension] @@ -132,7 +132,7 @@ Edit your Claude Desktop config file: "mcpServers": { "vibe": { "command": "npx", - "args": ["-y", "@vibebrowser/mcp"] + "args": ["-y", "@vibebrowser/mcp@latest", "start", "--remote", "wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000"] } } } @@ -145,34 +145,34 @@ Restart Claude Desktop after saving.
Cursor -1. Open Cursor Settings (Cmd/Ctrl + ,) -2. Go to "Features" -> "MCP Servers" -3. Click "Add Server" and add: +Edit `~/.cursor/mcp.json` (or use the Cursor MCP settings UI): ```json { - "vibe": { - "command": "npx", - "args": ["-y", "@vibebrowser/mcp"] + "mcpServers": { + "vibebrowser": { + "type": "stdio", + "command": "npx", + "args": ["-y", "@vibebrowser/mcp@latest", "start", "--remote", "wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000"] + } } } ``` -Or edit `~/.cursor/mcp.json` directly. -
VS Code (GitHub Copilot) -Add to your VS Code settings.json: +Edit `.vscode/mcp.json` (or run `MCP: Open User Configuration`): ```json { - "github.copilot.chat.mcpServers": { - "vibe": { + "servers": { + "vibebrowser": { + "type": "stdio", "command": "npx", - "args": ["-y", "@vibebrowser/mcp"] + "args": ["-y", "@vibebrowser/mcp@latest", "start", "--remote", "wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000"] } } } @@ -190,7 +190,7 @@ Edit `~/.codeium/windsurf/mcp_config.json`: "mcpServers": { "vibe": { "command": "npx", - "args": ["-y", "@vibebrowser/mcp"] + "args": ["-y", "@vibebrowser/mcp@latest", "start", "--remote", "wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000"] } } } @@ -201,16 +201,16 @@ Edit `~/.codeium/windsurf/mcp_config.json`:
OpenCode -Add to your `.opencode/config.json`: +Add to your OpenCode configuration: ```json { + "$schema": "https://opencode.ai/config.json", "mcp": { - "servers": { - "vibe": { - "command": "npx", - "args": ["-y", "@vibebrowser/mcp"] - } + "vibebrowser": { + "type": "local", + "command": ["npx", "-y", "@vibebrowser/mcp@latest", "start", "--remote", "wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000"], + "enabled": true } } } @@ -228,7 +228,7 @@ Add to `~/.gemini/settings.json`: "mcpServers": { "vibe": { "command": "npx", - "args": ["-y", "@vibebrowser/mcp"] + "args": ["-y", "@vibebrowser/mcp@latest", "start", "--remote", "wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000"] } } } @@ -244,18 +244,18 @@ All three share one config file: `~/.codex/config.toml`. Configure once, use eve Easiest — let Codex write the entry: ```bash -codex mcp add vibe -- npx -y @vibebrowser/mcp +codex mcp add vibebrowser -- npx -y @vibebrowser/mcp@latest start --remote wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000 ``` Or add the table to `~/.codex/config.toml` by hand (Codex uses **TOML**, not JSON): ```toml -[mcp_servers.vibe] +[mcp_servers.vibebrowser] command = "npx" -args = ["-y", "@vibebrowser/mcp"] +args = ["-y", "@vibebrowser/mcp@latest", "start", "--remote", "wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000"] ``` -In the ChatGPT desktop app you can also use the UI: **Settings → MCP servers → Add server**, choose **STDIO**, command `npx`, args `-y @vibebrowser/mcp`, then **Restart**. +In the ChatGPT desktop app you can also use the UI: **Settings -> MCP servers -> Add server**, choose **STDIO**, command `npx`, and enter each argument separately: `-y`, `@vibebrowser/mcp@latest`, `start`, `--remote`, and the full placeholder WSS URL above. Then restart the app. Verify with `codex mcp list`, or type `/mcp` in the Codex TUI or the desktop composer. @@ -280,7 +280,7 @@ npx -y -p @vibebrowser/mcp@latest vibebrowser-mcp --help npx -y -p @vibebrowser/mcp@latest vibe-mcp --help ``` -### Remote connector (hosted assistants — no install) +### Hosted remote MCP (no local process) Everything above assumes the client can spawn a local process. Hosted assistants cannot: Claude on the web / in Cowork / on mobile, and ChatGPT on @@ -288,15 +288,17 @@ the web, run in the vendor's cloud with no access to your machine. They accept a **remote MCP server URL** and nothing else — no command, no arguments, and no custom request headers. -For those, skip `@vibebrowser/mcp` entirely. The extension alone is enough: +For those, skip `@vibebrowser/mcp` entirely and use the canonical hosted endpoint: ``` -https://relay.api.vibebrowser.app/mcp/ +https://relay.api.vibebrowser.app/mcp ``` -Find `` in the extension: **Vibe icon → Settings → Agent -connection URL**. It is the same UUID the CLI takes as -`--remote wss://relay.api.vibebrowser.app/`. +It is a Streamable HTTP MCP endpoint with OAuth 2.1 and Dynamic Client Registration. +An unauthenticated request receives `401` plus OAuth discovery metadata. `browser:read` +is sufficient for MCP `initialize`, `ping`, and `tools/list`; every `tools/call` requires +`browser:control`, including tools annotated read-only. `browser:control` implies +`browser:read`. Where to paste it: @@ -307,25 +309,38 @@ Where to paste it: Custom connectors are a paid-plan feature in both products. -The relay also accepts the UUID as an `X-Remote-Session` or -`Authorization: Bearer` header on a bare `POST /mcp`. Use the header form from -anything that can send one (Codex CLI, scripts) — it keeps the credential out -of URLs, logs, and browser history. The path form exists specifically for the -UIs that cannot send a header. +For API-capable clients, hosted `/mcp` also supports `X-Remote-Session` containing +either a bare UUID or the canonical full WSS relay URL. Resolution order in the +hosted platform is OAuth bearer, `X-Remote-Session`, legacy non-OAuth bearer +compatibility, then path UUID. OAuth is preferred. + +The legacy hosted form remains supported for compatibility: + +``` +https://relay.api.vibebrowser.app/mcp/00000000-0000-0000-0000-000000000000 +``` + +The legacy URL is unscoped, grants full control, and is itself a bearer credential. +The UUID is the same value accepted by local clients as +`--remote wss://relay.api.vibebrowser.app/`. + +**Privacy and security:** -**Two things this path costs you, stated plainly:** +Hosted OAuth enforces `browser:read` for MCP `initialize`, `ping`, and `tools/list`. +Every `tools/call` requires `browser:control`, including tools annotated read-only; +`browser:control` implies `browser:read`. -1. **The URL is a credential.** Anyone who has that UUID can drive your +1. **Legacy URLs are credentials.** Anyone who has that UUID can drive your logged-in browser — read your mail, act as you on any site you are signed into. Treat it exactly like a password: never commit it, never paste it into a shared chat, an issue, a README, or a screenshot. If it leaks, revoke the session in the extension and generate a new one. (This is not hypothetical: a dev machine's UUID once shipped in these very docs.) -2. **Page content leaves your machine.** On the local STDIO path, tool calls - and page content never leave `127.0.0.1`. On the remote-connector path they - traverse `relay.api.vibebrowser.app`, because the assistant is in the - vendor's cloud and has no other route to your browser. If you need - on-device-only, use a desktop client with the local server instead. +2. **Page content leaves your machine on remote paths.** Local stdio and + loopback-only local HTTP without `--remote` keep browser-facing traffic on + `127.0.0.1`. Hosted MCP or local MCP configured with `--remote` sends browser + traffic through hosted infrastructure. If you need on-device-only, use local + stdio or loopback-only local HTTP without `--remote`. ### 3. Connect the Extension @@ -399,32 +414,34 @@ Claude / Cursor / VS Code (stdio) 3. The relay forwards commands to the extension on port `19889` 4. Results flow back to the agent -### Two legs, two protocols +### Supported paths -Don't conflate them — the transport to the MCP server is configurable, the transport to the extension is not. +Do not conflate the relay WebSocket protocol with MCP. The relay carries proprietary project JSON messages such as `list_tools`, `call_tool`, and `tool_result`; it does not carry MCP over WebSocket. -| Leg | Protocol | Configurable? | +| Path | Client-facing transport | Browser-facing transport | |---|---|---| -| MCP client → `vibebrowser-mcp` | stdio **or** streamable HTTP | Yes — `--transport stdio\|http` | -| `vibebrowser-mcp` ↔ relay ↔ extension | WebSocket only | No | +| Direct `@vibebrowser/cli` | CLI process | `src/browser-cli.ts` -> `ExtensionConnection` -> WS/WSS relay JSON | +| Local `@vibebrowser/mcp` | MCP over stdio (default) or Streamable HTTP at `http://127.0.0.1:8788/mcp` | `ExtensionConnection` -> local `ws://127.0.0.1:19888` or remote WS/WSS with `--remote` | +| Hosted remote MCP (preferred) | Streamable HTTP with OAuth at `https://relay.api.vibebrowser.app/mcp` | Separate hosted service routes to the extension relay | +| Hosted remote MCP (legacy) | Streamable HTTP at `https://relay.api.vibebrowser.app/mcp/00000000-0000-0000-0000-000000000000`, or hosted `/mcp` compatibility routing | Separate hosted service routes to the extension relay | +| Direct CLI or local MCP with `--devtools` | CLI output or MCP over stdio/local HTTP | Direct CDP; bypasses relay and extension | ``` -MCP client ──stdio──┐ - ├──> [vibebrowser-mcp] ──ws──> [relay] ──ws──> [extension] -MCP client ──http───┘ +MCP client --stdio/http--> [vibebrowser-mcp] --project JSON over WS/WSS--> [relay] --> [extension] ``` -Streamable HTTP lets a remote or hosted agent — one that can't spawn a stdio -subprocess — talk to the server over a URL: +Local Streamable HTTP is for clients on the same machine that cannot use stdio: ```bash npx -y @vibebrowser/mcp@latest start --transport http # serves POST/GET http://127.0.0.1:8788/mcp ``` -Defaults: `--host 127.0.0.1`, `--http-port 8788`, `--http-path /mcp`. Add -`--allow-host ` (repeatable) if you front it with a proxy or bind it -beyond localhost. +Defaults: `--host 127.0.0.1`, `--http-port 8788`, `--http-path /mcp`. This local +HTTP endpoint has no client authentication and must remain loopback-only. +`--allow-host ` (repeatable) only validates the HTTP `Host` header; it is +not authentication and does not make non-loopback or public exposure safe. Use +the hosted OAuth endpoint for remote MCP clients. ### Multi-Agent Mode @@ -435,52 +452,45 @@ When multiple agents connect, Vibe MCP automatically spawns a relay daemon: - Relay multiplexes all agent requests to the single extension connection - Each agent receives only its own responses -### Cloud OpenClaw -> Local Browser +### Remote relay target -> ⚠️ **Security:** The `--remote` value below is a live credential — a relay URL/UUID grants full control of the target browser session. It is the *sole* bearer credential (there is no second-factor token). Treat it like a password: keep it secret, never commit it or paste it into logs, and if it leaks, regenerate it in the Vibe extension Settings. +> **Security:** For the browser-facing WS/WSS `--remote` connection, the UUID path is the bearer capability. Treat the complete URL as a secret. This relay connection has no separate shared secret, authorization header, or query-string credential. Hosted MCP authentication is separate and described above. -If your agent runs in the cloud but you want it to control the user's real local browser, run `vibebrowser-mcp` in HTTP mode and connect it to the Vibe extension in remote relay mode. Pass either the extension UUID or the full WebSocket relay URL to `--remote`. +To connect a local MCP process to an extension through the remote relay, pass either the extension UUID or the full WebSocket relay URL to `--remote`. The MCP-facing transport should remain stdio or loopback-only HTTP. ```bash -VIBE_REMOTE_UUID="YOUR-EXTENSION-UUID" -VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/YOUR-EXTENSION-UUID" -npx -y @vibebrowser/mcp@latest start --transport http --remote "$VIBE_REMOTE_UUID" +VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000" npx -y @vibebrowser/mcp@latest start --transport http --remote "$VIBE_REMOTE_URL" ``` -This exposes a local MCP endpoint at `http://127.0.0.1:8788/mcp` by default. - -When OpenClaw runs on a different machine (for example cloud-hosted), provide a reachable URL: - -```bash -VIBE_REMOTE_UUID="YOUR-EXTENSION-UUID" -VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/YOUR-EXTENSION-UUID" -PUBLIC_MCP_URL="https://browser-bridge.example.com/mcp" -npx -y @vibebrowser/mcp@latest openclaw --remote "$VIBE_REMOTE_UUID" --public-url "$PUBLIC_MCP_URL" -npx -y @vibebrowser/mcp@latest openclaw --remote "$VIBE_REMOTE_URL" --public-url "$PUBLIC_MCP_URL" -``` +This exposes a local, unauthenticated MCP endpoint at `http://127.0.0.1:8788/mcp` +by default. Do not publish it or place it behind a public reverse proxy. Remote +MCP clients should use `https://relay.api.vibebrowser.app/mcp` with OAuth. -You can print the exact OpenClaw-friendly setup with: +For an OpenClaw client running on the same machine, you can print loopback-only +configuration with: ```bash -VIBE_REMOTE_UUID="YOUR-EXTENSION-UUID" -VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/YOUR-EXTENSION-UUID" -npx -y @vibebrowser/mcp@latest openclaw --remote "$VIBE_REMOTE_UUID" +VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000" npx -y @vibebrowser/mcp@latest openclaw --remote "$VIBE_REMOTE_URL" ``` -Use `--remote ` with the default public relay, or `--remote ` when you need an explicit relay endpoint. The UUID is the only credential — never share it, log it, or paste it into an untrusted chat. +This exact invocation uses the helper's default loopback settings and is safe +only for same-machine OpenClaw. The `openclaw` command exposes `--host` and +`--public-url`, but the local HTTP server has no client authentication, so do +not use those options to publish it. Remote OpenClaw clients should use +`https://relay.api.vibebrowser.app/mcp` with hosted OAuth. + +Use `--remote ` with the default relay, or `--remote ` for an explicit relay endpoint. This `src/connection.ts` validation applies to browser-facing WS/WSS relay targets; it is separate from the hosted `/mcp` service's OAuth and compatibility routing. For direct browser CLI checks, always use `npx -y @vibebrowser/cli@latest`: ```bash -VIBE_REMOTE_UUID="YOUR-EXTENSION-UUID" -VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/YOUR-EXTENSION-UUID" -npx -y @vibebrowser/cli@latest --remote "$VIBE_REMOTE_UUID" --json status +VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000" npx -y @vibebrowser/cli@latest --remote "$VIBE_REMOTE_URL" --json status ``` -`--remote ` uses the default public relay. `--remote ` targets an explicit relay endpoint. No second-factor secret is needed or accepted — the UUID alone authorizes the session. +`@vibebrowser/cli` calls `ExtensionConnection` directly and exchanges project JSON messages over WS/WSS. This is explicitly not MCP-over-WebSocket. For the full walkthrough, see `docs/openclaw-local-browser.md`. @@ -490,8 +500,8 @@ For the full walkthrough, see `docs/openclaw-local-browser.md`. ```bash npx -y @vibebrowser/cli@latest sessions -VIBE_REMOTE_UUID="YOUR-EXTENSION-UUID" -VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/YOUR-EXTENSION-UUID" +VIBE_REMOTE_UUID="00000000-0000-0000-0000-000000000000" +VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000" npx -y @vibebrowser/cli@latest --remote "$VIBE_REMOTE_UUID" status npx -y @vibebrowser/cli@latest --remote "$VIBE_REMOTE_URL" tabs npx -y @vibebrowser/cli@latest --remote "$VIBE_REMOTE_UUID" open https://example.com @@ -540,13 +550,14 @@ For a deeper skill definition (when to prefer this over a managed/headless brows There are two ways to use Vibe with OpenClaw: -**Option A: Cloud OpenClaw controlling local browser** +**Option A: Remote OpenClaw controlling the browser** -If OpenClaw runs in the cloud but you want it to control your local browser: +If OpenClaw runs remotely, configure it with the hosted OAuth MCP endpoint +`https://relay.api.vibebrowser.app/mcp`. Do not expose the local HTTP bridge. 1. Install the Vibe extension and enable **Remote** mode (see [docs/openclaw-local-browser.md](docs/openclaw-local-browser.md)) -2. Start the local HTTP bridge: `vibebrowser-mcp openclaw --remote "$VIBE_REMOTE_UUID" [--public-url "$PUBLIC_MCP_URL"]` or `vibebrowser-mcp openclaw --remote "$VIBE_REMOTE_URL" [--public-url "$PUBLIC_MCP_URL"]` -3. Register the MCP URL in OpenClaw +2. Register `https://relay.api.vibebrowser.app/mcp` in OpenClaw +3. Complete OAuth authorization with only the required scope; invoking any browser tool requires `browser:control` **Option B: OpenClaw skill for local agents** @@ -623,25 +634,20 @@ npx -y @vibebrowser/mcp@latest [start] [options] --host Host to bind the HTTP server to (default: 127.0.0.1) --http-port Port for streamable HTTP MCP transport (default: 8788) --http-path Path for streamable HTTP MCP transport (default: /mcp) - --allow-host Allowed host header for HTTP transport (repeatable) + --allow-host Allowed Host header for local HTTP transport (repeatable; validation only, not authentication or permission for public exposure) -r, --remote Extension UUID, or full ws(s) remote URL. This routing UUID is the sole bearer credential — treat it like a password. --devtools Drive your real running Chrome directly over the DevTools Protocol (bypasses the extension relay) # MCP server tool -set_remote { "url": "wss://relay.api.vibebrowser.app/" } +set_remote { "url": "wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000" } ``` The `set_remote` MCP server tool hot-reconnects the running MCP server to a different remote relay URL. It is an MCP tool, not a browser CLI subcommand. The URL's UUID is the only credential — never place it in logs shared with untrusted parties; regenerate it in extension Settings if exposed. ```bash -# OpenClaw helper -VIBE_REMOTE_UUID="YOUR-EXTENSION-UUID" -VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/YOUR-EXTENSION-UUID" -PUBLIC_MCP_URL="https://browser-bridge.example.com/mcp" -npx -y @vibebrowser/mcp@latest openclaw --remote "$VIBE_REMOTE_UUID" --public-url "$PUBLIC_MCP_URL" -npx -y @vibebrowser/mcp@latest openclaw --remote "$VIBE_REMOTE_URL" --public-url "$PUBLIC_MCP_URL" - # OpenClaw-compatible browser CLI +VIBE_REMOTE_UUID="00000000-0000-0000-0000-000000000000" +VIBE_REMOTE_URL="wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000" npx -y @vibebrowser/cli@latest --remote "$VIBE_REMOTE_UUID" status npx -y @vibebrowser/cli@latest --remote "$VIBE_REMOTE_URL" status --wait-for-extension --wait-timeout 10000 npx -y @vibebrowser/cli@latest --remote "$VIBE_REMOTE_UUID" tabs @@ -666,12 +672,11 @@ npx -y @vibebrowser/mcp@latest serve "$MODEL" 2. Click the extension icon and enable "MCP External Control" in Settings 3. Check that no firewall is blocking localhost connections -### "OpenClaw cannot reach my local browser bridge" +### "A remote MCP client cannot reach the local HTTP endpoint" -1. Start `vibebrowser-mcp` in HTTP mode instead of stdio -2. Make sure the bridge process is still running on the user's machine -3. Confirm the extension is in `Remote` mode and connected -4. Verify the MCP URL in OpenClaw matches the bridge URL. If OpenClaw is cloud-hosted, do not use `127.0.0.1`; use `openclaw --public-url` with a reachable host. +This is expected: the unauthenticated local HTTP transport is loopback-only and +must not be exposed. Configure the remote client with the hosted OAuth endpoint +`https://relay.api.vibebrowser.app/mcp` instead. ### Debug mode diff --git a/docs/chrome-devtools-relay.md b/docs/chrome-devtools-relay.md index a6ada0c..4570578 100644 --- a/docs/chrome-devtools-relay.md +++ b/docs/chrome-devtools-relay.md @@ -1,360 +1,97 @@ -# Chrome DevTools Relay — System Design +# Relay And MCP Architecture -> ⚠️ **Security:** A relay URL/UUID (`wss://relay.api.vibebrowser.app/`) grants **live control of your browser session** (read tabs, screenshots, page content). For the standard `vibebrowser-mcp`/`vibebrowser-cli` relay connection, this UUID is the *sole* bearer credential — treat it like a password, and regenerate it in extension Settings if exposed. Any example UUIDs are placeholders (`00000000-0000-0000-0000-000000000000`) and are not routable. (The separate cloud REST API design described below, for OpenClaw skills/CI callers hitting `relay.vibebrowser.com`, uses its own bearer token — that is a distinct mechanism from the agent's wss relay connection.) +Vibe Browser supports direct CLI, local MCP, and hosted MCP paths. The WebSocket relay speaks Vibe's proprietary project JSON protocol, not MCP-over-WebSocket. The hosted production platform is separate from this repository's local package. -## Problem +## Transport Matrix -Cloud AI agents (OpenClaw tenants, coding assistants, automation pipelines) need to -control a user's **local Chrome browser** — the one with their cookies, bookmarks, -logged-in sessions, and extensions. Today this requires the agent and browser to -be on the same machine or network. +| Path | Client-facing transport | Browser-facing transport | Ownership / entry point | +|---|---|---|---| +| Direct `@vibebrowser/cli` relay | command output | WS/WSS proprietary project JSON at relay root | This repo: `src/browser-main.ts` -> `src/browser-cli.ts` -> `ExtensionConnection` | +| Direct CLI with `--devtools` | command output | direct CDP; no extension or relay | This repo: `src/browser-main.ts` -> `src/browser-cli.ts` -> `src/chrome-use-connection.ts` | +| Local `@vibebrowser/mcp` relay | MCP over stdio by default, or local Streamable HTTP at `http://127.0.0.1:8788/mcp` | local `ws://127.0.0.1:19888` or remote WS/WSS with `--remote` | This repo: `src/cli.ts` -> `src/server.ts` -> `ExtensionConnection` | +| Local relay fallback when the extension is unavailable | existing local relay clients | shared `chrome-devtools-mcp` backend; local relay daemon/path only | This repo: `src/relay.ts` -> `DevtoolsFallbackConnection` | +| Local MCP with `--devtools` | MCP over stdio or loopback HTTP | direct CDP; no extension or relay | This repo: `src/cli.ts` -> `src/server.ts` -> `src/chrome-use-connection.ts` | +| Canonical hosted MCP | Streamable HTTP with OAuth 2.1/DCR at `https://relay.api.vibebrowser.app/mcp` | hosted platform routes to extension relay | Separate hosted production platform | +| Legacy hosted MCP | Streamable HTTP path UUID, or `/mcp` compatibility routing via header/bearer | hosted platform routes to extension relay | Separate hosted production platform | -The user's browser is behind NAT, on a laptop, on a home network. The agent runs -in a cloud Kubernetes pod. There is no direct path. +## Direct CLI -## Solution +`@vibebrowser/cli` does not start an MCP server. `src/browser-cli.ts` creates an `ExtensionConnection`, which connects locally or to: -A **secure relay** that bridges the gap: - -> Current `vibebrowser-mcp` behavior: relay+extension remains the primary backend. -> If `chrome-devtools-mcp` is available locally, the server also starts it in -> `--autoConnect` mode as a fallback inside the shared relay daemon (single -> instance for all local agents/CLIs). When extension is connected, extension -> tools remain authoritative. Fallback tools are exposed and used only when the -> extension is unavailable/disconnected. - -``` -┌──────────────────────────────────────────────────────────────────┐ -│ CLOUD │ -│ │ -│ ┌─────────────────────┐ ┌──────────────────────────┐ │ -│ │ OpenClaw Skill │ │ Vibe Relay Backend │ │ -│ │ (or any REST caller)│──POST─▶│ relay.vibebrowser.com │ │ -│ │ │◀─resp──│ │ │ -│ │ Authorization: │ │ • authenticates bearer │ │ -│ │ Bearer │ │ • routes by browser_uuid │ │ -│ └─────────────────────┘ │ • queues tool calls │ │ -│ │ • returns results │ │ -│ └────────────┬─────────────┘ │ -│ │ │ -└───────────────────────────────────────────────│───────────────────┘ - │ WSS (outbound - │ from user machine) -┌───────────────────────────────────────────────│───────────────────┐ -│ USER MACHINE │ │ -│ ▼ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ Relay Client (local process) │ │ -│ │ • connects outbound to wss://relay.vibebrowser.com │ │ -│ │ • registers browser_uuid │ │ -│ │ • receives tool calls from relay │ │ -│ │ • executes them via chrome-devtools-mcp tools │ │ -│ │ • returns results through relay │ │ -│ └──────────────────────────┬─────────────────────────────────┘ │ -│ │ CDP (localhost:9222) │ -│ ▼ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ Chrome (user's browser) │ │ -│ │ --remote-debugging-port=9222 │ │ -│ │ User's cookies, sessions, extensions, bookmarks │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ │ -└───────────────────────────────────────────────────────────────────┘ +```text +wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000 ``` -### Key properties - -- **No inbound ports** on the user's machine. The relay client connects outbound. -- **Bearer token auth** on every cloud API call. The browser UUID alone is not a secret. -- **Full chrome-devtools-mcp toolset** — 27 tools: click, fill, navigate, screenshot, - snapshot, network interception, performance tracing, Lighthouse, etc. -- **User stays in control** — they start/stop the relay client, they see their browser. -- **Works with any cloud caller** — OpenClaw skills, CI pipelines, REST scripts. - ---- - -## Components - -### 1. Relay Client (this repo, forked from chrome-devtools-mcp) - -A local CLI process the user runs. Based on a fork of -[chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp) -(Apache 2.0, by Google/ChromeDevTools team). - -**What chrome-devtools-mcp already provides:** - -- 27 browser automation tools (input, navigation, emulation, performance, - network, console, debugging, Lighthouse) -- Puppeteer-based Chrome control via CDP -- Connection to running Chrome via `--browser-url` or `--autoConnect` -- WebSocket endpoint connection with custom headers -- Slim mode (3 tools) for basic tasks -- Requires Node.js v20.19+, Chrome stable+ - -**What we add:** - -- Outbound WebSocket connection to cloud relay -- Tool call receive/execute/respond loop -- Auth handshake (browser_uuid + access_token) -- Reconnection with backoff -- Local config persistence (~/.vibe-relay/) +Messages are project JSON such as `list_tools`, `call_tool`, `tools_list`, and `tool_result`. Calling this MCP-over-WebSocket is incorrect. -**CLI interface:** +## Local MCP -```bash -# First time: authenticate -vibe-relay login --token +`@vibebrowser/mcp` exposes the Model Context Protocol to an MCP client: -# Connect to relay (browser must be running with --remote-debugging-port) -vibe-relay connect - -# Or with explicit browser URL -vibe-relay connect --browser-url http://localhost:9222 - -# Check status -vibe-relay status +```text +MCP client --stdio----------------------> Vibe MCP server +MCP client --Streamable HTTP /mcp------> Vibe MCP server + | + +--project JSON over WS/WSS--> relay --> extension or shared local fallback ``` -### 2. Vibe Relay Backend (cloud service) - -Stateless relay that authenticates callers and routes tool calls to connected -browser sessions. - -**API endpoints:** - -``` -GET /v1/browser-sessions/:uuid/status - → { connected: bool, tools: string[], connectedAt: ISO8601 } +The default client transport is stdio. `start --transport http` binds to `127.0.0.1`, port `8788`, path `/mcp` unless overridden. The MCP server's relay connection remains WebSocket: local mode uses `ws://127.0.0.1:19888`; `--remote` uses a WS/WSS relay target. -GET /v1/browser-sessions/:uuid/tools - → { tools: [ { name, description, inputSchema } ] } +Local HTTP has no client authentication and must remain loopback-only. `--allow-host` only validates the HTTP `Host` header; it is not authentication and does not authorize non-loopback or public exposure. Use hosted OAuth for remote MCP clients. -POST /v1/browser-sessions/:uuid/tools/:toolName - → { result: ... } - Body: { arguments: { ... } } +The local relay daemon in `src/relay.ts` listens for agents on `19888` and extensions on `19889`. It multiplexes agent requests, tracks extension sessions, routes responses to the requesting agent, and guards extension socket replacement races. It also owns one shared `chrome-devtools-mcp` fallback through `DevtoolsFallbackConnection`: extension tools are authoritative while connected, and local relay clients use the shared fallback when the extension is unavailable. This fallback is limited to the local relay daemon/path; it is not used by `--remote` or hosted MCP modes. -All require: Authorization: Bearer -``` +Explicit `--devtools` is different: it bypasses both relay and extension through `src/chrome-use-connection.ts` rather than using the relay-owned fallback. -**Responsibilities:** - -- Validate bearer tokens (scope: user, session, optional tenant/workspace) -- Maintain WebSocket connections to relay clients -- Queue tool call requests, match to responses -- Timeout handling (30s default, configurable) -- Rate limiting per token - -**Not responsible for:** - -- Executing any browser commands (that's the relay client) -- Storing browser state or screenshots -- Managing Chrome processes - -### 3. OpenClaw Integration (skill + config) - -OpenClaw does not speak MCP. It uses skills that call CLI tools or REST APIs. - -**Skill definition:** - -```yaml -name: vibe-browser -description: Control the user's local Chrome browser through Vibe relay -tools: - - navigate_page - - take_snapshot - - take_screenshot - - click - - fill - - press_key - - evaluate_script - - list_network_requests - - get_network_request - - lighthouse_audit - # ... all 27 chrome-devtools-mcp tools -``` +## Hosted MCP -Each tool in the skill calls the relay REST API: +Clients that cannot spawn a local stdio process should use the canonical hosted Streamable HTTP endpoint: -```bash -curl -X POST \ - -H "Authorization: Bearer $VIBE_ACCESS_TOKEN" \ - -H "Content-Type: application/json" \ - "https://relay.vibebrowser.com/v1/browser-sessions/$BROWSER_UUID/tools/navigate_page" \ - -d '{"arguments": {"url": "https://example.com", "type": "url"}}' +```text +https://relay.api.vibebrowser.app/mcp ``` ---- - -## Security Model - -### Identifiers vs Secrets - -| Value | Purpose | Secret? | Rotatable? | -|----------------|--------------------------------|---------|------------| -| browser_uuid | Identifies which browser | No | Yes | -| access_token | Proves caller is authorized | Yes | Yes | - -### Token Scoping +It uses OAuth 2.1 with Dynamic Client Registration. An unauthenticated request receives `401` plus discovery metadata. `browser:read` is sufficient for MCP `initialize`, `ping`, and `tools/list`; every `tools/call` requires `browser:control`, including tools annotated read-only. `browser:control` implies `browser:read`. -Access tokens are scoped to: +The legacy hosted URL remains supported and grants unscoped full control: -- **User** — which user owns this token -- **Session** (optional) — which browser session(s) it can access -- **Tenant/Workspace** (optional) — for multi-tenant scenarios -- **Expiration** — TTL, default 24h -- **Permissions** (future) — which tools are allowed - -### Auth Flow - -``` -1. User generates access_token via Vibe dashboard or CLI -2. User starts relay client with token (stored in ~/.vibe-relay/config.json) -3. Relay client connects to cloud relay, sends { browser_uuid, access_token } -4. Cloud relay validates token, registers session -5. Cloud caller (OpenClaw) sends tool call with Authorization: Bearer -6. Cloud relay validates caller's token, checks session access, forwards to client -7. Client executes tool locally, returns result through relay +```text +https://relay.api.vibebrowser.app/mcp/00000000-0000-0000-0000-000000000000 ``` -### Threat Model - -| Threat | Mitigation | -|----------------------------------|-------------------------------------------------| -| Stolen browser_uuid | UUID alone grants nothing; need valid token | -| Stolen access_token | Scoped + expiring; revocable via dashboard | -| MITM on relay connection | WSS (TLS) for client↔relay; HTTPS for API calls | -| Relay client compromise | Runs as user process; same trust as the browser | -| Cloud relay compromise | No browser access stored; just routing + auth | -| Unauthorized tool execution | Token scope + optional tool allowlists | - ---- - -## Why Not Other Approaches - -### Why not HTTP MCP bridge on vibebrowser-mcp? - -We explored adding `--transport http` to vibebrowser-mcp to expose a URL-addressable MCP -endpoint. This doesn't work for OpenClaw because: - -1. **OpenClaw doesn't speak MCP** — it calls CLI tools and REST APIs from skills. -2. **MCP HTTP still requires a reachable endpoint** — the user's machine is behind - NAT, so a localhost MCP server isn't reachable from cloud. -3. **It solves the wrong problem** — the issue isn't MCP vs HTTP, it's - cloud-to-local connectivity. - -### Why not CDP shim over Vibe browser extension? - -The Vibe browser extension (`chrome.debugger` API) only exposes a restricted -subset of CDP domains: - -- **Missing:** `Browser.*`, `SystemInfo.*`, `Security.*`, `ServiceWorker.*`, - `HeapProfiler`, `Memory`, `LayerTree`, `Media` -- **Conflict:** Opening Chrome DevTools terminates the extension's debugger session -- **Limited:** No `Browser.getWindowForTarget`, no full network interception +API-capable clients may also send `X-Remote-Session` to hosted `/mcp`, containing either a bare UUID or the canonical full WSS URL. Hosted production resolves credentials/routing in this order: OAuth bearer, `X-Remote-Session`, legacy non-OAuth bearer compatibility, then path UUID. Prefer OAuth. -The chrome-devtools-mcp approach via Puppeteer/CDP gives full access to all -Chrome DevTools Protocol domains. +These HTTPS endpoints carry MCP. They are distinct from `wss://relay.api.vibebrowser.app/`, which carries proprietary relay JSON at the relay root. Hosted compatibility routing is also distinct from the WS/WSS URL validation implemented by `src/connection.ts`. -### Why not tunnel/ngrok? +## Source Components -- Requires the user to install and configure a separate tunneling tool -- Exposes a raw CDP endpoint to the internet (massive attack surface) -- No built-in auth, rate limiting, or tool-level access control -- CDP is designed for localhost trust, not internet exposure +- `src/browser-cli.ts`: direct browser commands and tool selection. +- `src/connection.ts`: local and remote relay URL handling, WS/WSS lifecycle, project JSON requests, and response correlation. +- `src/server.ts`: MCP protocol server, stdio transport, and local Streamable HTTP transport. +- `src/cli.ts`: command-line transport and remote-target options. +- `src/relay.ts`: local multi-agent relay, extension session routing, and the shared `DevtoolsFallbackConnection` fallback. +- `src/chrome-use-connection.ts`: explicit `--devtools` backend that bypasses the extension relay. +- Hosted OAuth, discovery, and compatibility routing belong to the separate production platform, not this repository's local server implementation. -### Why fork chrome-devtools-mcp instead of building from scratch? +## Security -- **27 production-tested tools** with proper error handling and edge cases -- **Apache 2.0 license** — fork-friendly, no copyleft concerns -- **Active maintenance** by Google/ChromeDevTools team -- **Puppeteer integration** — handles Chrome lifecycle, reconnection, tab management -- We only need to add the relay transport layer; the tool implementations stay as-is +The canonical hosted endpoint should be used with scoped OAuth. `browser:read` is sufficient for MCP `initialize`, `ping`, and `tools/list`; every `tools/call` requires `browser:control`, including tools annotated read-only. `browser:control` implies `browser:read`. ---- +Legacy UUID URLs are bearer capabilities. Possession of either complete legacy URL can authorize full browser access: -## Data Flow: Tool Execution - -``` -OpenClaw skill Cloud Relay Relay Client Chrome - │ │ │ │ - │ POST /tools/take_snapshot │ │ │ - │ Authorization: Bearer xxx │ │ │ - │ ──────────────────────────────▶│ │ │ - │ │ validate token │ │ - │ │ lookup browser session │ │ - │ │ │ │ - │ │ WS: { call: take_snapshot } │ │ - │ │ ─────────────────────────────▶│ │ - │ │ │ CDP: getDocument │ - │ │ │ ────────────────────▶│ - │ │ │ │ - │ │ │ CDP: a11y snapshot │ - │ │ │◀────────────────────│ - │ │ │ │ - │ │ WS: { result: snapshot } │ │ - │ │◀─────────────────────────────│ │ - │ │ │ │ - │ 200 { result: snapshot } │ │ │ - │◀──────────────────────────────│ │ │ - │ │ │ │ +```text +wss://relay.api.vibebrowser.app/00000000-0000-0000-0000-000000000000 +https://relay.api.vibebrowser.app/mcp/00000000-0000-0000-0000-000000000000 ``` ---- - -## Implementation Plan - -### Phase 1: Relay Client (fork + relay layer) - -1. Fork `chrome-devtools-mcp` → `vibe-relay-client` -2. Add WebSocket client that connects outbound to relay backend -3. Implement tool call receive → execute → respond loop -4. Add `login`, `connect`, `status` CLI commands -5. Add config persistence (~/.vibe-relay/) -6. Test locally with a mock relay server - -### Phase 2: Relay Backend (cloud service) - -1. Minimal relay server (Node.js/Bun, deployable to Fly.io or AKS) -2. WebSocket handler for relay client connections -3. REST API for tool calls with bearer auth -4. Token validation + session management -5. Timeout + error handling -6. Deploy behind `relay.vibebrowser.com` - -### Phase 3: OpenClaw Skill - -1. Write skill that wraps relay REST API calls -2. Include setup instructions (install relay client, get token, connect) -3. Test with real OpenClaw tenant -4. Ship in OpenClawBot repo or as standalone installable skill - -### Phase 4: Polish - -1. Dashboard UI for token management -2. Session monitoring (connected browsers, active tools) -3. Tool-level permissions -4. Usage analytics -5. Documentation site / blog post - ---- +Treat the complete URL like a password. Do not commit it, publish it, place it in shared logs, or use a real value in documentation. Regenerate the UUID in extension settings if it is exposed. -## Open Questions +Hosted production supports OAuth `Authorization: Bearer`, `X-Remote-Session`, legacy bearer compatibility, and path-UUID routing as described above. By contrast, `src/connection.ts` validates browser-facing relay WS/WSS URLs and rejects userinfo, query strings, and fragments; that source rule does not describe hosted MCP authentication. -1. **Relay backend hosting** — Fly.io (simple, edge-deployed) vs AKS sidecar - (co-located with OpenClaw tenants, lower latency)? -2. **Binary vs screenshot transport** — should screenshots go through the relay - as base64, or should the relay client upload to object storage and return a URL? -3. **Multi-tab support** — should one relay client session expose all Chrome tabs, - or should users explicitly attach to specific tabs? -4. **Extension integration** — should the Vibe browser extension be able to act - as a relay client directly (no separate process)? +TLS protects hosted HTTPS/WSS traffic in transit. By default, local relay listeners and the unauthenticated local HTTP MCP endpoint bind to `127.0.0.1`; never override the HTTP bind for public exposure or place it behind a public reverse proxy. ---- +## Test Boundaries -## References +The hermetic E2E sub-suite run by `npm test` and `npm run test:ci` is exactly: `test:e2e:relay-race`, `test:e2e:relay-roundtrip`, `test:e2e:cli-relay`, `test:e2e:cli-autospawn`, `test:e2e:http`, `test:e2e:uuid-only-auth`, `test:e2e:tool-annotations`, `test:e2e:tools-list-budget`, `test:e2e:browser-cli`, `test:e2e:cli-package`, and `test:e2e:devtools-flag`. These package scripts are not all CI workflow gates. -- [chrome-devtools-mcp](https://github.com/ChromeDevTools/chrome-devtools-mcp) — base for relay client -- [Chrome DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/) — CDP spec -- [Puppeteer](https://pptr.dev/) — Chrome automation library used by chrome-devtools-mcp -- [OpenClaw Skills](https://github.com/openclaw/openclaw) — skill system documentation -- [Vibe Browser Extension](https://github.com/AnomalyCo/AnomalyBrowser) — existing Vibe browser tools +`test:e2e:browser-cli-live` and `test:e2e:agents` are excluded because they require a deliberately prepared real extension/browser or harness and external agent runtimes. Fake-extension coverage does not prove live browser behavior, and package scripts do not prove a workflow has executed. See `docs/eval.md` for the current matrix. diff --git a/docs/eval.md b/docs/eval.md index 307e21f..c7a6dba 100644 --- a/docs/eval.md +++ b/docs/eval.md @@ -1,329 +1,116 @@ -# vibebrowser-mcp Evaluation Process +# Evaluation Process -This document tracks the current validation matrix for the `vibebrowser-mcp` and `vibebrowser-cli` binaries, with an explicit split between: +This document separates hermetic repository checks, environment-dependent checks, CI workflow gates, and cross-repository behavioral evaluation. Script definitions prove intended coverage, not that a workflow ran. Output wording and timing need not be deterministic. -- local workspace validation -- packed package artifact validation (`npm pack`) -- published npm validation (`@vibebrowser/mcp@latest`) -- real-extension browser evals versus fake-extension protocol evals +## Hermetic E2E Sub-Suite -Command convention for this document: +`npm test` and `npm run test:ci` run the same hermetic E2E sub-suite. They are not all of CI. The current suite contains exactly these scripts: -- Direct package exec (preferred for MCP server): `npx -y @vibebrowser/mcp@latest ...` -- Explicit bin aliases (backward compatibility checks): `npx -y -p @vibebrowser/mcp@latest vibebrowser-mcp ...` and `npx -y -p @vibebrowser/mcp@latest vibe-mcp ...` +| Script | Boundary covered | +|---|---| +| `test:e2e:relay-race` | extension socket replacement and stale-close race | +| `test:e2e:relay-roundtrip` | agent `call_tool` -> extension -> `tool_result` round trip | +| `test:e2e:cli-relay` | direct browser CLI through the relay and a fake extension | +| `test:e2e:cli-autospawn` | local relay daemon startup and reuse | +| `test:e2e:http` | local Streamable HTTP MCP transport | +| `test:e2e:uuid-only-auth` | fake WS relay verifies browser CLI sends no `Authorization` for UUID-only outbound WS; browser and MCP CLIs reject removed `--remote-secret`; `VIBE_REMOTE_SECRET` is inert | +| `test:e2e:tool-annotations` | MCP tool annotations | +| `test:e2e:tools-list-budget` | bounded startup `tools/list` response | +| `test:e2e:browser-cli` | direct browser CLI command and relay routing behavior | +| `test:e2e:cli-package` | packaged CLI binary smoke coverage | +| `test:e2e:devtools-flag` | explicit `--devtools` backend selection | -Evaluation date: **March 26, 2026 (America/Los_Angeles)**. - -## Coverage Matrix - -| Surface | Harness | Source Modes | Backend | What It Proves | -|---|---|---|---|---| -| Relay race regression | `npm run test:e2e:relay-race` | local | fake extension socket | relay preserves in-flight tool calls across extension reconnects | -| HTTP MCP transport | `npm run test:e2e:http` | local | fake extension socket | streamable HTTP MCP path works end to end | -| Remote relay second-factor auth | `npm run test:e2e:remote-auth` | local | fake remote relay with auth gate | Bearer-token handshake on MCP/CLI clients, rejection on missing/wrong token, legacy tokenless compatibility, and token redaction in outputs | -| OpenClaw-compatible browser CLI | `npm run test:e2e:browser-cli` | `local`, `pack`, `npm` | fake extension socket | `vibebrowser-cli` command shape, JSON output, and tool routing work end to end | -| Live browser CLI regression | `npm run test:e2e:browser-cli-live` | local | real extension session | validates `open`/`snapshot` behavior on a real URL against the connected Vibe extension | -| Codex + OpenCode MCP bridge | `npm run test:e2e:agents` | `local`, `pack`, `npm` | real extension session | packaged `vibebrowser-mcp` can be launched by Codex/OpenCode tooling and route MCP traffic to a real Vibe-connected browser | -| OpenCode financial eval (`../vibe`) | `node tests/mcp-eval.test.js --skip-build --model github-copilot/gpt-4.1 --mcp-source ...` | `auto`, `local`, `pack`, `npm` | harness-managed browser + extension | full OpenCode browser task execution against the Vibe extension | - -Important scope note: -- There is **not yet** a full hosted OpenClaw runtime eval in this repo. -- Current OpenClaw coverage is the **OpenClaw-compatible CLI surface** (`vibebrowser-cli`) plus the `openclaw` helper and HTTP bridge docs. -- Do not claim full hosted OpenClaw runtime parity based only on the CLI test. - -## Source Selectors - -### Browser CLI harness - -`scripts/e2e-browser-cli.mjs` now supports: - -- `E2E_BROWSER_CLI_SOURCE=local` -- `E2E_BROWSER_CLI_SOURCE=pack` -- `E2E_BROWSER_CLI_SOURCE=npm` - -Optional override: - -- `E2E_BROWSER_CLI_PACKAGE=/absolute/or/relative/path/to/package.tgz` - -`pack` mode creates a temporary tarball with `npm pack --json --pack-destination ...` and runs: - -```bash -npx -y --package vibebrowser-cli ... -``` - -### MCP agent harness - -`scripts/e2e-mcp-agents.mjs` now supports: - -- `E2E_MCP_SOURCE=local` -- `E2E_MCP_SOURCE=pack` -- `E2E_MCP_SOURCE=npm` - -Optional override: - -- `E2E_MCP_PACKAGE=/absolute/or/relative/path/to/package.tgz` - -`pack` mode creates a temporary tarball and runs: +The UUID-only test does not validate hosted `/mcp` authentication, OAuth, `X-Remote-Session`, bare UUID normalization, or general URL-form rejection. ```bash -npx -y --package vibebrowser-mcp ... -``` - -### Cross-repo OpenCode eval - -`../vibe/tests/mcp-eval.test.js` now accepts: - -- `--mcp-source auto` -- `--mcp-source local` -- `--mcp-source pack` -- `--mcp-source npm` -- `--mcp-package ` - -Use `--mcp-source pack` when validating a release candidate before publish. - -## Required Commands - -### 1. Local regression suite - -```bash -cd /Users/engineer/workspace/vibebrowser/vibe-mcp -npm run build npm test +npm run test:ci ``` -Pass signal: +## Environment-Dependent Checks -- `npm test` exits `0` -- output contains: - - `e2e ok` - - `http e2e ok` - - `remote auth e2e ok` - - `browser cli e2e ok` +These are excluded from the hermetic sub-suite: -### 2. Packaged CLI artifact validation +| Script | External dependency | +|---|---| +| `test:e2e:browser-cli-live` | a reachable real Vibe extension and browser page | +| `test:e2e:agents` | supported agent CLIs plus a reachable extension or harness-managed browser | ```bash -cd /Users/engineer/workspace/vibebrowser/vibe-mcp -E2E_BROWSER_CLI_SOURCE=pack node scripts/e2e-browser-cli.mjs -``` - -Pass signal: - -- output contains `browser cli e2e ok` -- `vibebrowser-cli` is installed from a `.tgz` package artifact, not from the workspace - -### 3. Packaged binary smoke check - -```bash -cd /Users/engineer/workspace/vibebrowser/vibe-mcp -TMP_DIR="$(mktemp -d)" -npm pack --json --pack-destination "$TMP_DIR" -npx -y --package "$TMP_DIR"/vibebrowser-mcp-*.tgz vibebrowser-mcp --help -npx -y --package "$TMP_DIR"/vibebrowser-mcp-*.tgz vibebrowser-cli --help -``` - -Pass signal: - -- `vibebrowser-mcp --help` prints the branded CLI -- `vibebrowser-cli --help` prints the standalone OpenClaw-compatible CLI - -### 4. Published npm CLI validation - -```bash -cd /Users/engineer/workspace/vibebrowser/vibe-mcp -npx -y @vibebrowser/mcp@latest --version -npx -y -p @vibebrowser/mcp@latest vibebrowser-mcp --version -npx -y -p @vibebrowser/mcp@latest vibe-mcp --version -npx -y -p @vibebrowser/mcp@latest vibebrowser-cli --version -E2E_BROWSER_CLI_SOURCE=npm node scripts/e2e-browser-cli.mjs -``` - -Pass signal: - -- direct package invocation and all aliases print the published version -- `browser cli e2e ok` proves `vibebrowser-cli` works from the npm registry package, not only from a local tarball - -### 5. Real-extension agent eval - -```bash -cd /Users/engineer/workspace/vibebrowser/vibe-mcp -E2E_MCP_SOURCE=pack node scripts/e2e-mcp-agents.mjs -``` - -Pass signal: - -- output contains `e2e ok` -- Codex uses `vibe-browser.*` tools -- OpenCode can resolve the same MCP config and report `vibe-browser connected` - -Hard requirement: - -- a live Vibe extension session must already be connected or connectable on the relay path -- this harness does **not** prove anything if the extension is absent -- default Codex settings are tuned for current CLI compatibility: - - `E2E_CODEX_MODEL=gpt-5` - - `E2E_CODEX_REASONING_EFFORT=low` - - `E2E_CODEX_TIMEOUT_MS=480000` - -### 5a. Live browser CLI regression (real extension) - -```bash -cd /Users/engineer/workspace/vibebrowser/vibe-mcp npm run test:e2e:browser-cli-live +npm run test:e2e:agents +E2E_DEBUG=1 npm run test:e2e:agents ``` -Optional env overrides: +The live browser check defaults to `https://x.com/search?q=("YC W26" OR "YC Demo Day" OR "W26 Demo Day")&src=typed_query&f=live`, a `60000ms` command timeout, and `Require snapshot: false`. It passes when the process exits `0` and prints `live browser cli e2e ok`. The agents check passes only when the process exits `0` and prints `e2e ok`; the debug form exercises the same check with relay diagnostics. Both checks require a deliberately prepared real-browser or harness environment and are excluded from `test:ci`. -- `BROWSER_LIVE_URL` (default: X search URL from issue repro) -- `BROWSER_LIVE_TIMEOUT_MS` (default: `60000`) -- `BROWSER_LIVE_MIN_CONTENT_CHARS` (default: `200`) +A meaningful relay E2E observes `call_tool` sent by the agent, received by the extension, answered with `tool_result`, and read by the MCP client. `list_tools` success alone is insufficient. -Pass signal: +## CI Workflow Gates -- output contains `live browser cli e2e ok` -- output prints measured `open latency`, content lengths, and matched `pageId` -- test proves real-extension `open` + page-content + `snapshot --page-id` flow (not mocked) +`.github/workflows/ci.yml` runs these gates in order: -### 5b. Latest built extension eval via Chrome for Testing +1. `npm ci` +2. `npm run build` +3. `npx tsc --noEmit` +4. `npm run validate:skill` +5. `npm run test:ci` +6. `npm install -g @anthropic-ai/claude-code` (networked) +7. `npm run test:e2e:plugin-bundle` -Build the dev extension first: +The package scripts do not prove that the GitHub Actions workflow executed. -```bash -cd /Users/engineer/workspace/vibebrowser/vibe -npm run build:extension:dev -``` +## Cross-Repository Behavioral Eval -Use the direct script form when debugging the isolated path, especially from a detached worktree where `../vibe` discovery may not point at the intended sibling repo: +The full MCP behavioral eval lives in sibling repository `../vibe`. Use `gpt-4.1` for parity with the current free Copilot setup: ```bash -cd /Users/engineer/workspace/vibebrowser/vibe-mcp -E2E_DEBUG=1 \ -E2E_TEST_BROWSER=1 \ -E2E_VIBE_REPO_ROOT=/Users/engineer/workspace/vibebrowser/vibe \ -E2E_TEST_EXTENSION_PATH=/Users/engineer/workspace/vibebrowser/vibe/dist/extension/dev \ -E2E_MCP_SOURCE=npm \ -node scripts/e2e-mcp-agents.mjs +cd ../vibe && node tests/mcp-eval.test.js --skip-build --model github-copilot/gpt-4.1 ``` -Use the package-script entrypoint for the canonical repo-level check: - -```bash -cd /Users/engineer/workspace/vibebrowser/vibe-mcp -E2E_TEST_BROWSER=1 \ -E2E_VIBE_REPO_ROOT=/Users/engineer/workspace/vibebrowser/vibe \ -E2E_TEST_EXTENSION_PATH=/Users/engineer/workspace/vibebrowser/vibe/dist/extension/dev \ -E2E_MCP_SOURCE=npm \ -npm run test:e2e:agents -``` - -Pass signal: - -- output contains `e2e ok` -- the eval launches a separate Chrome for Testing instance -- the loaded extension comes from `../vibe/dist/extension/dev`, not the user’s daily Chrome profile - -### 6. Full OpenCode browser eval in sibling repo +Supported source selectors are exactly `--mcp-source auto|local|pack|npm`; use `--mcp-package ` to supply the package tested by `pack`. `local` validates workspace source, `pack` validates a release candidate, and `npm` validates the published package. Use `npm` for published-production claims; `auto` is not valid evidence for a production claim. ```bash -cd /Users/engineer/workspace/vibebrowser/vibe -node tests/mcp-eval.test.js --skip-build --model github-copilot/gpt-4.1 --mcp-source pack +cd ../vibe && node tests/mcp-eval.test.js --skip-build --model github-copilot/gpt-4.1 --mcp-source npm ``` -Pass criteria: +Pass criteria are all of: - `MCP External enabled: PASS` - `Relay connected: PASS` - `MCP tools used: PASS` +- MCP tool calls >= 4 +- `FINAL_TABLE marker: PASS` - `Tickers found: 6/6` - process exits `0` -Note: - -- this harness launches its own browser test environment from the `vibe` repo -- use it when you explicitly want the full browser-task eval, not just package smoke coverage - -## Latest Verification Snapshot - -Commands executed in this session: - -| Command | Result | Notes | -|---|---|---| -| `npm run build` | PASS | local TypeScript build succeeded | -| `npm test` | PASS | relay race, HTTP, and local browser CLI e2e all passed | -| `E2E_BROWSER_CLI_SOURCE=pack node scripts/e2e-browser-cli.mjs` | PASS | tarball-installed `vibebrowser-cli` passed end to end | -| `npm pack --json --pack-destination ` | PASS | tarball includes both `dist/cli.js` and `dist/browser-main.js` plus docs/openclaw skill files | -| `npx -y --package vibebrowser-mcp --help` | PASS | branded `vibebrowser-mcp` binary available from package artifact | -| `npx -y --package vibebrowser-cli --help` | PASS | standalone `vibebrowser-cli` binary available from package artifact | -| `gh workflow run "Publish to npm" --ref release/vibebrowser-cli-0.2.5` | PASS | GitHub publish workflow completed successfully | -| `npm view @vibebrowser/mcp version dist-tags.latest bin --json` | PASS | npm `latest` is now `0.2.5` and includes `mcp`, `vibebrowser-mcp`, `vibe-mcp`, and `vibebrowser-cli` | -| `npx -y @vibebrowser/mcp@latest --version` | PASS | direct package invocation resolves to published CLI | -| `npx -y -p @vibebrowser/mcp@latest vibebrowser-mcp --version` | PASS | published explicit binary resolves to `0.2.5` | -| `npx -y -p @vibebrowser/mcp@latest vibe-mcp --version` | PASS | legacy alias still resolves to `0.2.5` | -| `npm run test:e2e:agents` | PASS | local `dist/cli.js` path completed all three MiniWoB tasks with Codex + OpenCode against the live extension session | -| `E2E_MCP_SOURCE=npm npm run test:e2e:agents` | PASS | published npm binary path completed the same three MiniWoB tasks end to end | -| `E2E_DEBUG=1 E2E_TEST_BROWSER=1 E2E_VIBE_REPO_ROOT=/Users/engineer/workspace/vibebrowser/vibe E2E_TEST_EXTENSION_PATH=/Users/engineer/workspace/vibebrowser/vibe/dist/extension/dev E2E_MCP_SOURCE=npm node scripts/e2e-mcp-agents.mjs` | PASS | debug run against a separate Chrome for Testing instance with the unpacked extension build | -| `E2E_TEST_BROWSER=1 E2E_VIBE_REPO_ROOT=/Users/engineer/workspace/vibebrowser/vibe E2E_TEST_EXTENSION_PATH=/Users/engineer/workspace/vibebrowser/vibe/dist/extension/dev E2E_MCP_SOURCE=npm npm run test:e2e:agents` | PASS | canonical repo entrypoint passed against the latest built extension in isolated browser mode | -| `npx -y -p @vibebrowser/mcp@latest vibebrowser-cli --version` | PASS | standalone CLI is now available from npm `latest` | -| `E2E_BROWSER_CLI_SOURCE=npm node scripts/e2e-browser-cli.mjs` | PASS | npm-installed `vibebrowser-cli` passed end to end | -| `npm run test:e2e:browser-cli-live` | PASS | real extension run against X search URL; reported open latency `60718ms`, open content `9119` chars, snapshot content `9564` chars | -| `E2E_MCP_SOURCE=pack node scripts/e2e-mcp-agents.mjs` | FAIL (environment) | timed out waiting for a live Vibe extension connection on the relay path | - -Observed real-extension agent failure: - -```text -Error: Extension did not connect to relay within 120000ms. Ensure Vibe extension has MCP External enabled in the active Chrome profile. -``` - -Interpretation: - -- packaged artifacts are valid locally -- the standalone CLI branding and binaries are correct in the tarball -- published npm `latest` is now updated to `0.2.5` with all intended binaries -- real-agent validation currently depends on a live extension session and was not satisfiable in this shell session - -## Publish Reality - -Current npm state: +Run the final full scenario sweep with: -- `npm view @vibebrowser/mcp version dist-tags.latest bin --json` returns `0.2.5` -- `@latest` now includes: - - `mcp` - - `vibebrowser-mcp` - - `vibe-mcp` - - `vibebrowser-cli` -- publish succeeded via GitHub Actions workflow `Publish to npm` - -Consequences: - -- the new `vibebrowser-mcp` / `vibebrowser-cli` binaries are published to npm and verified from the registry -- the remaining non-green item is the real-extension agent eval, which still requires a live Vibe extension session on the relay path +```bash +cd ../vibe && node tests/eval.test.js --headless --model github-copilot/gpt-4.1 +``` -## Known root causes & regressions +The catalog currently has 28 scenarios. Use `--scenarios`, `--category`, or `--limit` only for debugging, not final verification. -### `tools/list` startup timeout (#14) +## LLM Judge And Langfuse -**Symptom:** Codex/OpenCode intermittently reported `MCP startup failed: timed out -awaiting tools/list after 10s`, or showed `vibe-browser` enabled with 0 tools. +`tests/lib/langfuse-eval.js` expects these environment variable names: -**Root cause:** when the tool cache was empty at startup, the `tools/list` handler -blocked on `refreshTools` (4s) **then** `waitForToolsUpdate` (1.5s) — up to ~5.5s -of in-handler blocking. Stacked on top of relay/extension connection setup, a -single `tools/list` could exceed the client's 10s startup budget, especially when -the extension was connected but had not yet published its tools. +- `LANGFUSE_BASE_URL` +- `LANGFUSE_PUBLIC_KEY` or `LANGFUSE_PUBLIC_KEY_DEV` +- `LANGFUSE_SECRET_KEY` or `LANGFUSE_SECRET_KEY_DEV` +- `AZURE_OPENAI_API_KEY`, falling back to `LITELLM_AZURE_OPENAI_API_KEY` +- `AZURE_OPENAI_ENDPOINT`, with fallback URL environment variables supported by the evaluator +- optional `AZURE_OPENAI_EVAL_MODEL` (default `gpt-4.1-mini`) -**Fix:** the handler now bounds the whole wait with a single -`STARTUP_TOOLS_LIST_BUDGET_MS` (3s) deadline (`src/server.ts`). Whatever is cached -at the deadline is returned (always at least `set_remote`); tools that arrive later -are pushed to the client via the `notifications/tools/list_changed` capability -(already advertised). This caps handler latency well under any client budget -regardless of connection state. +With Azure credentials only, the judge still runs but Langfuse trace logging is disabled. Review traces named `eval-mcp-eval-google-finance` (current) and `eval-mcp-eval-morningstar` (legacy). Inspect `task_completion`, judge comments about missing data, and the corresponding `../vibe/.test/.../logs/opencode-response.txt` artifact. -**Regression test:** `npm run test:e2e:tools-list-budget` -(`scripts/e2e-tools-list-startup-budget.mjs`) — a fake relay reports the extension -connected but never answers `list_tools`; the test asserts `tools/list` returns in -< 4.5s with `set_remote` present. Runs in CI via `npm run test:ci`. +## Latest Preserved Cross-Repository Profile -## Tracking +This is historical evidence, not a rerun for the current documentation change. At commit `7d4173f` dated 2026-03-05: -- Tracking issue: `VibeTechnologies/vibe-mcp#22` -- Release PR: `VibeTechnologies/vibe-mcp#23` +- Published-source `E2E_MCP_SOURCE=npm npm run test:e2e:agents` passed with `e2e ok`. +- The cross-repository production-source financial eval passed OpenCode and Codex at 6/6, with recorded score 1. +- One startup attempt ended with transient `Connection closed`; the immediate rerun passed. +- A local live-extension run can fail preflight when no extension socket is connected. +- The preserved profile records a score of 1 and complete ticker coverage (6/6); no newer Langfuse result is asserted here.