Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 18 additions & 39 deletions packages/dsh-plugin-browserskill/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,41 +5,20 @@ npm: [`@wxg-prc-cpg/browser-skill-dsh-plugin`](https://www.npmjs.com/package/@wx
A [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (dsh) tool plugin that exposes
[BrowserSkill](https://github.com/Tencent/BrowserSkill) (`bsk`) browser automation to the model.

Each tool maps to one `bsk <cmd> --json` invocation: the plugin spawns the bsk CLI, parses its
Each action maps to one `bsk <cmd> --json` invocation: the plugin spawns the bsk CLI, parses its
structured JSON output, and returns a canonical typed value. The bsk daemon, browser, and browser
extension keep owning the actual browser control — this package is a thin, well-typed bridge.

## Tools

| Tool | bsk command | Purpose |
| Tool | Actions | Purpose |
| --- | --- | --- |
| `browser_session_start` | `bsk session start` | Open an Agent Window session; optional initial URL, window size, and device emulation preset. Returns the session id and makes it the current session. |
| `browser_session_stop` | `bsk session stop` | Stop a session (the current one by default) and close its Agent Window. Only plugin-created sessions can be stopped. |
| `browser_session_list` | — (registry only) | List the sessions this plugin created, marking the current one. Foreign daemon sessions are never shown. |
| `browser_navigate` | `bsk navigate` | Navigate the active tab, with `waitUntil` / timeout control. |
| `browser_snapshot` | `bsk snapshot` | Indented aria-tree snapshot with `@eN` refs for interaction tools. |
| `browser_observe` | `bsk observe` | Semantic VOM observation (read-only) with `@eN` refs. |
| `browser_click` | `bsk click` | Click a snapshot ref or CSS selector (button / click-count options). |
| `browser_hover` | `bsk hover` | Hover a ref or selector and allow hover-triggered UI to settle. |
| `browser_fill` | `bsk fill` | Fill an input / textarea / contenteditable (clears first by default). |
| `browser_select` | `bsk select` | Select one or more option values on a `<select>`. |
| `browser_press` | `bsk press` | Dispatch a key or combo, optionally focusing a target first. |
| `browser_screenshot` | `bsk screenshot` | PNG capture of the tab or a ref-cropped element; inlines the image when the deployment supports image input, otherwise returns a file path. |
| `browser_emulate` | `bsk emulate` | Apply or clear mobile device emulation on the active tab. |
| `browser_tab_list` | `bsk tab list` | List user and/or Agent Window tabs visible to an owned session. |
| `browser_tab_create` | `bsk tab create` | Create a foreground or background tab in the Agent Window. |
| `browser_tab_select` | `bsk tab select` | Focus an Agent Window tab. |
| `browser_tab_close` | `bsk tab close` | Close an Agent Window tab. |
| `browser_tab_borrow` | `bsk tab borrow` | Move a user tab into the Agent Window for controlled interaction. |
| `browser_tab_return` | `bsk tab return` | Return a borrowed tab to its original user window. |
| `browser_navigate_back` / `browser_navigate_forward` | `bsk navigate-back` / `bsk navigate-forward` | Traverse tab history and wait for a lifecycle phase. |
| `browser_reload` | `bsk reload` | Reload the active tab, optionally bypassing cache. |
| `browser_wait_for_navigation` | `bsk wait-for-navigation` | Wait explicitly for a page lifecycle event. |
| `browser_request_help` | `bsk request-help` | Pause for login, captcha, OTP, confirmation, or another human browser step. |
| `browser_get_html` | `bsk get-html` | Read raw document or ref-scoped HTML after semantic observation is insufficient. |
| `browser_console` | `bsk console` | Read buffered console/log/exception entries without evaluating JavaScript. |
| `browser_network` | `bsk network` | Read buffered network response/failure metadata. |
| `browser_window_resize` | `bsk window resize` | Resize the owned session's Agent Window. |
| `browser_session` | `start`, `stop`, `list` | Manage plugin-owned Agent Window sessions. |
| `browser_page` | `navigate`, `back`, `forward`, `reload`, `wait` | Navigate the active tab and wait for page lifecycle events. |
| `browser_inspect` | `observe`, `snapshot`, `html`, `screenshot`, `console`, `network` | Read semantic or diagnostic page state and capture screenshots. |
| `browser_interact` | `click`, `hover`, `fill`, `select`, `press` | Interact with controls using fresh refs or selectors. |
| `browser_tabs` | `list`, `create`, `select`, `close`, `borrow`, `return` | Manage Agent Window tabs and temporarily borrow user tabs. |
| `browser_assist` | `resize`, `emulate`, `request-help` | Resize or emulate the browser and pause for human-only steps. |

`bsk evaluate` and `bsk record` are intentionally not exposed by this plugin: arbitrary page
evaluation is a higher-risk capability, while recording is long-running and needs a dedicated
Expand All @@ -63,7 +42,7 @@ seam degrade silently.

One agent conversation can drive several browser sessions at once:

- `browser_session_start` returns the session id and makes it the **current session**.
- `browser_session` with `action: start` returns the session id and makes it the **current session**.
- Every operation tool accepts an optional `session` argument. When omitted, the call acts on the
current session (the one most recently started or used); when given, that session becomes current.
- Every tool result echoes the session it actually acted on, so the model never has to guess.
Expand All @@ -72,9 +51,9 @@ One agent conversation can drive several browser sessions at once:

**Ownership boundary**: the bsk daemon may be shared with other agents, terminals, or dsh
instances. The plugin therefore only ever sees and operates on sessions it created itself —
an explicit `session` argument naming a foreign or unknown id is rejected, `browser_session_list`
shows plugin-created sessions only (no daemon-wide view), and stop/unload cleanup can never touch
a session owned by another program.
an explicit `session` argument naming a foreign or unknown id is rejected, the `list` action on
`browser_session` shows plugin-created sessions only (no daemon-wide view), and stop/unload cleanup
can never touch a session owned by another program.

## Installation

Expand Down Expand Up @@ -109,7 +88,7 @@ All fields are optional and validated through the plugin's Schemastery `Config`:
# lazyTools: true # reveal browser_* tools only after the skill is invoked
```

- **`lazyTools` (default `true`)** — the final progressive-disclosure stage: the 28
- **`lazyTools` (default `true`)** — the final progressive-disclosure stage: the six
`browser_*` tool schemas stay OUT of the system prompt (zero schema tokens) until the
`browser-skill` skill is actually invoked — the skill catalog entry is the only
advertisement. One successful invocation (model tool call, or a `/browser-skill` user
Expand All @@ -131,7 +110,7 @@ like the BrowserSkill extension without leaking a single selector into the host
shell's theme cannot bleed back in.

- **Lifecycle**: hidden while the plugin owns no sessions; appears on the first
`browser_session_start`; disappears when all sessions stop (or the plugin unloads).
`browser_session` start action; disappears when all sessions stop (or the plugin unloads).
- **Focus view**: status row (green/idle/red dot + session + action + mm:ss), the latest page
frame (refreshes every ~1.5s while active, ~8s when idle; the last good frame stays on
stage while the next one loads, and is kept on errors so the card does not flash),
Expand Down Expand Up @@ -173,11 +152,11 @@ shell's theme cannot bleed back in.
card). Screenshots additionally attach the image itself when the host mounts an attachment store
and the active model route declares image input; otherwise the PNG path is returned.
- **Web UI toolview (browser half)**: the package is dual-face. `dsh.client` (platform `web`) ships
`lib/client.cjs`, which registers a keyed `tool.call.toolview` view for `browser_screenshot`. The
custom view keeps the terminal block (command + output) and, when the settled result carries an
image block, resolves the durable attachment through the client session's authorized
`lib/client.cjs`, which registers a keyed `tool.call.toolview` view for `browser_inspect`. The
custom view keeps a terminal block for every inspect action and, when a screenshot result carries
an image block, resolves the durable attachment through the client session's authorized
`readAttachment` RPC and renders it with the shared `MessageImage` thumbnail/lightbox atoms.
Every other `browser_*` tool keeps the stock terminal card. The bundle follows the dsh client
The other five tools keep the stock terminal card. The bundle follows the dsh client
contract: a CJS closure factory handed to `window.__ModuleLoader__.load`, platform modules
(`react`, `dsh-client-ui-*`) external, everything else inlined, CSS Modules compiled by
lightningcss.
Expand Down
44 changes: 11 additions & 33 deletions packages/dsh-plugin-browserskill/scripts/build-skill-content.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -18,37 +18,15 @@ function stripFrontmatter(markdown) {
}

const SUPPORTED_BROWSER_TOOLS = [
"browser_session_start",
"browser_session_stop",
"browser_session_list",
"browser_navigate",
"browser_navigate_back",
"browser_navigate_forward",
"browser_reload",
"browser_wait_for_navigation",
"browser_snapshot",
"browser_observe",
"browser_get_html",
"browser_screenshot",
"browser_console",
"browser_network",
"browser_click",
"browser_hover",
"browser_fill",
"browser_select",
"browser_press",
"browser_tab_list",
"browser_tab_create",
"browser_tab_select",
"browser_tab_close",
"browser_tab_borrow",
"browser_tab_return",
"browser_request_help",
"browser_emulate",
"browser_window_resize",
"browser_session",
"browser_page",
"browser_inspect",
"browser_interact",
"browser_tabs",
"browser_assist",
];

function validateModelFacingSkill(markdown) {
function validateModelFacingSkill(markdown, supportedTools) {
if (/\bbsk\b/i.test(markdown)) {
throw new Error("DSH skill must not expose the internal CLI name");
}
Expand All @@ -57,9 +35,9 @@ function validateModelFacingSkill(markdown) {
}

const mentioned = new Set(markdown.match(/\bbrowser_[a-z][a-z_]*\b/g) ?? []);
const supported = new Set(SUPPORTED_BROWSER_TOOLS);
const supported = new Set(supportedTools);
const unknown = [...mentioned].filter((name) => !supported.has(name));
const missing = SUPPORTED_BROWSER_TOOLS.filter((name) => !mentioned.has(name));
const missing = supportedTools.filter((name) => !mentioned.has(name));
if (unknown.length > 0) {
throw new Error(`DSH skill mentions unsupported browser tools: ${unknown.join(", ")}`);
}
Expand All @@ -75,11 +53,11 @@ const DESCRIPTION =
"debug page activity, or smoke-test UI changes in a managed Agent Window.";

const content = `${stripFrontmatter(source).trim()}\n`;
validateModelFacingSkill(content);
validateModelFacingSkill(content, SUPPORTED_BROWSER_TOOLS);

const banner =
"// GENERATED by scripts/build-skill-content.mjs — do not edit.\n" +
"// Source: skill/SKILL.md (DeepSeek Harness browser_* tool instructions).\n";
"// Source: skill/SKILL.md (DeepSeek Harness six-tool instructions).\n";

writeFileSync(
join(pkg, "src", "skill-content.generated.ts"),
Expand Down
Loading