feat: annotate every MCP tool with title + behavioural hints - #125
Merged
Conversation
`tools/list` previously emitted only {name, description, inputSchema}. Both
the Anthropic Connectors directory and the OpenAI plugins directory require a
`title` plus accurate readOnly/destructive/openWorld hints on every tool, so
we failed that check on every tool in every directory — the cheapest of the
two hard blockers named in worklog/mcp-distribution-channels.md (#124).
Annotations are attached in the `ListToolsRequestSchema` handler, the single
choke point where the extension backend, the chrome-use `--devtools` backend,
and the `set_remote` meta-tool all converge on the wire.
Field names and semantics were taken from the current spec
(ToolAnnotationsSchema in @modelcontextprotocol/sdk types.d.ts: title,
readOnlyHint, destructiveHint, idempotentHint, openWorldHint) rather than
from memory, plus the top-level `Tool.title`.
Classification was done by reading each implementation, not by pattern-matching
the tool name. Three calls worth flagging:
- `click` is NOT read-only. It dispatches a real DOM click, which can submit a
form, follow a link to any host, or open a tab — the extension arms a new-tab
tracker specifically for it. Also open-world.
- `wait_for_condition` reads like an observer but evaluates a caller-supplied
JavaScript expression in the page. Arbitrary code execution, so not read-only
and open-world (it can `fetch()`).
- `secrets_manager` reads like a mutator but only supports `list`/`read` of
credential *metadata*, never writes and never returns plaintext. It is
read-only.
Unclassified tools from a future backend get the most cautious combination the
spec allows (writes / destructive / non-idempotent / open-world) so a client
prompts the user, rather than silently going unannotated.
Test: scripts/e2e-tool-annotations.mjs, wired into `test` and `test:ci`.
Given the #1856 false-green deletions, it proves itself in three parts:
A. contract — every exposed tool is classified, and no classification
contradicts the implementation (a mutating tool cannot claim readOnly,
an observer cannot be marked mutating, readOnly+destructive is rejected).
B. self-check — the part-A checks are shown to go red on seven broken
inputs, so a check that cannot fail is caught.
C. wire — boots a real server process and drives it with a real MCP client
over streamable HTTP; only Chrome is faked, advertising the genuine
27-tool core profile. Asserts the annotations are actually on the wire.
Nothing skips silently; missing prerequisites throw.
Version deliberately left at 0.3.2: annotations are runtime behaviour and are
not carried in the registry manifest, so no npm release is required and the
mcpb-vendors-npm release chain that CI rejected in #124 is not triggered.
Also untracks dist/server.js, a build artifact committed by accident (dist/ is
gitignored); leaving it would have meant committing a stale build.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Clears the first of the two hard blockers named in
worklog/mcp-distribution-channels.md(#124): we emitted no tool annotations at all, failing the listing requirement on every tool in every directory.What changed
tools/listused to emit only{name, description, inputSchema}. It now emits a top-leveltitleplus a fullannotationsobject on every tool. Attached in theListToolsRequestSchemahandler — the single choke point where the extension backend, the chrome-use--devtoolsbackend, and theset_remotemeta-tool converge.Field names/semantics taken from the current spec (
ToolAnnotationsSchemain the installed SDK:title,readOnlyHint,destructiveHint,idempotentHint,openWorldHint), not from memory.Classification
Done by reading each implementation. 41 tool names classified (58 registry entries incl. alias spellings); 29 on the wire in the core profile. Counts over the exposed set: readOnly 17 · destructive 8 · openWorld 12 · idempotent 28.
Three calls worth review — all three contradict what the tool's name suggests:
clickwait_for_conditionfetch().secrets_managerlist/readof credential metadata. Never writes, never returns plaintext.Ambiguous ones and how I called them:
press_keyis destructive (Enter submits, Ctrl+W closes a tab);web_fetchis read-only but open-world (reads only, arbitrary host);upload_fileis not read-only and open-world (pushes local file bytes into a page that may ship them anywhere);dragis destructive (can reorder or delete).Unclassified tools from a future backend get the most cautious combination the spec allows, so a client prompts the user rather than silently shipping an unannotated capability.
Test —
scripts/e2e-tool-annotations.mjsWired into
testandtest:ci. Given the #1856 false-green deletions, it proves itself in three parts:readOnly; an observer cannot be marked mutating;readOnly+destructiveis rejected).Nothing skips silently; missing prerequisites throw.
Fail-then-pass evidence
Injected a real unannotated tool into
CHROME_USE_TOOLS, rebuilt:Also proved it catches misclassification, not just absence — flipping
clickback toreadOnlyHint: true:tools/listwire output (verbatim, part C)[ { "name": "take_screenshot", "title": "Take Screenshot", "annotations": { "title": "Take Screenshot", "readOnlyHint": true, "destructiveHint": false, "idempotentHint": true, "openWorldHint": false } }, { "name": "click", "title": "Click Element", "annotations": { "title": "Click Element", "readOnlyHint": false, "destructiveHint": false, "idempotentHint": true, "openWorldHint": true } }, { "name": "wait_for_condition", "title": "Wait For Condition", "annotations": { "title": "Wait For Condition", "readOnlyHint": false, "destructiveHint": true, "idempotentHint": false, "openWorldHint": true } }, { "name": "secrets_manager", "title": "Read Saved Credential Metadata", "annotations": { "title": "Read Saved Credential Metadata", "readOnlyHint": true, "destructiveHint": false, "idempotentHint": true, "openWorldHint": false } }, { "name": "web_fetch", "title": "Fetch Web Page", "annotations": { "title": "Fetch Web Page", "readOnlyHint": true, "destructiveHint": false, "idempotentHint": true, "openWorldHint": true } } ]Build + full suite
Release / version
Version deliberately left at 0.3.2. Annotations are runtime behaviour and are not carried in the registry manifest, so no npm release is needed to make them take effect — and this avoids the
mcpb-vendors-npm release chain that CI correctly rejected on #124's first attempt. The live registry entry needs no republish.Drive-by
Untracks
dist/server.js, a build artifact committed by accident (dist/is gitignored). Leaving it tracked would have meant committing a stale build alongside this change.