From 4cafaca5ef8cac4cc8b021f95cf39b4c495ead3d Mon Sep 17 00:00:00 2001 From: Minsu Lee Date: Wed, 24 Jun 2026 23:22:46 +0900 Subject: [PATCH] feat: add agent-facing CLI skills Add two skills for building and evaluating agent-facing CLIs: - axi: Agent eXperience Interface (AXI) ergonomic standards for building CLI tools agents use via shell execution (installed via skills manager, tracked in skills-lock.json with .claude/skills/axi symlink) - agent-dx-cli-scale: 0-21 scoring rubric for evaluating how well a CLI is designed for AI agents --- .agents/skills/axi/SKILL.md | 231 +++++++++++++++++++++++++++++ .claude/skills/axi | 1 + skills-lock.json | 11 ++ skills/agent-dx-cli-scale/SKILL.md | 114 ++++++++++++++ 4 files changed, 357 insertions(+) create mode 100644 .agents/skills/axi/SKILL.md create mode 120000 .claude/skills/axi create mode 100644 skills-lock.json create mode 100644 skills/agent-dx-cli-scale/SKILL.md diff --git a/.agents/skills/axi/SKILL.md b/.agents/skills/axi/SKILL.md new file mode 100644 index 0000000..84cec72 --- /dev/null +++ b/.agents/skills/axi/SKILL.md @@ -0,0 +1,231 @@ +--- +name: axi +description: > + Agent eXperience Interface (AXI) — ergonomic standards for building CLI tools that agents + use via shell execution. Use when building, modifying, or reviewing any agent-facing CLI. +--- + +# Agent eXperience Interface (AXI) + +AXI defines ergonomic standards for building CLI tools that autonomous agents interact with through shell execution. + +## Before you start + +Read the [TOON specification](https://toonformat.dev/reference/spec.html) before building any AXI output. + +## 1. Token-efficient output + +Use [TOON](https://toonformat.dev/) (Token-Oriented Object Notation) as the output format on stdout. +TOON provides ~40% token savings over equivalent JSON while remaining readable by agents. +Convert to TOON at the output boundary — keep internal logic on JSON. + +``` +tasks[2]{id,title,status,assignee}: + "1",Fix auth bug,open,alice + "2",Add pagination,closed,bob +``` + +## 2. Minimal default schemas + +Every field in stdout costs tokens — multiplied by row count in collections. +Default to the smallest schema that lets the agent decide what to do next: typically an identifier, a title, and a status. + +- Default list schemas: 3-4 fields, not 10 +- Default limits: high enough to cover common cases in one call (if most repos have <100 labels, default to 100, not 30) +- Long-form content (bodies, descriptions) belongs in detail views, not lists +- Offer a `--fields` flag to let agents request additional fields explicitly + +## 3. Content truncation + +Detail views often contain large text fields. Omitting them forces agents to hunt; including them wastes tokens. +Truncate by default and tell the agent how to get the full version. + +``` +task: + number: 42 + title: Fix auth bug + state: open + body: First 500 chars of the issue body... + ... (truncated, 8432 chars total) +help[1]: Run `tasks view 42 --full` to see complete body +``` + +- Never omit large fields entirely — include a truncated preview +- Show the total size so the agent knows how much it's missing +- Suggest the escape hatch (`--full`) only when content is actually truncated +- Choose a truncation limit that covers most use cases (500-1500 chars) + +## 4. Pre-computed aggregates + +The most expensive token cost is often not a longer response — it's a follow-up call. If your backend has data that agents commonly need as a next step, compute it and include it. + +**Aggregate counts**: include the **total count** in list output, not just the page size. Agents need "how many are there?" and will paginate if the answer isn't definitive. + +``` +count: 30 of 847 total +tasks[30]{number,title,state}: + 1,Fix auth bug,open + ... +``` + +**Derived status fields**: when the next step almost always involves checking related state, include a lightweight summary inline. + +``` +task: + number: 42 + title: Deploy pipeline fix + state: open + checks: 3/3 passed + comments: 7 +``` + +Only include derived fields your backend can provide cheaply — a summary ("3/3 passed"), not the full data. + +## 5. Definitive empty states + +When the answer is "nothing", say so explicitly. Ambiguous empty output causes agents to re-run with different flags to verify. + +``` +$ tasks list --state closed +tasks: 0 closed tasks found in this repository +``` + +State the zero with context. Make it clear the command succeeded — the absence of results is the answer. + +## 6. Structured errors & exit codes + +### Idempotent mutations + +Don't error when the desired state already exists. If the agent closes something already closed, acknowledge and move on with exit code 0. Reserve non-zero exit codes for situations where the agent's intent genuinely cannot be satisfied. + +``` +$ tasks close 42 +task: #42 already closed (no-op) # exit 0 +``` + +### Structured errors on stdout + +Errors go to **stdout** in the same structured format as normal output, so the agent can read and act on them. Include what went wrong and an actionable suggestion. Never let raw dependency output (API errors, stack traces) leak through. + +``` +error: --title is required +help: tasks create --title "..." [--body "..."] +``` + +- Validate required flags before calling any dependency +- Translate errors — extract actionable meaning, discard noise +- Never leak dependency names — suggestions reference your CLI's commands, not the underlying tool + +### No interactive prompts + +Every operation must be completable with flags alone. If a required value is missing, fail immediately with a clear error — don't prompt for it. Suppress prompts from wrapped tools. + +### Output channels + +- **stdout**: all structured output the agent consumes — data, errors, suggestions +- **stderr**: debug logging, progress indicators, diagnostics (agents don't read this) +- **Exit codes**: 0 = success (including no-ops), 1 = error, 2 = usage error + +Never mix progress messages into stdout. An agent that reads "Fetching data..." will try to interpret it as data. + +## 7. Ambient context via session integrations + +Register your tool into the agent's session lifecycle so every conversation starts with relevant state already visible — before the agent takes any action. + +**Pattern:** + +1. Provide an explicit setup command that installs or repairs a session hook or plugin after user intent is clear +2. At session start, the integration runs your tool and provides a compact dashboard as context +3. The agent receives this as initial context and can act immediately + +``` +# Agent sees this at session start — no invocation needed: +specs[2]{id,title,status}: + 1,Fix auth bug,open + 2,Add pagination,in-progress + +help[2]: + Run `mytool specs view 1` for details + Run `mytool specs create --title "..."` to add a spec +``` + +**Rules:** + +- **Default app targets**: by default, support Claude Code, Codex, and OpenCode. Do not hard-code a single agent integration when the tool can reasonably support multiple agents +- **Explicit opt-in**: register hooks or plugins only from a user-invoked setup command, not from ordinary CLI commands +- **Portable commands**: hook commands should use a PATH-verified binary name when it resolves to the current executable, and fall back to the full absolute path otherwise. This keeps global installs portable while ensuring hooks do not accidentally run a different binary +- **Path repair**: setup commands should check existing hooks and update the executable path if it has changed (e.g., after reinstall or relocation) +- **Idempotent**: repeated installs with the same path are silent no-ops +- **Directory-scoped**: show only state relevant to the current working directory +- **Token-budget-aware**: this context loads on _every_ session — ruthlessly minimize it. Include just enough for the agent to orient and act; deep data belongs in explicit invocations +- **Lifecycle capture**: use session-end hooks to capture what happened (transcripts, files touched, specs referenced) so future session-start context gets richer over time + +**How to integrate with each app:** + +- **Claude Code**: use native hooks in `~/.claude/settings.json` or project `.claude/settings.json`. Prefer `SessionStart` to inject compact context via stdout +- **Codex**: use native hooks in `~/.codex/hooks.json` or `/.codex/hooks.json`, and ensure `[features].hooks = true` in `config.toml`. Prefer `SessionStart` for ambient context via stdout +- **OpenCode**: use a managed plugin in `~/.config/opencode/plugins/`. Prefer ambient system-context injection for the home view rather than adding a custom tool + +**Also ship an installable skill (secondary recommendation):** + +The session hook is the primary integration, but it only helps agents whose harness supports hooks, and it loads on _every_ session. +Offer an installable [Agent Skill](https://agentskills.io) as a secondary discovery path. +It loads on demand when the agent recognizes a matching task, carries no per-session token cost, and works in any agent that supports the skill format. +Recommend the hook first (ambient context plus live state) and the skill second (lower overhead, broader agent support) - they are complementary, and a user installs whichever fits, or both. + +```sh +npx skills add / --skill +``` + +- **Single source of truth**: generate `SKILL.md` from the same content your no-args home view prints, so the skill never drifts from the CLI's own guidance. Add a `--check` build step to CI that fails if the committed skill is stale +- **Strip live state**: a skill is static, so omit dynamic data (open sessions, current items) that only the hook can show +- **Non-interactive commands**: rewrite command examples to a form the agent can run without a global install (e.g. `npx -y mytool ...`), since a skill may be installed without the binary on PATH +- **Trigger-shaped frontmatter**: include `name` and a `description` written as a trigger — terse and outcome-focused so the agent loads it on the right intent +- **Document both paths**: in your README, present the hook and the skill as two ways to achieve the same thing, and make clear the user only needs one + +## 8. Content first + +Running your CLI with no arguments should show the most relevant live content — not a usage manual. +When an agent sees actual state it can act immediately. When it sees help text, it has to make a second call. + +``` +$ tasks +tasks[3]{id,title,status}: + 1,Fix auth bug,open + 2,Add pagination,open + 3,Update docs,closed +help[2]: + Run `tasks view ` to see full details + Run `tasks create --title "..."` to add a task +``` + +## 9. Contextual disclosure + +Include **a few next steps** that follow logically from the current output. +The agent discovers your CLI's surface area organically by using it, not by reading a manual upfront. + +Rules: + +- **Relevant**: after an open item → suggest closing; after an empty list → suggest creating; after a list → suggest viewing +- **Actionable**: every suggestion is a complete command (or template) carrying forward any disambiguating flags from the current invocation (e.g., `--repo`, `--source`) +- **Parameterize dynamic values**: when a suggested command needs a runtime value such as an ID, title, branch, URL, or path, use placeholders like `` or `""` instead of guessing a concrete value that may mislead the agent +- **Omit when self-contained**: when the output fully answers the query (a detail view, a count, a confirmation), suggestions are noise — leave them out. Include them on list and mutation responses where the next step isn't obvious. +- **Guide discovery, not workflows**: suggest a variety of possible next actions, don't prescribe a fixed sequence. An agent that already knows what it wants should never be nudged into an extra step. +- **Reveal truncated lists**: when a list shows only the most recent N items out of a larger total, add a help hint telling the agent how to see all of them (e.g., `Run 'mytool list' for all 47 items`). Don't encode pagination into TOON array headers — use help hints instead. +- **Resolve errors**: on errors, suggest the specific command that fixes the problem, not "see `--help`" + +## 10. Consistent way to get help + +The top-level home view should also identify the tool itself before the live data: + +- Include the absolute path of the current executable, with the user's home directory collapsed to `~` +- Include a one-sentence description of what this AXI does + +``` +$ tasks +bin: ~/.local/bin/tasks +description: Manage project tasks in the current workspace +... +``` + +Every subcommand should support `--help` with a concise, complete reference: available flags with defaults, required arguments, and 2-3 usage examples. Keep it focused on the requested subcommand — don't dump the entire CLI's manual. diff --git a/.claude/skills/axi b/.claude/skills/axi new file mode 120000 index 0000000..51a5e82 --- /dev/null +++ b/.claude/skills/axi @@ -0,0 +1 @@ +../../.agents/skills/axi \ No newline at end of file diff --git a/skills-lock.json b/skills-lock.json new file mode 100644 index 0000000..d02aa0b --- /dev/null +++ b/skills-lock.json @@ -0,0 +1,11 @@ +{ + "version": 1, + "skills": { + "axi": { + "source": "kunchenguid/axi", + "sourceType": "github", + "skillPath": ".agents/skills/axi/SKILL.md", + "computedHash": "e3993d4b3be5c816b83a0957951906d3223dd4e6df2f19869eef6607170b6838" + } + } +} diff --git a/skills/agent-dx-cli-scale/SKILL.md b/skills/agent-dx-cli-scale/SKILL.md new file mode 100644 index 0000000..23748df --- /dev/null +++ b/skills/agent-dx-cli-scale/SKILL.md @@ -0,0 +1,114 @@ +--- +name: agent-dx-cli-scale +description: A scoring scale for evaluating how well a CLI is designed for AI agents, based on the "Rewrite Your CLI for AI Agents" principles. +--- + +# Agent DX CLI Scale + +Use this skill to **evaluate any CLI** against the principles of agent-first design. Score each axis from 0–3, then sum for a total between 0–21. + +> Human DX optimizes for discoverability and forgiveness. +> Agent DX optimizes for predictability and defense-in-depth. +> — [You Need to Rewrite Your CLI for AI Agents](https://justin.poehnelt.com/posts/rewrite-your-cli-for-ai-agents) + +--- + +## Scoring Axes + +### 1. Machine-Readable Output + +Can an agent parse the CLI's output without heuristics? + +| Score | Criteria | +| ----- | ----------------------------------------------------------------------------------------------------- | +| 0 | Human-only output (tables, color codes, prose). No structured format available. | +| 1 | `--output json` or equivalent exists but is incomplete or inconsistent across commands. | +| 2 | Consistent JSON output across all commands. Errors also return structured JSON. | +| 3 | NDJSON streaming for paginated results. Structured output is the default in non-TTY (piped) contexts. | + +### 2. Raw Payload Input + +Can an agent send the full API payload without translation through bespoke flags? + +| Score | Criteria | +| ----- | ------------------------------------------------------------------------------------------------------------------------------------- | +| 0 | Only bespoke flags. No way to pass structured input. | +| 1 | Accepts `--json` or stdin JSON for some commands, but most require flags. | +| 2 | All mutating commands accept a raw JSON payload that maps directly to the underlying API schema. | +| 3 | Raw payload is first-class alongside convenience flags. The agent can use the API schema as documentation with zero translation loss. | + +### 3. Schema Introspection + +Can an agent discover what the CLI accepts at runtime without pre-stuffed documentation? + +| Score | Criteria | +| ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0 | Only `--help` text. No machine-readable schema. | +| 1 | `--help --json` or a `describe` command for some surfaces, but incomplete. | +| 2 | Full schema introspection for all commands — params, types, required fields — as JSON. | +| 3 | Live, runtime-resolved schemas (e.g., from a discovery document) that always reflect the current API version. Includes scopes, enums, and nested types. | + +### 4. Context Window Discipline + +Does the CLI help agents control response size to protect their context window? + +| Score | Criteria | +| ----- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0 | Returns full API responses with no way to limit fields or paginate. | +| 1 | Supports `--fields` or field masks on some commands. | +| 2 | Field masks on all read commands. Pagination with `--page-all` or equivalent. | +| 3 | Streaming pagination (NDJSON per page). Explicit guidance in context/skill files on field mask usage. The CLI actively protects the agent from token waste. | + +### 5. Input Hardening + +Does the CLI defend against the specific ways agents fail (hallucinations, not typos)? + +| Score | Criteria | +| ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0 | No input validation beyond basic type checks. | +| 1 | Validates some inputs, but does not cover agent-specific hallucination patterns (path traversals, embedded query params, double encoding). | +| 2 | Rejects control characters, path traversals (`../`), percent-encoded segments (`%2e`), and embedded query params (`?`, `#`) in resource IDs. | +| 3 | Comprehensive hardening: all of the above, plus output path sandboxing to CWD, HTTP-layer percent-encoding, and an explicit security posture — _"The agent is not a trusted operator."_ | + +### 6. Safety Rails + +Can agents validate before acting, and are responses sanitized against prompt injection? + +| Score | Criteria | +| ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0 | No dry-run mode. No response sanitization. | +| 1 | `--dry-run` exists for some mutating commands. | +| 2 | `--dry-run` for all mutating commands. Agent can validate requests without side effects. | +| 3 | Dry-run plus response sanitization (e.g., via Model Armor) to defend against prompt injection embedded in API data. The full request→response loop is defended. | + +### 7. Agent Knowledge Packaging + +Does the CLI ship knowledge in formats agents can consume at conversation start? + +| Score | Criteria | +| ----- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0 | Only `--help` and a docs site. No agent-specific context files. | +| 1 | A `CONTEXT.md` or `AGENTS.md` with basic usage guidance. | +| 2 | Structured skill files (YAML frontmatter + Markdown) covering per-command or per-API-surface workflows and invariants. | +| 3 | Comprehensive skill library encoding agent-specific guardrails (_"always use --dry-run"_, _"always use --fields"_). Skills are versioned, discoverable, and follow a standard like OpenClaw. | + +--- + +## Interpreting the Total + +| Range | Rating | Description | +| ----- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------- | +| 0–5 | **Human-only** | Built for humans. Agents will struggle with parsing, hallucinate inputs, and lack safety rails. | +| 6–10 | **Agent-tolerant** | Agents can use it, but they'll waste tokens, make avoidable errors, and require heavy prompt engineering to compensate. | +| 11–15 | **Agent-ready** | Solid agent support. Structured I/O, input validation, and some introspection. A few gaps remain. | +| 16–21 | **Agent-first** | Purpose-built for agents. Full schema introspection, comprehensive input hardening, safety rails, and packaged agent knowledge. | + +--- + +## Bonus: Multi-Surface Readiness + +Not scored, but note whether the CLI exposes multiple agent surfaces from the same binary: + +- [ ] **MCP (stdio JSON-RPC)** — typed tool invocation, no shell escaping +- [ ] **Extension / plugin install** — agent treats the CLI as a native capability +- [ ] **Headless auth** — env vars for tokens/credentials, no browser redirect required