Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,9 @@ jobs:
- name: Test recovery MCP product
run: pnpm --dir components/products/mcp test

- name: Test pi adapter
run: pnpm --dir components/adapters/pi test

- name: Test OpenCode adapter
run: pnpm --dir components/adapters/opencode test

Expand Down
43 changes: 26 additions & 17 deletions components/adapters/HOSTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ TokenPilot is now structured as a reusable LightRSI component with multiple host
| `Codex CLI` | available | hooks + local Responses proxy + shared CLI | `npm --prefix components/adapters/codex run build` then `npm --prefix components/adapters/codex run install:codex` | [codex/README.md](./codex/README.md) |
| `Claude Code` | available | gateway routing + observability hooks + shared CLI | `npm --prefix components/adapters/claude-code run build` then `npm --prefix components/adapters/claude-code run install:claude-code` | [claude-code/README.md](./claude-code/README.md) |
| `DeepSeek Harness` | available | Cordis plugin + durable projection | `pnpm --filter @lightrsi/deepseek-harness-adapter compatibility:smoke -- --dsh-checkout=/absolute/path/to/deepseek-harness` | [GUA-06](../../docs/acceptance/GUA-06.md) |
| `pi` | available | in-process extension + native recovery tool + shared CLI | `npm --prefix components/adapters/pi run build` then `npm --prefix components/adapters/pi run install:pi` | [pi/README.md](./pi/README.md) |
| `OpenCode` | available | in-process v1 plugin + recovery MCP + shared CLI | `npm --prefix components/adapters/opencode run build` then `npm --prefix components/adapters/opencode run install:opencode` | [opencode/README.md](./opencode/README.md) |

Each full TokenPilot host adapter binds the versioned preset explicitly. OpenClaw and Codex declare Stabilizer, Reduction, and Eviction; Claude Code currently declares Stabilizer and Reduction. DeepSeek Harness is a separate compatibility adapter with a narrower projection surface. The same adapter-owned product registrations are used by the shared CLI and browser Visual surface where applicable.
Expand All @@ -22,22 +23,22 @@ Legend:
- `partial`: available, but intentionally narrower than the OpenClaw path
- `no`: not supported in the current public adapter

| Capability | OpenClaw | Codex CLI | Claude Code | OpenCode |
| :-- | :--: | :--: | :--: | :--: |
| Stable-prefix rewriting | yes | yes | yes | partial (`developer` target only) |
| Before-call reduction | yes | yes | yes | yes |
| Real MCP-backed `memory_fault_recover` | yes | yes | yes | yes |
| Standalone `lightrsi <host> ...` CLI | yes | yes | yes | yes |
| `status` / `doctor` / `report` | yes | yes | yes | yes |
| `visual` | yes | yes | yes | yes |
| `mode conservative` / `mode normal` | yes | yes | yes | yes |
| `mode aggressive` | yes | no | no | no |
| Estimator-driven lifecycle eviction runtime | yes | yes | yes | yes (durable request overlay) |
| Lifecycle eviction controls | yes | no | no | `on` / `off` / `minBlockChars` |
| In-host slash commands | yes | no | no | no |
| Hook-based observability | partial | yes | yes | in-process |
| Local proxy / gateway runtime | yes | yes | yes | not needed (in-process) |
| Session-state / ux-effects persistence | yes | yes | yes | yes |
| Capability | OpenClaw | Codex CLI | Claude Code | pi | OpenCode |
| :-- | :--: | :--: | :--: | :--: | :--: |
| Stable-prefix rewriting | yes | yes | yes | yes | partial (`developer` target only) |
| Before-call reduction | yes | yes | yes | yes | yes |
| Real MCP-backed `memory_fault_recover` | yes | yes | yes | native tool (pi has no MCP) | yes |
| Standalone `lightrsi <host> ...` CLI | yes | yes | yes | yes | yes |
| `status` / `doctor` / `report` | yes | yes | yes | yes | yes |
| `visual` | yes | yes | yes | yes | yes |
| `mode conservative` / `mode normal` | yes | yes | yes | yes | yes |
| `mode aggressive` | yes | no | no | no | no |
| Estimator-driven lifecycle eviction runtime | yes | yes | yes | yes (native `context_edit`) | yes (durable request overlay) |
| Lifecycle eviction controls | yes | no | no | `on` / `off` / `minBlockChars` | `on` / `off` / `minBlockChars` |
| In-host slash commands | yes | no | no | no | no |
| Hook-based observability | partial | yes | yes | in-process | in-process |
| Local proxy / gateway runtime | yes | yes | yes | not needed (in-process) | not needed (in-process) |
| Session-state / ux-effects persistence | yes | yes | yes | yes | yes |

## Host Notes

Expand Down Expand Up @@ -70,6 +71,14 @@ Legend:
- supports TokenPilot status projection and compatibility smoke verification
- canonical surface eviction remains opt-in and is guarded by estimator, registry, safety, and revision checks

### pi

- in-process pi extension (verified on pi 0.87.1); no proxy, no host config mutation
- stable prefix via structured prompt sections (`before_agent_start`), so pi keeps appending section deltas instead of replacing the prompt
- request-time reduction in the `context` hook; recovery as a native tool with the MCP tool's exact schema
- opt-in eviction through native `context_edit` entries at `turn_end`, before pi's own threshold compaction
- design note: [docs/adapters/pi-design.md](../../docs/adapters/pi-design.md)

### OpenCode

- in-process v1 plugin (verified on OpenCode 1.18.33) plus the shared recovery MCP server in `opencode.json`
Expand All @@ -81,7 +90,7 @@ Legend:
### Shared Visual Surface

- `lightrsi visual` now provides a standalone browser visual entrypoint
- the shared visual can switch between `openclaw`, `codex`, `claude-code`, and `opencode` hosts
- the shared visual can switch between `openclaw`, `codex`, `claude-code`, `pi`, and `opencode` hosts
- today, the browser visual is backed by snapshot data; OpenClaw still has the richest dataset, while Codex and Claude Code now route their `visual` commands into the shared browser surface

## Boundary
Expand Down
3 changes: 3 additions & 0 deletions components/adapters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ Adapter inventory:
- adapter for Claude Code
- `deepseek-harness/`
- Cordis adapter and compatibility smoke path for DeepSeek Harness
- `pi/`
- in-process extension adapter for the pi coding agent
- `opencode/`
- in-process v1 plugin adapter for OpenCode
- `shared/canonical/`
Expand Down Expand Up @@ -130,4 +132,5 @@ This keeps the first working version small and makes boundary mistakes easier to
- [codex/README.md](./codex/README.md)
- [claude-code/README.md](./claude-code/README.md)
- [deepseek-harness/](./deepseek-harness/)
- [pi/README.md](./pi/README.md)
- [opencode/README.md](./opencode/README.md)
137 changes: 137 additions & 0 deletions components/adapters/pi/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# TokenPilot pi Adapter

This package integrates TokenPilot with [pi](https://pi.dev) (`@earendil-works/pi-coding-agent`, verified on **0.87.1**) as an in-process pi extension. No proxy or gateway is involved: TokenPilot runs inside pi's own request lifecycle.

This adapter explicitly binds the TokenPilot `stabilizer`, `reduction`, and `eviction` features. Its product registration provides pi state discovery to the shared CLI and Visual surface.

Design, host facts and gaps: [`docs/adapters/pi-design.md`](../../../docs/adapters/pi-design.md).

For the component-level overview and shared command surface, see:

- [`components/presets/tokenpilot/README.md`](../../presets/tokenpilot/README.md)
- [`components/adapters/README.md`](../README.md)
- [`components/adapters/HOSTS.md`](../HOSTS.md)

## Supports

| Feature | pi hook | Notes |
| :-- | :-- | :-- |
| Stable prefix | `before_agent_start` | Shared stabilizer over prompt inputs (`appendSystemPrompt`, custom sections, context files). Volatile lines go to a trailing `tokenpilot_dynamic` section (target `developer`, default) or to the first user message (target `user`). Never forces a full system-prompt replacement, so pi keeps appending section deltas. |
| Reduction | `context` | Request-time reduction over the whole history, the same position as the Codex/Claude Code proxies. Tool-result text only; pi's session file keeps the originals. |
| Recovery | native tool `memory_fault_recover` | pi has no MCP. The tool has the same name, description and schema as the shared recovery MCP tool and calls the same resolver. The recovery protocol is added as a stable `tokenpilot_recovery` prompt section. |
| Eviction (opt-in, off by default) | `turn_end` → `context_edit` entries | Estimator-driven lifecycle eviction through the shared canonical-surface core in `@lightrsi/eviction`. Evicted originals are archived and recoverable. `turn_end` entries commit before pi's threshold auto-compaction check, so eviction runs before native compaction. |
| Modes | | `conservative`, `normal` (default). `aggressive` is not exposed. |

Current limitations:

- no in-host slash commands; use the shared `lightrsi pi ...` CLI
- the stabilizer is mostly a no-op on pi's default prompt, which has no volatile lines; it acts only on user and project prompt inputs that contain them
- the Context Cleaner is not integrated yet (see the design note)

## Install

```bash
pnpm install
npm --prefix components/adapters/pi run build
npm --prefix components/products/cli run build # for the lightrsi CLI
npm --prefix components/adapters/pi run install:pi
```

Install writes:

- `~/.pi/agent/extensions/tokenpilot/index.js`: a marker-tagged loader that `require`s `components/adapters/pi/dist/extension.js`. pi auto-loads it. A pre-existing non-TokenPilot file at that path is moved to `index.js.bak-<timestamp>` first.
- `~/.pi/agent/tokenpilot.json`: runtime config in `normal` mode, created only if missing and never containing credentials.
- `~/.pi/agent/tokenpilot-install.json`: install manifest used by uninstall.
- `lightrsi` launcher in `~/.local/bin` (or `$LIGHTRSI_BIN_DIR`).

No pi-owned file (`settings.json`, `models.json`, `auth.json`) is modified. `PI_CODING_AGENT_DIR` and `TOKENPILOT_PI_CONFIG` are honoured.

Uninstall:

```bash
npm --prefix components/adapters/pi run uninstall:pi # remove the loader, restore any backup
npm --prefix components/adapters/pi run uninstall:pi -- --purge # also remove the created config and state
```

## Verify

1. Start pi, or run `/reload` in a running session, so the extension loads.
2. `lightrsi pi doctor`
3. After a few turns: `lightrsi pi report`

A healthy doctor shows the loader and bundle present, a writable state dir, `declared features: stabilizer, reduction, eviction`, and the latest session once pi has started one.

## Commands

```bash
lightrsi pi status
lightrsi pi report
lightrsi pi doctor
lightrsi pi visual
lightrsi pi mode conservative
lightrsi pi mode normal
lightrsi pi stabilizer on|off
lightrsi pi stabilizer target developer|user
lightrsi pi reduction on|off
lightrsi pi reduction mode light|balanced|aggressive
lightrsi pi reduction pass <name> on|off
lightrsi pi eviction status|on|off
lightrsi pi eviction set minBlockChars <number>
```

Reduction passes: `readStateCompaction`, `toolPayloadTrim`, `htmlSlimming`, `execOutputTruncation`, `agentsStartupOptimization`.

## Configuration

`~/.pi/agent/tokenpilot.json` uses the same keys and defaults as the Claude Code adapter:

| Key | Default | Notes |
| :-- | :-- | :-- |
| `enabled` | `true` | Master switch; `false` makes every hook a no-op. |
| `modules.stabilizer` | `true` | |
| `hooks.dynamicContextTarget` | `developer` | `developer` or `user`. |
| `modules.reduction` | `true` | |
| `reduction.triggerMinChars` / `maxToolChars` | `2200` / `1200` | `conservative`: `4000` / `1800`. |
| `reduction.passes.*` | all `true` | |
| `modules.eviction` + `eviction.enabled` | `false` | Both must be on, plus the estimator settings below. |
| `eviction.minBlockChars` | `4000` | |
| `taskStateEstimator.baseUrl` / `model` / `apiKey` | unset | OpenAI-compatible endpoint for the task-state estimator. Keep the key out of checked-in files. |

Enabling eviction:

```bash
lightrsi pi eviction on
# then set taskStateEstimator.baseUrl / model / apiKey in ~/.pi/agent/tokenpilot.json
lightrsi pi doctor # "eviction: active" once complete
```

## Runtime Files

```text
~/.pi/agent/tokenpilot-state/tokenpilot/
```

- `tokenpilot/adapter.log`: fail-open reasons and debug lines. Nothing is written to pi's terminal.
- `tokenpilot/tool-result-archives/<session>/`: originals of trimmed and evicted content (recovery)
- `tokenpilot/reduction-memo/<session>.json`: which read disclosed which file, so a restarted pi (`pi -p`, `--continue`) still honours deliberate re-reads
- `session-state/latest.json`, `session-state/bindings/<session>.jsonl`
- `ux-effects/latest.json`, `ux-effects/sessions/<session>.json`
- the registry under the shared history layout (eviction task state, when enabled)

## Debugging

- `lightrsi pi doctor` lists every problem it finds.
- `tail ~/.pi/agent/tokenpilot-state/tokenpilot/tokenpilot/adapter.log`: every hook that failed open logs `<hook> failed open <error>`. pi then continues with its own, unmodified data.
- Set `"logLevel": "debug"` in `tokenpilot.json` for eviction readiness details.
- To confirm the extension loaded, run pi with `--verbose` and look for the `tokenpilot` extension.

## Package Scripts

```bash
npm --prefix components/adapters/pi run build
npm --prefix components/adapters/pi run typecheck
npm --prefix components/adapters/pi test
npm --prefix components/adapters/pi run install:pi
npm --prefix components/adapters/pi run uninstall:pi
npm --prefix components/adapters/pi run doctor:pi
```
31 changes: 31 additions & 0 deletions components/adapters/pi/build.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { build } from "esbuild";
import { join } from "node:path";

async function main() {
await build({
entryPoints: {
extension: "src/extension.ts",
"install-pi": "scripts/install-pi.ts",
"uninstall-pi": "scripts/uninstall-pi.ts",
"doctor-pi": "scripts/doctor-pi.ts",
},
bundle: true,
// ../shared/canonical has no node_modules of its own; resolve workspace deps from here.
nodePaths: [join(__dirname, "node_modules")],
outdir: "dist",
platform: "node",
target: "node20",
format: "cjs",
sourcemap: true,
minify: false,
logLevel: "info",
logOverride: {
"empty-import-meta": "silent",
},
});
}

main().catch((err) => {
console.error(err);
process.exit(1);
});
44 changes: 44 additions & 0 deletions components/adapters/pi/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"name": "@lightrsi/pi-adapter",
"version": "0.1.0-beta.1",
"description": "TokenPilot adapter for the pi coding agent (in-process extension).",
"lightrsi": {
"role": "adapter"
},
"main": "dist/extension.js",
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/zjunlp/LightRSI.git",
"directory": "components/adapters/pi"
},
"engines": {
"node": ">=20.0.0"
},
"dependencies": {
"@lightrsi/artifact-store": "workspace:*",
"@lightrsi/eviction": "workspace:*",
"@lightrsi/history": "workspace:*",
"@lightrsi/host-adapter": "workspace:*",
"@lightrsi/kernel": "workspace:*",
"@lightrsi/mcp": "workspace:*",
"@lightrsi/product-surface": "workspace:*",
"@lightrsi/reduction": "workspace:*",
"@lightrsi/stabilizer": "workspace:*",
"@lightrsi/tokenpilot": "workspace:*"
},
"scripts": {
"build": "tsx build.ts",
"doctor:pi": "node --import tsx scripts/doctor-pi.ts",
"install:pi": "node --import tsx scripts/install-pi.ts",
"uninstall:pi": "node --import tsx scripts/uninstall-pi.ts",
"test": "node --import tsx --test tests/*.test.ts",
"typecheck": "tsc -p tsconfig.json --noEmit"
},
"devDependencies": {
"esbuild": "^0.25.10",
"tsx": "^4.21.0",
"typescript": "^5.9.3"
},
"keywords": ["pi", "tokenpilot", "context-management", "extension"]
}
16 changes: 16 additions & 0 deletions components/adapters/pi/scripts/doctor-pi.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/usr/bin/env node
import { defaultTokenPilotPiConfigPath, loadTokenPilotPiConfig } from "../src/config.js";
import { formatPiDoctorReport, inspectPiDoctor } from "../src/doctor.js";

async function main() {
const configPath = process.env.TOKENPILOT_PI_CONFIG ?? defaultTokenPilotPiConfigPath();
const config = await loadTokenPilotPiConfig(configPath);
const report = await inspectPiDoctor({ config, configPath, agentDir: process.env.PI_CODING_AGENT_DIR });
console.log(formatPiDoctorReport(report));
if (!report.healthy) process.exitCode = 1;
}

main().catch((error) => {
console.error(error instanceof Error ? error.message : String(error));
process.exit(1);
});
29 changes: 29 additions & 0 deletions components/adapters/pi/scripts/install-pi.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/usr/bin/env node
import { installPiTokenPilot } from "../src/install.js";

async function main() {
const result = await installPiTokenPilot({
agentDir: process.env.PI_CODING_AGENT_DIR,
configPath: process.env.TOKENPILOT_PI_CONFIG,
});
console.log([
"TokenPilot pi install complete:",
`- extension loader: ${result.paths.loaderPath}`,
`- extension bundle: ${result.bundlePath}`,
...(result.backupPath ? [`- previous loader backed up to: ${result.backupPath}`] : []),
`- tokenpilot config: ${result.paths.configPath} (${result.configCreated ? "created, normal mode" : "kept existing"})`,
`- state dir: ${result.stateDir}`,
`- install manifest: ${result.paths.manifestPath}`,
result.cliBin?.installed
? `- lightrsi CLI bin: installed at ${result.cliBin.launcherPath ?? result.cliBin.binPath}`
: `- lightrsi CLI bin: skipped (build components/products/cli first)`,
...(result.cliBin && !result.cliBin.binDirOnPath ? [`- lightrsi CLI PATH note: add ${result.cliBin.binDir} to PATH if 'lightrsi' is unavailable.`] : []),
"- next step: start (or /reload) pi so the extension loads",
"- verify: lightrsi pi doctor",
].join("\n"));
}

main().catch((error) => {
console.error(error instanceof Error ? error.message : String(error));
process.exit(1);
});
24 changes: 24 additions & 0 deletions components/adapters/pi/scripts/uninstall-pi.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env node
import { uninstallPiTokenPilot } from "../src/install.js";

async function main() {
const purge = process.argv.includes("--purge");
const result = await uninstallPiTokenPilot({
agentDir: process.env.PI_CODING_AGENT_DIR,
configPath: process.env.TOKENPILOT_PI_CONFIG,
purge,
});
console.log([
"TokenPilot pi uninstall:",
`- loader removed: ${result.loaderRemoved ? "yes" : "no"}`,
`- previous loader restored: ${result.backupRestored ? "yes" : "no"}`,
...(result.foreignLoaderKept ? ["- a non-TokenPilot loader exists at the path and was left untouched"] : []),
`- config removed: ${result.configRemoved ? "yes" : purge ? "no (not created by install)" : "no (pass --purge)"}`,
`- state removed: ${result.stateRemoved ? "yes" : purge ? "no" : "no (pass --purge)"}`,
].join("\n"));
}

main().catch((error) => {
console.error(error instanceof Error ? error.message : String(error));
process.exit(1);
});
Loading
Loading