feat(integrations): give Tau persistent Basic Memory continuity

.github/workflows/consolidated-packages.yml

@@ -97,6 +97,16 @@ jobs:
       - name: Validate manifest and run unit tests
         run: just check
 
+  tau:
+    name: Tau extension
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: extractions/setup-just@v4
+      - uses: astral-sh/setup-uv@v3
+      - name: Verify Tau extension
+        run: just package-check-tau
+
   openclaw:
     name: OpenClaw package
     permissions:

integrations/tau/DESIGN.md

@@ -0,0 +1,97 @@
+# Basic Memory for Tau: continuity design
+
+Issue: https://github.com/basicmachines-co/basic-memory/issues/1487
+Integration: https://github.com/basicmachines-co/basic-memory/pull/1489
+Required host work: https://github.com/huggingface/tau/pull/683
+
+## Product contract
+
+A fresh or compacted session recovers the objective, decisions, unfinished work,
+verified findings, and next action through the shared Basic Memory graph. Full
+MCP tool access supports that loop; it is not a substitute for it.
+
+## Host dependencies, implemented separately
+
+Stock Tau 0.4.1 only notifies extensions around overflow compaction. Its queued
+custom messages run as follow-ups, which can cause an extra model response even
+with trigger_turn=False. Its public context cannot read persisted custom receipts
+or request a tool-free summary through the active provider.
+
+Tau #683 supplies:
+
+1. Awaited extension start/end notifications around manual, detailed manual,
+   threshold and overflow compaction. No-op checks emit nothing. Failure and
+   cancellation emit aborted end events. The original context remains available
+   until start handlers finish.
+2. `context.branch_entries`: deep-copied persisted active-path entries for receipt
+   recovery and lineage without session-file scraping.
+3. `context.summarize`: bounded tool-free active-model synthesis, no agent turn,
+   history mutation, exposed credentials, or detached task.
+4. `tau.append_message`: persist idle reference context before the next prompt,
+   without queuing another turn.
+5. Shutdown/start notifications around in-place tree branches on the same runtime.
+
+The package pins the tested fork SHA until these interfaces are released upstream.
+It does not modify installed Tau or pretend #506 is fully closed: that issue's
+threshold/manual frontend-iterator/TUI-status work is separate from extension
+callback delivery. Persisted-entry notifications are not required; branch snapshots
+provide authoritative receipt reconstruction.
+
+## Explicit ownership
+
+`bridge.py` finishes paginated discovery during synchronous setup using a joined
+temporary thread/process, because Tau composes tools before session_start. The
+probe closes before setup returns. Runtime MCP contexts belong to one async owner
+task; calls share them. Close cancels active requests and retires contexts in their
+owner task. Restart creates a fresh stop event, including same-runtime tree branches.
+
+`extension.py` registers every advertised tool, preserving schemas and forwarding
+arguments unchanged. `results.py` converts native text/images, preserves other
+blocks and structured data, and validates write receipts. Routing and auth remain
+BM's responsibility; automatic memory requires an explicit project destination.
+
+## Knowledge and receipt flow
+
+`continuity.py` reads public persisted message entries, excluding reasoning,
+tools, synthetic summaries, and injected context. The last confirmed handoff plus
+new public messages are synthesized in bounded chunks under one checkpoint deadline.
+Knowledge capture at settled/compaction/shutdown uses the same source-tip identity;
+unchanged state reuses its receipt instead of making redundant model requests.
+
+Before a remote write, append a pending intent containing project, capture id,
+kind, source tip, and content digest. After a validated BM result, append a confirmed
+receipt with its returned path. Reload/start reconciles pending intents by reading
+remote content, never resubmitting writes. A sibling branch can recover the same
+source-tip capture by identity and byte digest. Divergent source tips produce
+separate snapshots linked to the prior active-branch checkpoint. Transcript notes
+are distinct, opt-in, immutable segments; handoffs link their captured sources.
+
+Startup reads confirmed active-branch checkpoints before broader scoped results,
+expands the checkpoint's graph neighborhood, and includes shared recent activity.
+Filter-only search supplies an epoch after_date to obtain BM's newest-first order
+without excluding long-idle modern sessions. Topic search also retrieves shared
+coding_session/task/decision notes. The inserted brief is bounded and labeled as
+untrusted historical reference, not current repository facts.
+
+## Failure and privacy policy
+
+Automatic-memory failure is visible but does not stop coding. Host observation
+hooks are awaited, not veto hooks; a failed write can precede a compaction that
+continues. Only a confirmed checkpoint is referenced afterward. Closing always
+releases MCP, even if summary generation fails. No shutdown promise applies to
+process kills. No ambiguous remote write is retried or overwritten automatically.
+
+Common credential patterns are masked before automatic capture and in generated
+output, but arbitrary public-text secrets cannot be reliably detected. Controls
+and destination disclosure are part of the privacy boundary. No raw tool payloads,
+hidden reasoning, model credentials, or arbitrary server error text are captured.
+
+## Evidence
+
+Tests cover real host registration, persisted sessions, paginated stdio tools,
+source-tip deduplication, branch lineage, all compaction paths, write/receipt
+failures, cancellation, shutdown, fresh resume, and headless TUI reload. The real-BM
+suite proves file writes, reads, searches, transcripts, checkpoints, compaction
+reference restoration, reload and resume in temporary local projects. Synthesis
+uses deterministic providers, so these tests do not claim live-model quality or
+paid/cloud account end-to-end verification.

integrations/tau/README.md

@@ -0,0 +1,168 @@
+# Basic Memory for Tau
+
+Every server-advertised MCP tool, plus automatic continuity across sessions:
+startup recall, ongoing knowledge capture, awaited pre-compaction checkpoints,
+optional public-message transcripts, and shutdown summaries.
+
+**Upstream dependency:** [Tau PR #683](https://github.com/huggingface/tau/pull/683).
+The isolated environment pins its tested fork commit. Stock Tau 0.4.1 lacks the
+required APIs; the extension refuses to load there rather than silently offering
+weaker continuity. No installed Tau files are patched.
+
+## Run from this repository
+
+Prerequisites: Python 3.13+, uv, and an installed/configured Basic Memory CLI.
+
+```bash
+uv sync --project integrations/tau
+uv run --project integrations/tau tau -e ./integrations/tau
+```
+
+This uses the pinned Tau fork and MCP 2 in a separate environment. The MCP server
+defaults to `bm mcp --transport stdio` from PATH. No extension operation installs
+dependencies, creates projects, or changes credentials. Do not load two copies.
+
+For development, `/reload` reads the explicitly loaded source directory. For a
+copied install, use `tau install ./integrations/tau` **from the compatible Tau
+environment**; update it with `tau install --force ./integrations/tau`, then
+`/reload`. Tau's installer does not install dependencies. Wait for an upstream
+release containing #683 before using an ordinary released Tau environment.
+
+## Configure the destination and capture policy
+
+Create `~/.tau/basic-memory.json`, choosing an existing Basic Memory project:
+
+```json
+{
+  "project": "my-memory-project",
+  "auto_recall": true,
+  "capture_knowledge": true,
+  "checkpoint_on_compact": true,
+  "summarize_on_shutdown": true,
+  "capture_transcript": false
+}
+```
+
+**Setting a project enables automatic synthesized writes by default.** Summaries
+use the active Tau model/provider and its ordinary network routing and billing.
+They add model requests and latency. Turn off `capture_knowledge` to summarize
+only at compaction/shutdown, or disable those settings too for tools/recall only.
+
+Only user-level configuration is discovered. `TAU_BASIC_MEMORY_CONFIG` can select
+an explicit alternate file. Ambient repository files cannot select an executable
+or capture destination. Unknown/invalid settings fail validation. Reload changes.
+
+| Setting | Default | Meaning |
+| --- | --- | --- |
+| `command` | `bm` | Executable, launched directly without a shell |
+| `args` | `["mcp", "--transport", "stdio"]` | Server arguments |
+| `project` | unset | Explicit automatic-memory destination, including workspace/project routing |
+| `auto_recall` | `true` | Restore branch checkpoint and relevant shared context on start/reload/resume/branch |
+| `capture_knowledge` | `true` | Synthesize new public conversation at `agent_settled` |
+| `checkpoint_on_compact` | `true` | Await a checkpoint before manual, threshold, or overflow compaction |
+| `summarize_on_shutdown` | `true` | Save outstanding public work before closing/replacing a session |
+| `capture_transcript` | `false` | Separate immutable public user/final-assistant message notes |
+| `capture_folder` | `tau/transcripts` | Transcript directory within the project |
+| `checkpoint_folder` | `tau/checkpoints` | Synthesized handoff directory |
+| `timeout_seconds` | `30` | MCP initialization/discovery/call timeout, at most 300 seconds |
+| `summary_timeout_seconds` | `60` | Entire checkpoint deadline, including synthesis and persistence, at most 300 seconds |
+| `summary_chunk_chars` | `16000` | Public input processed per summary request; previous handoff is also included |
+| `recall_chars` | `12000` | Maximum reference payload, plus its fixed warning/truncation marker |
+
+With no project, tools remain available but automatic memory stays off. Tool
+arguments are forwarded unchanged; the plugin does not inject its capture project
+into arbitrary agent calls. Configure local/cloud routing and authentication
+through Basic Memory. Choosing a cloud or team project sends automatic writes
+there; use a team destination only when you intend that disclosure.
+
+## Continuity lifecycle
+
+- **Start/resume/reload/branch:** reconcile outstanding write receipts by reading,
+  restore the latest receipt on the active branch, read its graph neighborhood,
+  then retrieve cwd-scoped checkpoints/tasks/decisions and topic matches. A shared
+  recent-activity feed discovers work from other agents. References are inserted
+  before the next prompt, not queued as an extra model turn. They are untrusted
+  historical evidence; the agent must verify live repository state.
+- **After work settles:** summarize new public messages together with the prior
+  handoff. Chunked synthesis processes the selected public text without silently
+  dropping its oldest portion. Oversized model output or the deadline fails
+  visibly instead of manufacturing a fallback summary.
+- **Before compaction:** finish the same receipt-backed write while original
+  context exists. Already-saved state is reused. After successful compaction,
+  insert the confirmed checkpoint reference into the new context. Aborted
+  compaction does not announce a restored reference.
+- **Shutdown/replacement:** summarize any outstanding work, then close MCP even
+  when memory fails. No new agent turn or detached background writer is needed.
+
+Knowledge snapshots are ordinary `coding_session` Markdown notes with
+observations and relations, linked to the previous checkpoint and, when enabled,
+the captured source messages. Explicit remember workflows search/update existing
+knowledge notes. Other agents can read the same graph using normal BM tools.
+
+## Tools and commands
+
+Every discovered tool becomes `bm_<original-name>` with its actual schema and
+description. Discovery follows all pages, rejects duplicate names/cursor cycles,
+and respects server feature gates. `/reload` refreshes the inventory.
+
+- `/bm-orient [topic]`: retrieve and insert relevant notes without an agent turn.
+- `/bm-checkpoint [focus]`: synthesize and persist through the serialized input
+  hook; reuse a confirmed checkpoint when the source state has not changed.
+- `/bm-remember <text>`: ask the agent to search/update or create connected knowledge.
+- `/bm-status`: connection, destination, controls, last checkpoint and failure.
+
+A command's initial acknowledgement is a request, not a save receipt. Checkpoint
+success is notified only after a BM write/read reconciliation and a durable Tau
+receipt. Tool errors remain errors; non-text/non-image MCP blocks are explicitly
+serialized rather than discarded, and structured results remain intact.
+
+## Replay, branches, and failures
+
+A capture identity includes project, session id, kind, and persisted source entry
+id. An intent is appended to Tau before a write; confirmation follows a validated
+BM receipt. Reload/resume reads these records on the active branch. Sibling
+branches sharing a source tip can recover the same immutable capture by identity
+and content digest. Divergent tips get separate notes and parent-checkpoint links.
+
+Pending writes are reconciled by reads only, including at startup. Missing,
+ambiguous, or changed remote content stays visibly unconfirmed. No ambiguous write
+is automatically resubmitted and no existing capture is overwritten. Inspect the
+configured destination and BM availability, then `/reload` to reconcile again.
+A deliberately abandoned pending intent remains diagnostic rather than being
+silently marked successful.
+
+Automatic-memory failures warn rather than stop coding. Tau's observation hooks
+are awaited but are not veto hooks: if persistence fails, compaction may continue.
+A process kill cannot run shutdown handlers. Neither case is reported as a save.
+
+## Privacy
+
+Raw transcripts are off by default. Only actual persisted public user text and
+final assistant text qualify. Hidden reasoning, raw tool arguments/results,
+images, custom reference messages, and synthetic compaction summaries are not
+captured as conversation segments. Synthesized handoffs use that same public
+input, not a hidden tool-output dump.
+
+Common credential forms (private-key blocks, recognized token prefixes, Bearer
+values, JWT-shaped strings, and credential assignments) are masked before
+synthesis/transcript writes and again in synthesized output. **This is not a
+general secret detector.** Unknown secrets in public text may remain. Disable
+automatic capture for sensitive conversations; do not rely on a model or a regex
+as a privacy boundary. Server stderr and arbitrary background exception payloads
+are not echoed into Tau notifications.
+
+## Verification
+
+```bash
+just package-check-tau
+BM_TAU_TEST_COMMAND="$PWD/.venv/bin/bm" \
+  uv run --project integrations/tau pytest -c integrations/tau/pyproject.toml \
+  integrations/tau/tests -q
+```
+
+Tests exercise real Tau sessions/storage, real stdio MCP processes, all four
+compaction entry points, reload, branch/resume, cancellation, receipt failures,
+privacy controls, and headless Textual `/reload`. The opt-in real-BM tests isolate
+HOME/configuration/notes, force local routing, and disable updates, semantic
+model downloads, and telemetry. Model behavior is tested with deterministic fake
+providers, not a paid live model or production memory project.