flip cross-call diffing to default-on; add --no-diff opt-outs

README.md

@@ -29,12 +29,13 @@ then (optionally) serves it through a per-tool policy that decides which tiers r
 - **Tier 0.5 — dictionary code**: repeated string values *and repeated whole subtrees*
   are folded into an inline legend (`~0`, `~1`, …) proven disjoint from every literal in
   the payload. Committed only when it actually saves tokens, so it never regresses.
-- **Tier 0.7 — cross-call diff (opt-in, stateful)**: when the same tool is called
-  repeatedly, the proxy can emit a lossless delta against the prior result instead of
+- **Tier 0.7 — cross-call diff (stateful, ON by default)**: when the same tool is called
+  repeatedly, the proxy emits a lossless delta against the prior result instead of
   the full payload (the 91%-overlap headroom). Self-describing, verified to reconstruct
-  exactly, and emitted only when smaller — falls back to the full form otherwise. OFF by
-  default (`proxy --diff`, or `install-mcp --diff` when wrapping Claude Code servers);
-  its model-fluency is validated by `terse fluency --diff`.
+  exactly, and emitted only when smaller — falls back to the full form otherwise.
+  Default-on since its validation program completed (fluency, nested-record coverage,
+  and the drift soak — see Status); opt out with `proxy --no-diff` / `install-mcp
+  --no-diff` or a policy-file `"diff": false`.
   Record-shaped JSON gets a row/key diff; non-JSON results (file reads, source excerpts,
   log tails) get a separate content-defined-chunking (CDC) diff — a rolling hash cuts
   chunk boundaries by content, not position, so an edit anywhere only perturbs the
@@ -123,13 +124,13 @@ stdio proxy. The proxy's open question — *does a model read the compressed for
 well as raw JSON?* — now has a measured answer: on a stress corpus, Claude Haiku 4.5
 and Gemini 2.5 Flash match raw-JSON accuracy on the compressed form (100% paired) at a
 37% token saving (`terse fluency`; see TECHNICAL.md). Whole-subtree aliasing (folding
-repeated objects, not just strings) is built. Cross-call diffing is built as an
-**opt-in** lossless tier (`proxy --diff` / `install-mcp --diff`); its fluency gate
-(`fluency --diff`) has passed on both the record surface (4-model panel, 100%) and the
-nested-record surface (`structure`: diff 100% vs full-terse 94%), so it's cleared for
-live use. Long-chain drift is soaked from both sides: mechanically
+repeated objects, not just strings) is built. Cross-call diffing is a lossless tier
+that is now **on by default** — its full validation program passed: pair fluency
+(`fluency --diff`, 4-model panel 100%), the nested-record surface (`structure`: diff
+100% vs full-terse 94%), and long-chain drift soaked from both sides — mechanically
 (`tests/test_diff_soak.py` — exact reconstruction hundreds of chained hops deep) and
-behaviorally (`fluency --diff-soak` — model accuracy vs chain depth up to the keyframe
-bound). The Tier 1 lossy modes `truncate` and
+behaviorally (`fluency --diff-soak` — no depth-correlated accuracy loss up to the
+keyframe bound). Opt out per proxy (`--no-diff`) or per policy (`"diff": false`).
+The Tier 1 lossy modes `truncate` and
 `drop-to-retrieve` are built (opt-in, off by default); `summarize` remains designed but
 not yet built — see TECHNICAL.md "Known Limitations".

TECHNICAL.md

@@ -256,17 +256,19 @@ gitignored because captured tool output may contain real data.
   merging, and any client method outside `initialize`/`tools/list`/`tools/call` falls back
   to peer 0 (debug-logged). Each is a documented v1 limitation, not a silent gap — revisit
   if a real workload needs it.
-- **Cross-call diffing is built, opt-in (`proxy --diff`).** The probe shows 91% overlap
-  between successive same-tool calls; the proxy can now emit a lossless delta against the
-  prior result (keyed row diff for record arrays, shallow key diff for objects) instead
-  of the full payload. It is stateful (per-tool last result), self-verifying (a diff is
-  sent only when it provably reconstructs the result), and fail-open (full form whenever
-  a diff doesn't apply or isn't smaller — the dangling-reference fallback). It ships OFF
-  by default because two risks are model-side, not codec-side: (1) the round-trip gate
-  proves the diff reconstructs but **not** that a model *reads* it as well as the full
-  form — checked by `terse fluency --diff`; (2) the diff references the prior result in
-  the model's context, which a context compaction could evict. Enable only after the diff
-  fluency check passes for your consumer. Risk (2) is bounded by **keyframes**: the proxy
+- **Cross-call diffing is built and ON by default (`proxy --no-diff` / policy
+  `"diff": false` to opt out).** The probe shows 91% overlap between successive
+  same-tool calls; the proxy emits a lossless delta against the prior result (keyed row
+  diff for record arrays, shallow key diff for objects) instead of the full payload. It
+  is stateful (per-tool last result), self-verifying (a diff is sent only when it
+  provably reconstructs the result), and fail-open (full form whenever a diff doesn't
+  apply or isn't smaller — the dangling-reference fallback). It shipped opt-in until its
+  two model-side risks were measured, and flipped default-on when that program completed:
+  (1) the round-trip gate proves the diff reconstructs but **not** that a model *reads*
+  it as well as the full form — since PASSED by `terse fluency --diff` (4-model panel,
+  incl. the nested-record surface, #72) and at chain depth by `fluency --diff-soak`
+  (#75); (2) the diff references the prior result in the model's context, which a
+  context compaction could evict. Risk (2) is bounded by **keyframes**: the proxy
   forces a self-contained full result after every K consecutive diffs per tool, so a
   chained diff can never drift more than K turns from an anchor a model can reconstruct
   from scratch (`diff_keyframe_interval` policy field / `proxy --diff-keyframe-interval K`,
@@ -279,8 +281,8 @@ gitignored because captured tool output may contain real data.
   unbounded chain — with an independent client-side reconstructor asserting exactness at
   every hop; `terse fluency --diff-soak` measures the model-side analogue (accuracy vs
   chain depth, up to the keyframe bound). The residual gap — a context *compaction* with
-  no reconnect — is unobservable over stdio, and is the standing reason `--diff` is
-  opt-in.
+  no reconnect — is unobservable over stdio; keyframes cap its blast radius at K turns,
+  and `--no-diff` remains the escape hatch for a consumer where that residual matters.
 - **Text diff (Tier 0.7 text, #25) covers non-JSON results, but only the codec side is
   measured so far.** File reads, source excerpts, and log tails get `applicable: False`
   in `measure_payload` — zero Tier-0 compression — and used to get zero cross-call

USAGE.md

@@ -129,12 +129,17 @@ uv run terse uninstall-mcp --all          # every terse-managed server
 the entry's `env`/`cwd`/etc. The original is saved verbatim in a sidecar stash
 (`.terse-mcp-stash.json` next to the config), so `uninstall-mcp` restores it
 byte-for-byte. It's **idempotent** (re-running re-wraps from the stashed original
-instead of nesting proxies) and never enables `--diff` implicitly — pass
-`install-mcp --diff` (optionally `--diff-keyframe-interval K`) to wrap with
-cross-call diffing on; a re-install without the flag drops it again. It honors
-`$CLAUDE_CONFIG` if your config isn't at `~/.claude.json`. Start with one
-high-win, read-only server (e.g. `runecho`) and confirm it works before
-wrapping more.
+instead of nesting proxies). Cross-call diffing is the proxy default, so a plain
+wrap inherits it; pass `install-mcp --no-diff` to bake an opt-out into a server's
+entry (or `--diff` to override a policy-file `"diff": false`), and
+`--diff-keyframe-interval K` to tune re-anchoring. Flags always reflect the latest
+install invocation. It honors `$CLAUDE_CONFIG` if your config isn't at
+`~/.claude.json`. Start with one high-win, read-only server (e.g. `runecho`) and
+confirm it works before wrapping more.
+
+One sharp edge worth knowing: a re-wrap rebuilds the entry from the **stashed
+original**, so any hand-edit you made to the *wrapped* entry (say, an `env.PATH`
+pin) is dropped — apply such edits to the stash original too.
 
 Claude Code has three MCP scopes, and `--scope` targets any of them (default
 `user`, i.e. today's behavior):
@@ -335,31 +340,34 @@ final-answer accuracy — gated on the worst model. Run this before enabling
 
 ### Cross-call diffing and its fluency check
 
-The proxy can emit a lossless **delta** against the prior same-tool result instead of
+The proxy emits a lossless **delta** against the prior same-tool result instead of
 the full payload — big in agent loops that call the same tool repeatedly (~91% overlap).
-It is **opt-in** and stateful:
+It is stateful and **on by default** (its validation program — pair fluency,
+nested-record coverage, and the drift soak — has passed; see TECHNICAL.md):
 
 ```
-# enable it on the proxy (off by default):
-uv run terse proxy --diff -- uvx some-mcp-server
+# nothing to enable — a plain proxy diffs. Opt OUT per proxy:
+uv run terse proxy --no-diff -- uvx some-mcp-server
+# or per policy file: {"diff": false, ...}
 
-# before trusting it, check a model still reads the diff as well as the full result.
+# re-run the fluency gate against your own consumer/model anytime.
 # needs same-tool PAIRS in the corpus (capture a tool 2+ times) + a configured model:
 TERSE_FLUENCY_BASE_URL=... TERSE_FLUENCY_API_KEY=... TERSE_FLUENCY_MODELS=... \
   uv run terse fluency --diff --corpus corpus
 ```
 
 `fluency --diff` reports diff-form accuracy vs full-result accuracy on the same
-questions and PASS/FAILs on the worst model — run it before enabling `proxy --diff` for
-your consumer. The diff is always lossless and only sent when smaller; it falls back to
-the full compressed form whenever no diff applies or the prior result isn't available.
+questions and PASS/FAILs on the worst model — re-run it for a new/weaker consumer
+before trusting the default. The diff is always lossless and only sent when smaller;
+it falls back to the full compressed form whenever no diff applies or the prior
+result isn't available.
 
 ### The depth dimension: `fluency --diff-soak`
 
 `--diff` tests one hop (full result + one diff). In production a model reads up to
 `--diff-keyframe-interval` (default 5) **consecutive** diffs off one full anchor
-before the proxy re-anchors — so the question gating a default-flip is whether
-comprehension *drifts* with chain depth:
+before the proxy re-anchors — so the question that gated the default-flip (and gates
+raising the keyframe interval) is whether comprehension *drifts* with chain depth:
 
 ```
 # real corpus runs, depths 1..5 (6 chain windows per depth, round-robin across tools):
@@ -377,9 +385,9 @@ reconstructed exactly, keyframe cadence, reconnect resets — is pinned in
 
 The diff above only reasons about JSON. Non-JSON tool output (file reads, source
 excerpts, log tails) gets its own diff codec (Tier 0.7, `text_diff.py`) — same
-lossless/opt-in/falls-back-to-full contract, applied to unstructured text instead of
-records. `--text-diff-eval` is its behavioral check, the text-payload analogue of
-`--diff`:
+lossless/on-by-default/falls-back-to-full contract, applied to unstructured text
+instead of records. `--text-diff-eval` is its behavioral check, the text-payload
+analogue of `--diff`:
 
 ```
 # needs same-tool TEXT payload PAIRS in the corpus (capture a text-producing tool
@@ -389,11 +397,11 @@ TERSE_FLUENCY_BASE_URL=... TERSE_FLUENCY_API_KEY=... TERSE_FLUENCY_MODELS=... \
 ```
 
 It asks whether a model reconstructs the current text as accurately from (previous text
-+ text-diff) as from the full current text, and PASS/FAILs on the worst model — run it
-before enabling `proxy --diff` for text-heavy tools. There's no separate proxy flag: the
-same `proxy --diff` above already emits a text diff instead of a JSON diff whenever the
-payload isn't JSON, so this eval is a risk-item check on that existing behavior, not a
-new switch.
++ text-diff) as from the full current text, and PASS/FAILs on the worst model — re-run
+it for a new consumer of text-heavy tools. There's no separate switch: the same
+default-on diffing emits a text diff instead of a JSON diff whenever the payload isn't
+JSON (and `--no-diff` turns both off together), so this eval is a risk-item check on
+existing behavior.
 
 ### Building a sample set
 

src/terse/cli.py

@@ -171,20 +171,26 @@ def _cmd_proxy(args: argparse.Namespace) -> int:
     except (OSError, ValueError) as e:
         print(f"proxy: {e}", file=sys.stderr)
         return 2
-    if args.diff:
-        pol.diff = True  # CLI opt-in overrides the policy default (off)
+    # tri-state: --diff / --no-diff override the policy value; neither flag = keep it
+    # (the Policy default is ON since the validation program completed — see #75).
+    if args.diff and args.no_diff:
+        print("proxy: --diff and --no-diff are mutually exclusive", file=sys.stderr)
+        return 2
+    diff_override = True if args.diff else (False if args.no_diff else None)
+    if diff_override is not None:
+        pol.diff = diff_override
     if args.diff_keyframe_interval is not None:
         pol.diff_keyframe_interval = args.diff_keyframe_interval
 
     if args.config:
         from .multiproxy import run_multi_proxy
         try:
             # diff_override/diff_keyframe_override (not just the mutated `pol` above)
-            # so --diff also applies to a peer with its OWN policy_path, not just
-            # peers using this default policy.
+            # so --diff/--no-diff also applies to a peer with its OWN policy_path,
+            # not just peers using this default policy.
             return run_multi_proxy(args.config, pol, debug=args.debug,
                                    capture_dir=args.capture_dir, debug_log=args.debug_log,
-                                   diff_override=args.diff,
+                                   diff_override=diff_override,
                                    diff_keyframe_override=args.diff_keyframe_interval)
         except (OSError, ValueError) as e:
             print(f"proxy --config: {e}", file=sys.stderr)
@@ -581,9 +587,13 @@ def _short_cmd(entry) -> str:
 def _cmd_install_mcp(args: argparse.Namespace) -> int:
     from .install_mcp import do_install
 
+    if args.diff and args.no_diff:
+        print("install-mcp: --diff and --no-diff are mutually exclusive", file=sys.stderr)
+        return 2
+    diff = True if args.diff else (False if args.no_diff else None)
     try:
         res = do_install(args.servers, args.policy, dry_run=args.print,
-                         capture_dir=args.capture_dir, diff=args.diff,
+                         capture_dir=args.capture_dir, diff=diff,
                          diff_keyframe_interval=args.diff_keyframe_interval,
                          scope=args.scope, file=args.file, repo_path=args.repo_path)
     except (FileNotFoundError, ValueError) as e:
@@ -597,8 +607,10 @@ def _cmd_install_mcp(args: argparse.Namespace) -> int:
     print(f"config: {res['config']}  scope: {res['scope']}  policy: {res['policy']}")
     if res.get("capture_dir"):
         print(f"capture: raw tool results → {res['capture_dir']}")
-    if res.get("diff"):
-        print("diff: cross-call diffing enabled (proxy --diff)")
+    if res.get("diff") is True:
+        print("diff: explicit --diff baked in (overrides a policy-file opt-out)")
+    elif res.get("diff") is False:
+        print("diff: DISABLED for these server(s) (--no-diff baked in)")
     if res["backup"]:
         print(f"backup: {res['backup']}")
     if not res["dry_run"] and res["changes"]:
@@ -788,9 +800,12 @@ def main(argv: list[str] | None = None) -> int:
                     help="with --diff, force a full result every K consecutive diffs per tool "
                          "to bound dangling-reference drift (default 5; 0 disables)")
     px.add_argument("--diff", action="store_true",
-                    help="enable cross-call diffing (stateful; emits a lossless delta vs the "
-                         "prior same-tool result when smaller). Opt-in; model-fluency "
-                         "validated by `terse fluency --diff`/`--text-diff-eval`")
+                    help="force cross-call diffing ON, overriding a policy file's "
+                         '"diff": false (diffing is the DEFAULT since #75, so this is '
+                         "only needed against such a policy)")
+    px.add_argument("--no-diff", action="store_true",
+                    help="disable cross-call diffing (emit the full compressed form "
+                         "every call), overriding the default and any policy value")
     px.add_argument("--debug", action="store_true", help="log compressions to stderr")
     px.add_argument("--capture-dir", metavar="DIR",
                     help="tee each raw tool-result payload into this corpus dir for later "
@@ -873,11 +888,15 @@ def main(argv: list[str] | None = None) -> int:
                     help="also tee raw tool results into this corpus dir for later "
                          "`terse measure`/`verify` (opt-in; never affects forwarding)")
     im.add_argument("--diff", action="store_true",
-                    help="wrap with cross-call diffing enabled (`proxy --diff`); a "
-                         "re-install without this flag drops it again")
+                    help="bake an explicit `--diff` into the wrapped entry (diffing is "
+                         "already the proxy DEFAULT since #75 — only needed to override "
+                         'a policy file\'s "diff": false)')
+    im.add_argument("--no-diff", action="store_true",
+                    help="bake `--no-diff` into the wrapped entry: this server gets "
+                         "full results every call, no cross-call diffing")
     im.add_argument("--diff-keyframe-interval", type=int, default=None, metavar="K",
-                    help="with --diff, force a full result every K consecutive diffs "
-                         "per tool (default 5; 0 disables)")
+                    help="force a full result every K consecutive diffs per tool "
+                         "(default 5; 0 disables)")
     im.add_argument("--print", action="store_true",
                     help="dry-run: show the before/after without writing")
     im.set_defaults(func=_cmd_install_mcp)