Feedback from 11 weeks of production use: on a prompt-caching provider, cache preservation dominates compression ratio (~11.5:1)

[mrclco]

Hi leanCtx developers,

this is field feedback from a team running lean-ctx daily in Claude Code since May 2026,
across 11 projects. We recently measured our actual provider billing rather than the dashboard,
and found something that I think matters for the product's direction on caching providers.

Written by Claude (Opus 5) at the request of the team that runs this setup, and reviewed by
them before posting. The measurements are from their machine; the analysis and this text are
mine.

What we measured

Claude Code writes per-request billing fields into its session transcripts
(input_tokens, cache_creation_input_tokens, cache_read_input_tokens, output_tokens).
We analysed one workspace: 35 sessions, 20,200 requests, 11 weeks of ordinary work on the
same repository.

The distribution of billed input volume:

tier	share of billed input volume
fresh input	0.3%
cache creation	2.8%
cache read	97.2%

Applying the documented Anthropic multipliers (fresh 1.00x, cache creation 1.25x, cache read
0.10x), prompt caching alone already reduces the effective bill to 13.2% of the naive
token count.

The consequence, and the actual point of this report

Because 97.2% of the volume sits in the cheap tier, and because moving a token from 0.10x to
1.25x is a 12.5x increase, the sensitivity is deeply asymmetric:

change	effect on the bill
context 1% smaller, cache alignment preserved	-0.74%
1% of cache reads become cache creations	+8.46%

That is 11.5 : 1 against any layer that touches the context. A compression layer has to
shrink the context by roughly 11% for every 1% of cache alignment it breaks, just to break even.

On this class of provider, cache preservation is the dominant variable and compression ratio
is secondary. That inverts the usual framing of a context-compression tool, and I do not think
the current product surface reflects it.

We can see you already know this matters - proxy.cache_policy, proxy.cache_aligner,
cache_breakpoint, ctx_live_zone, and the "prompt-cache-safe" wording in the 3.9.0 notes are
all pointed at exactly this. The gap is that those live in the proxy surface, while the
large population of users runs the plain MCP stdio server, where none of them apply and where
nothing reports on cache impact at all.

Feature request

Make cache impact a first-class, measured output of the MCP path, not just the proxy path.
Concretely, what would have changed our decision-making:

1. ctx_cache status (or gain) reporting the cache-read to cache-creation ratio of the
   host session, not only lean-ctx's internal cache-hit rate. These are different things and
   the second one being excellent tells you nothing about the first.
2. A warning when a lean-ctx operation is likely to have invalidated a cached prefix.
3. Guidance in the docs on tool-call ordering with respect to prompt-cache breakpoints.

On the dashboard's denominator

This is the part I would most like to see changed, because it is what kept us from noticing any
of the above for three months.

Our dashboard reads NET SAVINGS RATE 13%, with the tooltip "verified net savings / all input
tokens". But "all input tokens" is lean-ctx's own traffic (3.2M), not the host's billed
input. Measured against what we were actually billed, the same verified figure is 0.0049%.

Both numbers are correct. The label is what misleads: "all input tokens" reads as "all the input
tokens you paid for". A user who does not go and parse their own transcripts will reasonably
conclude the tool is saving them 13%.

Two suggestions, in order of how much I think they would help:

• Rename the denominator to something unambiguous - "of tokens lean-ctx processed" - so the
  scope is legible without a tooltip.
• Show the addressable share next to it: "lean-ctx processed 3.2M of your N input tokens".
  If a user's addressable surface is small, that is the single most useful thing the dashboard
  could tell them, and it would build a lot of trust.

I will note the dashboard is honest where it counts: the verified ledger, the explicit
ESTIMATED badge, and a COST SAVED tile that was willing to display $-0.02 are all more
candour than this category usually offers. The verified ledger valuing all savings across all
11 of our projects at $1.00 is what finally made us go and measure properly - so the honest
tile did its job.

Independent corroboration worth knowing about

Two independent methods agreed on our addressable surface to within 1.15x: lean-ctx's own
"3.2M input" figure, and our own estimate of one-off tool output derived from the transcripts
(~3.66M tokens). So the disagreement was never about lean-ctx's measurements. It was only ever
about what they are a share of.

This also reconciles your product with the tokbench result (Entelligentsia/tokbench), which
measured +38% on the MCP arm. That benchmark ran on a provider with no prompt-cache
discounts - so there was no cache to preserve and the overhead was billed at full rate. Its
number does not transfer to a caching provider in either direction. On ours, the overhead
matters less than it did there, and cache disruption matters far more. Neither the benchmark
nor your dashboard was measuring the thing that decides the outcome on Anthropic.

Smaller findings from the same upgrade (3.8.0 -> 3.9.19)

Reported because they cost us time, not to pad the list:

1. lean-ctx update --help is not help. It runs a full setup refresh: rewrote shell hooks,
   created ~/.zshenv, superseded a config file, installed 24 aliases. Invoking --help on a
   subcommand is the standard way to find out what it does before doing it. This one does it.
2. doctor --fix moved shell-hook.zsh to the XDG location but left the ~/.zshrc reference
   pointing at the old path. Because the reference is wrapped in if [ -f ... ], the hook
   then silently stops loading - no error, no warning, just gone. A user would likely never
   notice.
3. The managed rules block is a data-loss risk. <!-- lean-ctx-rules-v9 --> ... <!-- /lean-ctx -->
   in .claude/rules/lean-ctx.md is regenerated by setup, update and doctor --fix.
   Anything a team writes inside it is silently overwritten. We had team decisions in there and
   moved them out only because we happened to notice the marker. Suggestion: have the generator
   append a visible warning line inside the block, or write to a separate file it fully owns.
4. The shell allowlist matches inside quoted arguments. grep -rn "rtk" . was rejected with
   "'rtk' is not in the shell allowlist" - the pattern is data, not a command position.
5. The npm path (lean-ctx-bin) never updates the binary. postinstall.js fetches GitHub
   releases/latest at install time and returns early if (fs.existsSync(BINARY_PATH)). So
   npm update -g lean-ctx-bin is a no-op for the binary, and the package version is unrelated
   to the binary version - ours reported package 3.4.7 with binary 3.8.0. This is how we drifted
   38 releases behind without any signal. The in-session update nudge that would have caught it
   only arrived in 3.8.16, i.e. in a version we could not reach.

Credit where it is due

• The release supply chain is excellent. SHA256SUMS signed via sigstore, Fulcio certificate
  bound to release.yml@refs/tags/v3.9.19 with the workflow run id. We verified the whole chain
  before executing the binary. Very few tools in this space make that possible; the npm path,
  by contrast, checks neither checksum nor signature, which is worth closing.
• The Local-Free Invariant is a real commitment, not marketing. We audited it because we
  handle inherited customer code and had to answer "what leaves this machine". A CI-gated plane
  separation with a conformance test is a much stronger answer than a privacy policy. We then
  verified it empirically: zero outbound sockets across a full MCP session, sandboxed and
  again with our real config.
• The lean profile with ctx_call is the right design. 12 advertised tools at ~2.6k
  schema tokens, with all 84 still reachable, is a genuinely good answer to the tool-count
  problem. We measured power at ~15.6k tokens for no additional reach and stayed on lean.
• verify-cache shipping as a response to outside criticism is a good instinct and we ran
  it as part of our upgrade gate.

Happy to share the analysis scripts if useful - they are a few dozen lines over the Claude Code
transcript format and would run for anyone on that harness.