Skip to content

beta-stabilize: hold anthropic-beta at its first-seen value per session - #340

Open
deafsquad wants to merge 2 commits into
cnighswonger:mainfrom
deafsquad:feature/beta-stabilize
Open

beta-stabilize: hold anthropic-beta at its first-seen value per session#340
deafsquad wants to merge 2 commits into
cnighswonger:mainfrom
deafsquad:feature/beta-stabilize

Conversation

@deafsquad

Copy link
Copy Markdown
Contributor

Closes the cache-key churn in #326.

What

CC toggles the anthropic-beta set between consecutive turns of one session,
and each toggle is a different cache key for an otherwise identical request.
This snapshots the set at first sight per session and emits it on every
subsequent turn. Deltas CC tries to introduce are reported and not forwarded.

Opt-in via CACHE_FIX_BETA_STABILIZE=1, default off — it changes what we send
upstream, which is the discipline #326 asks for.

Evidence

The test replays the sequence measured on visits-01 in #326 through one
session, and asserts both halves — the defect and the fix:

gate OFF → 3 distinct header values for one conversation   (the premise)
gate ON  → 1                                               (the fix)

Asserting the OFF case matters: without it the test could pass against a
neutered predicate. node --test test/proxy-beta-stabilize.test.mjs → 19/19.

Design notes

  • Order 530, after auto-1m-guard (520) — load-bearing, not cosmetic.
    auto-1m-guard in strip mode removes context-1m from this same header, so
    snapshotting before it would freeze a token the next stage removes, and the
    emitted value would differ from the snapshot on every turn.
  • No extensions.json entry. loadExtensions resolves
    cfg?.order ?? ext.order ?? 1000 and cfg?.enabled ?? ext.enabled ?? true,
    so the module-declared order is the default — same as auto-1m-guard. Say
    the word if you would rather it were listed explicitly.
  • Reuse, not restatement. findBetaHeader / parseBetaTokens /
    joinBetaTokens come from auto-1m-guard, resolveSessionId from
    cache-telemetry.
  • No session id → header untouched. Sharing one snapshot across unrelated
    sessions would send a set the caller never asked for, which is worse than not
    stabilizing.

Non-Functional Requirements

Under the ~300-line production threshold (140 lines), but the checklist is
cheap and #326 is a wire-affecting change:

  • Size/complexity — 140 production lines, one module, no new dependency.
    The decision itself is one exported pure function; everything else is header
    plumbing borrowed from auto-1m-guard.
  • Threat model — reads and rewrites one request header. No credential
    surface, nothing persisted, nothing logged beyond beta token names (already
    public identifiers). The snapshot map holds token strings keyed by session
    id, in memory only.
  • Maintainability — no new abstraction. The one piece of state is a
    module-level Map, bounded at 500 sessions with oldest-out eviction so a
    long-lived proxy cannot accumulate an entry per session seen.
  • Performance — one map lookup and a join per request.
  • Load-bearing?yes. It changes an outbound header that is part of
    Anthropic's cache key, so it wants a human look regardless of the size.

Known divergence from our own implementation

We run a variant in a private proxy that pins only when set membership
matches
and lets a genuine beta change through, on the reasoning that
suppressing a real change sends Anthropic a header the caller did not ask for.
This PR deliberately implements what #326 specifies — first-seen wins, hold
through the change — rather than substituting our design. Happy to add the
set-match behaviour as a second mode if you want it; it is a few lines on top
of planStableBetas.

Caveat

Your full suite exceeds 10 minutes on this machine and was not run to
completion. Verified: the new tests (19/19), proxy-auto-1m-guard (23/23, the
module imported from), proxy-pipeline (15/15, the loader). absence-scan is
red on main for an unrelated Windows reason — see #339, which is independent
of this PR.

— Claude Opus 5, working with @deafsquad

Closes the cache-key churn described in cnighswonger#326: CC toggles the beta set between
consecutive turns of one session, and each toggle is a different cache key for
an otherwise identical request.

Snapshots the set at first sight per session and emits it on every subsequent
turn. Deltas CC tries to introduce are reported on ctx.meta and to stderr,
never forwarded — first-seen wins, and the extension makes no judgement about
which betas are desirable.

Opt-in via CACHE_FIX_BETA_STABILIZE=1, default off, matching the discipline
cnighswonger#326 asks for: it changes what we send upstream.

Order 530, after auto-1m-guard (520). That ordering is load-bearing rather than
cosmetic — auto-1m-guard in strip mode removes context-1m from the same header,
so snapshotting before it would freeze a token the next stage then removes and
the emitted value would differ from the snapshot on every turn.

Reuses findBetaHeader / parseBetaTokens / joinBetaTokens from auto-1m-guard and
resolveSessionId from cache-telemetry rather than restating them.
session-key-invariants caught this: betaSessionKey returned the bare session
id, so two conversations under one session id shared a snapshot. Every subagent
of a session runs the same agent prompt under the same session id — the
collision that put 39 conversations in one insertion-normalization bucket and
that deferred-tool-rewrite inherited.

Now the same key shape as resolveToolRewriteSessionKey:
s-<sid>-<systemPromptSubKey>-<conversationSubKey>.

It matters here even though anthropic-beta is CC-process-global: a coarse key
would impose conversation A's first-seen set on conversation B and send B a
header nobody asked for. The reverse — more keys than processes — costs nothing
in this design, because a new key snapshots on its first turn rather than
waiting to promote a baseline.

Three tests added for the invariants directly, plus an end-to-end case showing
a subagent under the same session id keeps its own set. 22/22 here,
session-key-invariants 4/4.
@deafsquad

Copy link
Copy Markdown
Contributor Author

CI caught a real one, thank you — session-key-invariants was right and I was
wrong.

betaSessionKey returned the bare session id. Two conversations under one
session id therefore shared a snapshot, which is exactly the collision that
file exists to carry: every subagent of a session runs the same agent prompt
under the same session id, so (session-id, system-prompt) put 39
conversations in one insertion-normalization bucket and deferred-tool-rewrite
inherited it because nothing connected the two. The discovery-by-naming design
did its job on a brand-new extension it had never seen.

Fixed in a771678 with the same key shape as resolveToolRewriteSessionKey:
s-<sid>-<systemPromptSubKey>-<conversationSubKey>.

Worth recording why the invariant holds here even though anthropic-beta is
CC-process-global, since that could look like a reason to want a coarser key: a
coarse key would impose conversation A's first-seen set on conversation B and
send B a header nobody asked for. The reverse risk — more keys than there are
CC processes — costs nothing in this design, because a new key snapshots on its
first turn rather than waiting to promote a baseline. (Our own private variant
does pay for extra keys, because its baseline only promotes on a confirmed
cache hit; that is a property of our design, not of this one, and it does not
transfer.)

Added three tests against the invariants directly plus an end-to-end case
showing a subagent under the same session id keeps its own set. 22/22 locally,
session-key-invariants 4/4.

One correction to my earlier note: I wrote that your full suite had not been run
here. That was a Windows machine, where a large number of cases fail for
platform reasons — POSIX permission bits (0600/0700), symlink and worktree
paths. Your Linux CI is the authoritative signal and it ran on this fork PR
despite CONTRIBUTING saying fork PRs do not get CI. I should have waited for it
rather than caveating around a local run.

— Claude Opus 5, working with @deafsquad

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant