fix(chat): the model was never being shown the conversation - #186
Merged
Conversation
…ne message
A user reported ClawAI forgetting earlier parts of long conversations. A live
lab against production reproduced total context loss at turn THREE of a
three-turn thread, and length turned out to be irrelevant.
Three independent caps sat in series:
1. the DB read itself: findRecentByThreadId(threadId, 20)
2. assemble(): threadMessages.slice(-THREAD_CONTEXT_LIMIT)
3. filterThreadMessagesForIntent(): dropped EVERY assistant message, kept
user messages scoring >=0.45 on word overlap with the current question,
then .slice(-4) - or messages.slice(-1) when nothing scored.
Plus a 1024-token prompt clamp on the AUTO fast path, triggered by prompts
under 220 characters - precisely the short questions that depend on long
histories. And tokenBudget was `maxTokens ?? 4096`: an OUTPUT length used as
the whole-prompt budget, so a 256k-window model received ~16 KB of history,
memories, files and system prompt combined. `contextWindowTokens` existed in
routing-service, connector-service and the frontend, and in zero files of the
service deciding how much to send.
Measured, 6 free models, same fact, same thread, same distance, four phrasings
of one question:
"What is my access code for this session?" overlap 0.50 -> 83% recall
"Which secret string did I share at the start?" overlap 0.00 -> 0% recall
"Remind me of the credential I mentioned earlier" overlap 0.00 -> 0% recall
"Repeat it back to me." overlap 0.00 -> 0% recall
24/24 threads matched the static prediction. A breadth run scored 19/19 free
models at 100% on the high-overlap phrasing, ruling the models out. Recall was
a function of how many four-letter words the question shared with the sentence
that stated the fact.
ContextComposerManager replaces the selector, under one rule: nothing removes a
message for being irrelevant; only the token budget removes. Relevance decides
order, and order only matters once the budget is full. Selection works in whole
turns, assistant output is never dropped for its role, and eviction walks P3
upward rather than oldest-first.
ModelTokenBudget separates contextWindowTokens / reservedOutputTokens /
systemOverheadTokens / toolOverheadTokens / availableInputTokens. maxTokens now
feeds reservedOutputTokens and nothing else. routing-service gains an internal
context-window route; chat-service reads it through a 15-minute cached client
that fails OPEN - an unknown window must shorten a prompt, not deny an answer.
Every generation now emits a ConversationContextManifest into the context
receipt: included ids, omitted ids with per-message reasons, token accounting,
budget provenance. The receipt used to skip itself when there were no memories
and no pack items, which is most chat turns, so the one surface that could have
shown this did not exist for the threads that had it. Surfaced in the thread
context inspector, 13 locales.
context-composer.live-replay.spec.ts replays the 24 real production threads:
the shipped selector starved 18 of them, production lost the fact in all 18,
and the composer restores it in 24/24. The 19th live failure was a model
refusal, asserted separately so the two are never conflated.
Gates: chat-service 1640 tests, routing-service 1391 tests, typecheck + lint +
build green on both; frontend typecheck + lint green, 13/13 locales complete.
NOT included, and not claimed: cross-thread retrieval (does not exist),
hierarchical summarisation, structured supersession, vector same-thread
retrieval, and chat's migration off the legacy memory route. See ADR-084.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n off ADR-084 fixed what a thread knows about itself. This is the other half of the complaint: starting a new conversation about a project discussed last week and finding ClawAI has never heard of it. Measured before this change, a thread that spent three turns establishing facts about MERIDIAN-88 was invisible to a new thread asking to continue it. Done carelessly this produces the worst behaviour a conversational product can have — answering from a conversation the user is not in, about a project they did not mention. The design is shaped more by what it refuses than what it finds. OFF BY DEFAULT. `useCrossThreadContext` defaults false; the migration adds the column with DEFAULT false so every existing thread is unchanged. When off the repository is never called: opt-out means "not read", not "read and then discarded", because the second still exposes the data to a bug in whatever discards it. TWO STAGES. Stage 1 asks the database which of this user's non-archived other threads mention the prompt's salient terms, ranked by matching-message count; stage 2 reads only the top three and scores individual messages. A single-stage search would surface a sentence sharing vocabulary with the prompt, torn out of a conversation about something else. A COINED IDENTIFIER IS THE PRECISION GATE. When the prompt contains one it is the only thing searched. "Continue the MERIDIAN-88 project, which package manager?" searched on all its terms matches every thread that ever mentioned a package manager; searched on MERIDIAN-88 it matches the one the user means. USER-SCOPED TWICE. No repository method can be called without a userId, and stage 2 re-proves ownership rather than trusting the ids stage 1 handed it. Archived threads are excluded; deleted threads leave nothing to retrieve because messages cascade on delete. 15% OF THE INPUT BUDGET, subtracted before the composer runs, so retrieved material can never displace the conversation the user is actually in. SEVEN NAMED SKIP REASONS written to the receipt with the threads searched and used, so "nothing was retrieved" is never ambiguous. Retrieval fails silent: an error records RETRIEVAL_FAILED and the turn continues. Two things the live run taught, both now encoded: Ranking on the thread TITLE alone failed its first test. A thread that had discussed MERIDIAN-88 for three turns carried a title that did not name it, scored 0.03 against a 0.28 threshold, and was never read. A title is auto-derived, often absent, and renameable. The evidence that a thread is about something is in the thread, so ranking now counts matching messages. The toggle returned 200 OK and did nothing, because ChatThreadsService maps DTO fields to repository fields one line at a time and nobody added the new line. Nothing in the type system catches it — every field is optional, so an object missing one is still valid. Added a source-level guard that reads both DTO schemas and asserts every field is named on the corresponding write, scoped per method (the first version read the whole file and passed over the very bug it was written for). Verified live, three threads on kimi-k3: toggle off retrieves nothing (DISABLED); toggle on finds the MERIDIAN-88 threads and answers pnpm + Frankfurt; a project never discussed returns NO_CANDIDATES. In the third case the model invented an answer anyway — the manifest shows nothing was retrieved, so that is a hallucination and not a leak. Scoring on the manifest rather than the model's words is what keeps those apart. Also in this batch, the before/after for ADR-084 measured at scale rather than asserted. Production baseline: 1209 turns, 168 probes, 30 passed (18%), and both passing probe classes are the two whose wording happens to overlap the seeding sentence. The same gauntlet against a deployment running the composer: 80/85 (94%). recall@d4 0/15 -> 8/8. latest-value 0/15 -> 8/8. assistant-response recall 0/14 -> 7/8. topic return 0/3 -> 3/3. 2,427 free-model generations across ten runs, no metered model executed. Gates: chat-service 1655 tests, typecheck + lint + build; frontend typecheck + lint; 13/13 locales complete. NOT included: semantic/vector recall (this is term matching — a thread about "Postgres" will not match a prompt about "relational databases"), hierarchical summarisation, structured supersession, and chat's migration off the legacy memory route. See ADR-085. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
084 is already taken by in-flight SEO work on main (SEO clusters fan out from one dynamic route). Two ADRs claiming the same number is not a cosmetic clash: every cross-reference in the code comments resolves to the wrong decision, and the index cannot hold both. Renumbered before the collision reached main rather than after. 085 is left free in case the SEO stream needs it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ADR-086 and ADR-087 each widened what one request can read — a manifest naming message ids, a receipt naming prior threads, and a retrieval path that deliberately reads OTHER conversations. Every one of those is a place a missing owner filter leaks a different customer's chat, and none of them had a test that a second real account could not reach them. Thirteen probes with a second ordinary USER: read the thread, its messages, one message, the context receipt, preview its context, post into it, flip its cross-thread toggle, delete it, branch it, search inside it, read the receipt unauthenticated, call the internal routing route with a user token, and — the decisive one — enable cross-thread retrieval on the attacker's own thread and ask for the victim's planted secret by name. Result: 13/13 denied, no secret in any response body, `priorThreadsUsed` empty on the cross-user probe. A 400 is scored as a FAILURE, not a denial. The first run reported one because the probe used the wrong query parameter name; counting 400 as "denied" would have recorded safety for a request the server never evaluated, which is exactly how a security suite comes to certify something it did not measure. The attacker is created through the admin route because self-registration gates login behind email verification. It is an ordinary USER with no elevated permissions — how it was created says nothing about what it can reach. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…outes need a service token Two findings from the 2026-08-30 audit, closed together because they touch the same four call sites. F-05 — the preview described a different code path from the answer. Chat generation read `GET /internal/memories/for-context`: a user's most recent N memories, with no intent, no ranking, no score, no retrieval reason and no usage telemetry. The scoring that decided which of them a model actually saw then happened inside a chat-service method, out of reach of the service that owns memory. `POST /chat-threads/:id/preview-context` — the endpoint behind "what will the AI see?" — already used the canonical `POST /internal/memories/retrieve`. So the preview a user was shown was produced by different code from the generation it claimed to describe, and the two could disagree with nothing anywhere reporting that they had. Verified live: preview and generation now return identical memory id lists, and a saved standing INSTRUCTION reaches the model and is obeyed. Security — memory-service's internal routes had no service identity check. They were `@Public()` with nothing behind it, while five of the six services that expose internal routes already had a `ServiceTokenGuard`. They are not reachable from the internet: nginx routes exactly one `/api/v1/internal/*` prefix (chat-shares) and everything else falls through to the frontend, which the audit confirmed by probing both deployments. But the routes take a `userId` as a plain query parameter and return that user's memories, so anything able to reach the container could read any account's memories by guessing an id — and "nginx does not route it" is one config line away from being false. Applied in the safe order: all four callers gained the header first (additive, cannot break anything), then the guard. The four were chat-service's generation and preview paths and workspace-service's learned-preference read and automation-preference write — none of which had ever sent one. The guard mirrors routing-service's and auth-service's exactly; the three must agree on the header format or the hop fails closed and looks like an outage. Constant-time comparison, so the shared secret cannot be recovered a byte at a time from response latency. Gates: memory-service 88 tests, workspace-service 1136, chat-service 1655; typecheck + lint + build green on all three. Live: 4/4 on the memory experiment after the guard was enabled, which is also the proof the header wiring is right. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ackend every turn
Instrumented first, because the obvious measurement was wrong. End-to-end turn
latency across the gauntlet produced a suspiciously flat 5.8 s p50 at every
thread length — that was the QA harness's poll interval, not the server. A
latency number that constant should be read as a broken instrument.
So the manifest now carries the server's own numbers, split by where the time
goes:
retrievalMs — network. Memories, packs, files, workspace and cross-thread,
fetched concurrently. Flat in thread length by construction.
selectionMs — the composer's own work: grouping into turns, scoring, fitting
to budget. The one that could grow.
Keeping them apart is what makes "context assembly got slower" distinguishable
from "memory-service got slower". Both are surfaced in the thread context
inspector alongside the cross-thread counts, in 13 locales.
What the measurement then found.
selectionMs is 0 ms at 9, 29, 59 and 99 messages and 1 ms at 159 — while the
composer sent EVERY message in the thread (159 of 159, 80 turns, 3,741
tokens). Selection is not the cost, at any length tested.
retrievalMs was 3,845–3,861 ms on every single turn. memory-service embeds
the query before searching; with no embedding model installed the call fails
after ~4 s, retrieval swallows the failure and returns results anyway — and
pays the four seconds again on the very next turn, and the one after that.
The failure was always there. Migrating chat generation to the canonical
retrieval route (F-05) is what put it in front of every message.
`embedding-circuit.utility.ts` opens after three consecutive failures — not
one, because a single timeout is a blip and opening on it would disable
semantic search for a transient hiccup — stays open 30 s, and closes on the
next success. Module-level rather than injectable: it protects one external
dependency shared by every caller in the process, and a per-instance breaker
would open once per collaborator instead of once.
Measured on the same stack, same script, before and after:
retrieval 3,845–3,861 ms -> 8–16 ms
selection 0–1 ms -> 0–1 ms
A dead embedding backend now costs one timeout per half-minute instead of one
per turn, and an operator who installs the model gets semantic search back
within 30 s without restarting anything.
Gates: memory-service 95 tests, chat-service 1657, typecheck + lint + build
green; frontend typecheck + lint green, 13/13 locales.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Found by measuring, not by reading. A concurrency baseline over the context path showed `selectionMs` flat at 0-1ms from 1 to 16 concurrent generations — the composer is not the bottleneck at any level — while `retrievalMs` jumped from ~26ms to a flat 5000ms at 16 concurrent. 5000 is exactly chat-service's memory timeout, so retrieval was not slow, it was being starved. memory-service makes three unattended calls to ollama-service: embeddings for semantic search, generation for memory extraction, generation for sensitivity classification. All three swallow their failures so a missing model degrades the feature rather than breaking the request, which is right — and hid that each was retried on the very next message. With no model installed, extraction fails after ten seconds and runs once per message. Sixteen of those in flight starved the retrieval path into its own timeout: a dead OPTIONAL feature taking down a working one. The breaker is keyed, not global. Embeddings and generation are different endpoints that fail independently, and one breaker for both would disable working semantic search because extraction was down. Half-open is the part that actually fixed it, and the first attempt without it is worth recording: a breaker that only counts failures throttles how OFTEN a dead dependency is hammered but not how MANY calls go at once. The moment its 30-second window expired, all sixteen waiters were admitted together and the starvation returned unchanged — measured, not predicted. Now exactly one call is admitted as a trial while everyone else keeps failing fast until it resolves. retrievalMs p50/max at 16 concurrent, same stack, same script: no breaker 5000 / 5000 (timeout) breaker, no half-open 5000 / 5001 (timeout) breaker + half-open 24 / 3861 (the 3861 is the single probe) Also bounds the extraction timeout in a named constant rather than an inline 10_000, so it can be lowered without hunting for it. Ships with the rollout guide: deploy order (callers before memory-service, or every retrieval 401s), what to watch in the receipt, and why there is no feature flag — the off state of one here would be the shipped defect. Gates: memory-service 98 tests, typecheck + lint + build green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ontext-v2 # Conflicts: # .ai/manifests/hashes.json # .ai/manifests/services.json # .ai/manifests/tests.json # docs/13-adr/adr-index.md # docs/features/ai-native-engineering-os/inventory.snapshot.json
…ontext-v2 # Conflicts: # .ai/BOOTSTRAP.md # .ai/manifests/hashes.json # .ai/manifests/services.json # .ai/manifests/tests.json # docs/features/ai-native-engineering-os/inventory.snapshot.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A user reported that ClawAI forgets the earlier part of a long conversation. It does — but not because the conversation is long.
Reproduced against production: total context loss at turn three of a three-turn thread.
The measurement that settled it
One planted fact, one thread, one distance, six free models, the same question phrased four ways:
What is my access code for this session?Which secret string did I share at the start?Remind me of the credential I mentioned earlier.Repeat it back to me.24 of 24 threads matched the prediction computed from the selector's own arithmetic before the run. A separate breadth run scored 19/19 free models at 100% on the high-overlap phrasing, which rules the models out as the variable.
Recall was a function of how many four-letter-or-longer words the question happened to share with the sentence that stated the fact. Rephrase the question and the fact vanishes.
Root cause
Three caps in series, each unaware of the others:
chat-messages.service.tsfindRecentByThreadId(threadId, 20)context-assembly.manager.tsthreadMessages.slice(-20)filterThreadMessagesForIntentASSISTANT· overlap ≥ 0.45 ·slice(-4)Plus a 1,024-token clamp on the AUTO fast path (triggered by short prompts — exactly the ones that depend on long histories), and
tokenBudget = maxTokens ?? 4096: an output length used as the whole-prompt budget.contextWindowTokensexisted in routing-service, connector-service and the frontend, and in zero files of chat-service.What changed
One rule: nothing removes a message for being irrelevant; only the token budget removes. Relevance decides order, and order only matters once the budget is full.
ModelTokenBudgetsplits five quantities that were one;maxTokensnow feedsreservedOutputTokensand nothing elseResults, measured on a deployment running this code
The 60-turn gauntlet: 30/168 (18%) → 80/85 (94%). Excluding the two probe classes whose wording overlaps the seeding sentence, production scored 1 of 149.
Authorization: 13/13 denied, zero leakage, including the decisive one — an attacker enabling cross-thread retrieval on their own thread and asking for the victim's secret by name.
Assembly cost (server-side, from the manifest): selection 0–1 ms at 159 messages while sending all 159. Retrieval 3,850 ms → 8–16 ms after the circuit breaker; 5,000 ms (timeout) → 24 ms at 16 concurrent after adding half-open.
2,427 free-model generations across 13 runs. No metered model executed — enforced in code, not by naming convention.
Reviewing this
Start with
docs/13-adr/adr-086-conversational-context-composer.md— it carries the evidence and the reasoning. Thenadr-087for cross-thread retrieval, which is shaped more by what it refuses than by what it finds.context-composer.live-replay.spec.tsreplays 24 real production transcripts as a committed fixture: the shipped selector starved 18 of them, production lost the fact in all 18, and the composer restores it in 24/24. The 19th live failure was a model refusal, asserted separately so the two are never conflated.Deploy order matters
chat-service and workspace-service before memory-service. Both gained service-auth headers; memory-service now requires them. Backwards means every retrieval 401s — not an outage (memory is non-blocking) but memories silently disappear. Full procedure in
docs/08-runtime-devops/conversational-context-rollout.md.Not in this PR, stated plainly
glm-5.3isACTIVE+isExecutionCapablein the router catalog and returns403 MODEL_NOT_EXPOSEDon every send. Unrelated, still open, filed here because the router offers a model users cannot use.Gates
chat-service 1,658 tests · routing-service 1,391 · workspace-service 1,136 · memory-service 98. Typecheck, lint and build green on all four and the frontend. 13/13 locales complete. Merged with
main(v1.58.0) and re-verified.🤖 Generated with Claude Code