fix(knowledge): rank session-start injection by relevance, cap per entity, disclose the rest

.claude/rules/65-capped-selection-must-rank-and-disclose.md

@@ -0,0 +1,135 @@
+# A Capped Selection Must Rank By Its Purpose, And Must Disclose What It Dropped
+
+## When This Applies
+
+Any query, filter, or fan-out that returns **a bounded subset of a larger set to a
+consumer who cannot see the rest** — `LIMIT N`, `slice(0, N)`, top-K, "first N matching",
+sampling. It applies with full force when the consumer is an LLM, because a model cannot
+notice an absence: it will confidently reason from a truncated set as though it were
+complete.
+
+It does NOT apply to paginated reads where the consumer receives a cursor, a `hasMore`, or
+a total — those already disclose the truncation.
+
+## Why This Rule Exists
+
+`getAllHighConfidenceKnowledge` selected session-start knowledge injection with:
+
+```sql
+WHERE confidence >= 0.8 ORDER BY entity_name, last_confirmed_at DESC LIMIT 50
+```
+
+`ORDER BY entity_name` is alphabetical. Alphabetical order has nothing to do with how
+useful an observation is. So the cap did not select the 50 most useful observations — it
+selected the 50 that sorted earliest, which in production meant **all 50 slots went to
+`AccountMap` through `AgentReliability`, 46 of them to a single `AgentBehavior`
+grab-bag entity.**
+
+Meanwhile the very same payload instructed the agent to consult `ContentStyle`,
+`CodeQuality`, `User`, `Architecture`, and `BusinessStrategy` before making decisions.
+Every one of those sorts after "A". None of them had **ever** been injected. Worse, the
+payload gave no hint they existed, so an agent could not have searched for them — it had
+no reason to believe there was anything to search for.
+
+The result was stable, silent, and systematically biased: the same topics were excluded
+every session, for months, while the feature looked like it was working. It was found by
+measurement, not by failure.
+
+## Class of Bug
+
+**A cap whose ordering key is uncorrelated with the cap's purpose, with no disclosure of
+what was excluded.** Two independent defects that compound:
+
+1. *Wrong ranking* — the ordering encodes an incidental property (name, insertion order,
+   id) rather than the value the consumer needs. The cap then acts as a systematic filter
+   on that incidental property.
+2. *Silent truncation* — the consumer receives no count, index, or marker, so the missing
+   data is indistinguishable from data that does not exist.
+
+Either alone is a bug. Together they are undetectable from the inside: every component is
+"working", the payload is well-formed, and the tests pass.
+
+The tells:
+
+- `ORDER BY <name|id|created_at>` immediately above a `LIMIT` on a set the consumer treats
+  as authoritative.
+- A comment justifying the cap on size grounds ("for typical projects this is small")
+  without saying what happens when it is not typical.
+- Instructions that reference specific items by name, with no guarantee those items are in
+  the payload.
+- A grab-bag entity/bucket/tag that legitimately holds far more rows than its siblings.
+
+## Hard Requirements
+
+1. **Rank by the consumer's purpose.** If the cap exists because the consumer can only use
+   N items, order by how useful an item is to that consumer — relevance, confidence,
+   recency, priority. Never by a name, id, or insertion order that merely happens to be
+   available. If you cannot articulate why the ordering key correlates with usefulness,
+   the ranking is wrong.
+
+2. **Reuse the ranking the system already has.** If a sibling read already scores the same
+   entities (here `getRelevantKnowledge`'s confidence x recency), use that formula rather
+   than inventing a second one, so two paths cannot disagree about what "most relevant"
+   means (rules 24, 59).
+
+3. **Cap per group when one group can dominate.** A global `LIMIT` over a skewed
+   distribution is a cap in name only — the largest group absorbs it. Add a per-entity /
+   per-group / per-tenant bound, env-configurable with a `DEFAULT_*` constant.
+
+4. **Disclose the truncation.** The consumer must receive enough to know the set is
+   partial and to retrieve the remainder: a complete lightweight index, a total count, or
+   an explicit "N more, retrieve with X". Naming the retrieval tool is part of the
+   disclosure — "there is more" without "here is how" is not actionable.
+
+5. **The disclosure must survive the failure of the main read.** Fetch it independently.
+   If ranked retrieval throws and the catch swallows it, the consumer should still learn
+   what exists rather than silently receiving nothing — which is exactly the pre-existing
+   behaviour that made this class invisible.
+
+6. **Give ties a total order.** Equal-scoring rows must be broken deterministically (a
+   secondary key, then a unique id). Otherwise the payload permutes between identical
+   calls, which defeats prompt caching and makes differences unreproducible.
+
+## Required Tests
+
+- **The skew case**: seed one group large enough to exhaust the global cap on its own, plus
+  groups that sort *after* it under the old ordering. Assert the later groups are selected.
+  This must FAIL against the old ordering — verify that once.
+- **The per-group cap holds**, with an owner control proving the freed slots actually reach
+  the other groups rather than vanishing.
+- **A raised-cap control** proving the cap parameter is what constrains the dominant group,
+  not the shape of the seeded data.
+- **Disclosure completeness**: an item excluded from the capped set still appears in the
+  index. Assert the count, not just presence.
+- **Determinism**: identical repeated calls return an identical sequence.
+- Ranking that is a SQL predicate must be tested against a **real** SQL engine, not a
+  mock that ignores `ORDER BY` (rule 28). For Durable Object SQLite that means
+  `@cloudflare/vitest-pool-workers` in `apps/api/tests/workers/`, driven through the real
+  RPC — not the SQL re-implemented in TypeScript, which would pass even if the query never
+  ran (rule 62).
+
+## Quick Compliance Check
+
+- [ ] The ordering key is one a reader would call "most useful first", not a name or id
+- [ ] An existing scoring formula was reused rather than duplicated
+- [ ] A per-group cap exists wherever one group can dominate; both bounds are env-configurable
+- [ ] The consumer is told the set is partial, how many exist, and which tool retrieves the rest
+- [ ] The disclosure is fetched independently of the capped read
+- [ ] Ties break to a total order
+- [ ] The skew test was verified to fail against the pre-fix ordering
+
+## References
+
+- Task: `tasks/active/2026-08-23-knowledge-injection-relevance-ranking.md` (moves to
+  `tasks/archive/` on completion); research `/engineering/research/token-optimization-research.md` §3.2, §8/R3
+- Implementation: `apps/api/src/durable-objects/project-data/knowledge.ts`
+  (`getAllHighConfidenceKnowledge`, `getKnowledgeEntityIndex`)
+- `.claude/rules/53-scheduled-handler-isolation-and-liveness-signals.md` — a signal that
+  cannot answer the question being asked of it; "the symptom is an absence"
+- `.claude/rules/50-list-read-row-fault-isolation.md` — the other silent-truncation mode
+- `.claude/rules/02-quality-gates.md` — the absence of failures and the absence of results
+  are indistinguishable if you only look at the failure count
+- `.claude/rules/28-credential-resolution-fallback-tests.md` — SQL predicates need a real
+  SQL engine, and every case needs a control
+- `.claude/rules/62-tests-must-observe-the-real-trigger.md` — prove the guard discriminating
+- `.claude/rules/24`, `.claude/rules/59` — one implementation per operation

CLAUDE.md

@@ -248,6 +248,8 @@ Tasks tracked as markdown in `tasks/` (backlog -> active -> archive). See `tasks
 
 Use the `/changelog` skill for structured queries.
 
+- knowledge-injection-relevance-ranking: Session-start knowledge injection (`get_instructions`) is ranked instead of alphabetical. `getAllHighConfidenceKnowledge` previously used `ORDER BY e.name ... LIMIT 50`, so the cap filtered on entity *spelling*: in production all 50 slots went to `AccountMap`..`AgentReliability`, 46 of them to the one `AgentBehavior` grab-bag, while `ContentStyle`/`CodeQuality`/`User`/`Architecture`/`BusinessStrategy` — entities the same payload tells agents to consult — had never been injected once, with no hint they existed. Now scored by the EXISTING formula (`computeRelevanceScore`: confidence × 1/(1 + age/30d) on `last_confirmed_at`, so `confirm_knowledge` restores rank), mirrored into SQL and pinned by a parity test; per-entity cap via `ROW_NUMBER() OVER (PARTITION BY entity_id)`; `now` bound as a parameter and ties broken to a total order (`score DESC, last_confirmed_at DESC, id ASC`) so output is reproducible. New `getKnowledgeEntityIndex` appends a compact `Name (type, N)` index to `knowledgeDirectives` disclosing what was NOT injected and naming `search_knowledge`/`get_relevant_knowledge` as the retrieval path; it returns `{entries, totalEntities}` (total via `COUNT(*) OVER ()`) so a truncated index can never be labelled "full". The three DO reads run as an isolated `Promise.allSettled` fan-out — a failed ranked read still leaves the index. Limits are clamped at the DO boundary (`clampRowLimit`): a negative `perEntityLimit` would otherwise make `entity_rank <= ?` unsatisfiable and inject nothing project-wide, and `LIMIT -1` means *unbounded* in SQLite. New env vars `KNOWLEDGE_AUTO_RETRIEVE_PER_ENTITY_LIMIT` (8), `KNOWLEDGE_ENTITY_INDEX_LIMIT` (200). Process fix: `.claude/rules/65-capped-selection-must-rank-and-disclose.md`.
+
 - byo-mcp-servers: Bring-your-own MCP endpoints. Users store `{name, url, authType, token, enabled}` at personal (`/api/mcp-connections`) or project (`/api/projects/:projectId/mcp-connections`) scope; SAM injects them into every agent session alongside `sam-mcp`. Both URL and token are AES-256-GCM encrypted and never returned by a read path (several providers issue pre-signed URLs with the credential in the URL, so the URL is a secret; `url_host` is the display value). Migration `0120_mcp_connections`. `buildSessionMcpServers` is the single composition point, called by `agent-session-bootstrap.ts` (covers VM + cf-container, rule 61) and the manual workspace agent-session route; the anonymous trial path is deliberately pinned to `sam-mcp` only. Resolution skips-and-warns per row so one bad connection cannot brick session start (rules 41/50). vm-agent: `McpServerEntry.Name` (additive, rule 54) plus `ResolveMcpServerNames` as the single naming source of truth — it replaced three drifted copies, one of which named a lone server `sam-mcp-0` for Vibe and `sam-mcp` everywhere else; persistence `migrateV12`. Codex's startup precondition, which required a bearer token for EVERY injected server, is now scoped to the reserved `sam-mcp` entry so a no-auth connection cannot break all Codex sessions. UI: Settings → MCP Servers (personal) and Project Settings → Runtime (project), one shared `McpServersManager`. v1 is bearer/none auth and personal/project scope only; custom headers and profile/skill attachment are tracked in idea `01M0QDASJCK3YWVX1GETZTSFWZ`. Limits: `MAX_MCP_CONNECTIONS_PER_SCOPE`, `MCP_CONNECTION_URL_MAX_BYTES`, `MCP_CONNECTION_TOKEN_MAX_BYTES`.
 - report-issue-idea-flow: Hosted "Report an Issue" flow that groups reports into private feedback incidents and creates/updates a linked draft Idea in the effective private feedback project (Admin → Integrations runtime setting, falling back to `PLATFORM_FEEDBACK_PROJECT_ID`). Two entry points: SessionHeader action row (expanded) and ErrorBoundary crash screen. Users explicitly consent before technical refs (sessionId, taskId, nodeId) are attached. Server-side cross-tenant ref authorization validates project membership before storing references. User text sanitized with secret/PII redaction and fenced with provenance markers. Feature auto-hidden when no effective feedback project exists or it does not reference an existing project in the current deployment database. Configurable limits: `REPORT_ISSUE_TITLE_MAX_LENGTH`, `REPORT_ISSUE_DESCRIPTION_MAX_LENGTH`, `REPORT_ISSUE_CONTENT_MAX_LENGTH`.
 - claude-opus-5-model-catalog: Claude Opus 5 (`claude-opus-5`, released 2026-07-24, $5/$25 per MTok, native 1M context) added to both canonical model lists — `CLAUDE_MODELS` dropdown catalog and `PLATFORM_AI_MODELS` proxy allowlist/pricing. Retired `claude-sonnet-4-20250514` pruned; stale defaults bumped to `claude-sonnet-5` (`DEFAULT_SAM_MODEL` — was pointing at the retired model, `DEFAULT_AI_PROXY_ANTHROPIC_MODEL`, `DEFAULT_TRIAL_MODEL_PRODUCTION`). New regression tests pin retired-Anthropic-model exclusion and default↔catalog registration; process rule `.claude/rules/52-model-catalog-lifecycle.md`.

apps/api/.env.example

@@ -224,6 +224,26 @@ BASE_DOMAIN=workspaces.example.com
 # TRIAL_SSE_MAX_DURATION_MS=1800000
 # TRIAL_CRON_ROLLOVER_CRON="0 5 1 * *"
 # TRIAL_CRON_WAITLIST_CLEANUP="0 4 * * *"
+# ─── Knowledge Graph ─────────────────────────────────────────────────────────
+# Per-project knowledge store, and the session-start injection that feeds it to agents
+# via the MCP `get_instructions` tool. All optional; defaults shown.
+# KNOWLEDGE_MAX_ENTITIES_PER_PROJECT=500
+# KNOWLEDGE_MAX_OBSERVATIONS_PER_ENTITY=100
+# KNOWLEDGE_SEARCH_LIMIT=20
+# KNOWLEDGE_SEARCH_MAX_LIMIT=100
+# KNOWLEDGE_AUTO_RETRIEVE_LIMIT=20
+# KNOWLEDGE_AUTO_RETRIEVE_MIN_CONFIDENCE=0.8      # Confidence bar for session-start injection
+# KNOWLEDGE_AUTO_RETRIEVE_HIGH_CONFIDENCE_LIMIT=50 # Total observations injected at session start
+# KNOWLEDGE_AUTO_RETRIEVE_PER_ENTITY_LIMIT=8      # Max from any ONE entity, so a grab-bag
+                                                  # entity cannot consume every slot
+# KNOWLEDGE_ENTITY_INDEX_LIMIT=200                # Entities listed in the injected index that
+                                                  # discloses what was NOT injected
+# KNOWLEDGE_OBSERVATION_MAX_LENGTH=1000
+# KNOWLEDGE_ENTITY_NAME_MAX_LENGTH=200
+# KNOWLEDGE_DESCRIPTION_MAX_LENGTH=2000
+# KNOWLEDGE_LIST_PAGE_SIZE=50
+# KNOWLEDGE_LIST_MAX_PAGE_SIZE=200
+
 # TRIAL_KNOWLEDGE_GITHUB_TIMEOUT_MS=10000
 # TRIAL_KNOWLEDGE_MAX_EVENTS=50
 # TRIAL_ORCHESTRATOR_OVERALL_TIMEOUT_MS=300000

apps/api/src/durable-objects/project-data/index.ts

@@ -1357,8 +1357,16 @@ export class ProjectData extends DurableObject<Env> {
     return knowledge.getRelevantKnowledge(this.sql, context, limit);
   }
 
-  async getAllHighConfidenceKnowledge(minConfidence: number, limit: number) {
-    return knowledge.getAllHighConfidenceKnowledge(this.sql, minConfidence, limit);
+  async getAllHighConfidenceKnowledge(
+    minConfidence: number,
+    limit: number,
+    perEntityLimit?: number
+  ) {
+    return knowledge.getAllHighConfidenceKnowledge(this.sql, minConfidence, limit, perEntityLimit);
+  }
+
+  async getKnowledgeEntityIndex(limit?: number) {
+    return knowledge.getKnowledgeEntityIndex(this.sql, limit);
   }
 
   async createKnowledgeRelation(