| type | Reference | ||||||
|---|---|---|---|---|---|---|---|
| title | Current ELF product direction | ||||||
| description | Separate the active memory-value hypothesis from stable intent and historical delivery plans. | ||||||
| status | active | ||||||
| authority | informative | ||||||
| owner | product | ||||||
| last_verified | 2026-10-15 | ||||||
| related |
|
Purpose: Identify the next useful product question and its evidence boundary. Read this when: Planning, implementing, or reviewing an ELF change. Not this document: A second product contract, a task queue, or permission to run a paid experiment. The product contract owns the goal and stable principles.
ELF should prove that source-backed project memory improves continuity and decision updates across real agent sessions. Its incremental value over capable source access is still a hypothesis. Provenance, correction history, and scoped access are useful foundations, but their existence alone is not unique market value or customer acceptance.
The small native system comparison adds unchanged ELF to the frozen OpenViking and RAGFlow cases. ELF completes 13/13 answers semantically and 12/13 with valid exact quotes; each other product completes 11/13 on both measures. A native parent read supplies a current retry decision missing from selected memory text. The checker detects the one bad quote. Keep the working source and evidence path; this does not justify replacing ELF for answer quality or expanding all its engines. The run has no source-only arm, no graph use, and no native keyed corrections. It does not overturn the real-project incremental-value limits below.
The final citation gate regression now blocks a rejected draft before delivery in the reference Agent host. The same thirteen saved drafts pass after one bounded correction of the failed quote; all fourteen originals remain unchanged. This verifies the output gate, not a new competitor score. External Agent clients still need to connect the native check to their own final-output hook.
The native capture/read loop now has evidence: MCP preserved six selected originals, and a fresh reference Agent read memory-linked originals and cited them with valid bindings. Four formal full-ELF readers used memory; three cited a memory-delivered original, while one used a current-source fallback. This does not install automatic capture into an external Agent client.
The resulting handoff comparison still did not establish incremental memory value. Full ELF and indexed originals both covered 21/24 frozen obligations and completed 1/4 strict checklists. Full ELF cost about 29% more per reader set. Files plus the same authored handoff covered 19/24 and took about half the time. These are narrow handoff results, not coding completion or customer acceptance. See the native loop evidence.
The earlier optional-tool maintenance trial had no native memory use and suffered prototype defects. Keep its limitations separate; it is not a before/after score for the native loop.
The preceding read-only evidence change raised original-task coverage from 46/64 to 49/64, while frozen new variants changed from 27/32 to 26/32. Those archive results remain mixed evidence. Keep their original scoring and boundaries separate from the maintenance trial; these are different tasks and integrations.
Keep the bounded source-admission fix, shared capture contract, HTTP setup, and actual-use gate. The temporary maintenance adapter remains retired. Do not add another framework, representation, store, or repair batch to obtain a favorable score. Preserve both completed experiments and their failed answers.
For short repository maintenance with an available current checkout, use normal file tools and an explicit handoff as the working approach. This is a scoped choice while incremental memory benefit is unproved. It does not remove memory from other workflows or change the shipped memory-first reader contract.
The historical 512-character replay recovered seven earlier rejections, but a fresh 520-character capture/calibration candidate still failed. The accepted correction now removes note and structured-text character admission gates and the silent three-candidate discard. Complete candidates share an ingestion budget; deferred outcomes and incomplete read packs are explicit. Search preview length no longer determines whether a qualified claim can be stored. Existing PostgreSQL authority, Qdrant retrieval, and original-source identity remain.
The complete-memory replay admitted 24 of 26 unchanged old candidates; two still failed the original importance/confidence policy. All three later candidates, including the 520-character one, were admitted without rewriting them. Their native text, child chunks, original hashes, and citation bindings were verified. The original four reader cells covered 22/24 obligations and completed 2/4 strict checklists, compared with 21/24 and 1/4 in the prior 512-character run. This small sequential change does not establish repeatable incremental memory value. All four readers used memory, but only one final answer cited the exact source span delivered with memory; the others used wider original-source windows.
The two remaining omissions were a capture hash/span detail and historical budget amounts. Both were already present in the returned original context. Do not treat these as evidence for another admission limit, store, or retrieval algorithm. Keep the frozen misses and the separately scored boundary cases in the complete-memory report. The request-budget decision supersedes the 512-character admission choice, while the previous results remain dated evidence.
Any later value test still needs a real continuity need, the intended host, verified capture, and actual memory use before comparison. The reference gate now enforces a memory-linked original read and checked final citation in each full-ELF calibration cell. Check semantic correctness separately; a binding can pass while the reader misinterprets a budget amount. A capable baseline may keep its own handoff notes. Keep human corrections and product benefit separate from functional and integration checks.
Current-original selection and subject/time misuse remain known boundaries. Reopen them only for an isolated regression or a real workflow, with a specific cause and an acceptance test. Future paid work still requires a frozen scope and admission under the existing cumulative ledger; unused budget is not a reason to continue experiments.
| Choice or hypothesis | Current disposition | Reconsider when |
|---|---|---|
| PostgreSQL authority plus Qdrant retrieval | Retain the supported deployment. Qwen3-Embedding-8B uses the full 4096-dimensional profile in the current experiment. | Representative quality, load, recovery, or operating-cost evidence supports a simpler or better design. A vendor speed claim is insufficient. |
| MMR | Keep enabled during the complete-memory regression. | A separate controlled test demonstrates better task outcomes. |
| Memory processing budget | Preserve complete claims. Use aggregate ingestion and read-response byte budgets with explicit gaps; 512 characters is a display preview only. | Representative workloads show excessive deferral, cost, or loss of qualified meaning. Measure the affected stage before changing a budget. |
| Mandatory memory opening in the reference host | Retain the shipped contract. The optional-tool maintenance trial did not exercise it. | A frozen comparison can show when memory-first, source-first, or direct current-file access is useful. This review does not change the shipped reader contract. |
| Compact facts as the main derived representation | Keep concise, complete claims with existing child-chunk retrieval and read-time bound originals. The extra model transcription into structured facts was rejected. | Observed workflows require a different representation and existing source/procedure references cannot meet the obligation. |
| Stronger reader or extraction model | Evaluate roles separately when justified. Model size and coding scores do not grant authority. | A bounded diagnostic separates generation, retrieval, and use failures and reports total cost. |
| Live verification of reusable lessons | Candidate for a narrow read-only probe through the application. | Current-state failures persist and an authorized source can resolve the uncertainty. Do not add broad background crawling. |
| More databases, a universal ontology, or a new agent harness | Defer. | A measured obligation cannot be met within the current owners at acceptable cost. |
Use a real workflow with new sessions and decision changes. Compare ordinary source access, indexed originals, and full ELF with the same available evidence, model, permissions, and budget. Include maintained native/file memory or a strong long-context control when that is the user's actual alternative. Do not require every baseline on every regression run.
Freeze the task outcome, minimum useful effect, critical errors, repetition plan, and budget before execution. Track task completion, user corrections, stale or wrong-scope actions, repeated discovery, elapsed time, and total cost. Include capture, extraction, indexing, maintenance, failures, and infrastructure; separate cold setup from reused preparation. No held-out task or answer may enter memory before its turn.
A limited pilot needs repeatable benefit on the selected workflow and no failures in its critical provenance, scope, correction, and deletion cases. Passing those cases does not prove universal reliability. If derived memory adds no useful benefit after a bounded test, narrow its use and simplify the path instead of expanding the architecture to defend it.
Update this reference when accepted evidence changes a disposition or next step. Link the dated report or decision and reconcile affected current specs in the same PR. Do not mark unrelated claims as freshly verified merely by changing a date.
Review is warranted when a relevant model/tool change challenges an assumption, a requirement changes, or a regression or pilot contradicts the expected result. The shared review method and any recurring coordination belong to Agentry issue 25, currently pending. ELF's existing weekly radar is a metadata signal source; it does not perform this judgment. No additional ELF review automation is established by these documents.