Skip to content

Connector-synced memories are never retrieved: deficient vectors, provider signature mismatch, raw Gmail bodies, global-only auto-recall #5300

Description

@yh928

Summary

Memories synced from a connector are unretrievable. A Gmail thread that was ingested successfully never comes back from recall, on a live instance.

Problem

Four independent failures stack up, each enough on its own to make a synced memory invisible:

  1. Deficient vectors are invisible and never repaired. A chunk whose batch embed failed is stored text-only, and a partial cloud failure writes degenerate short vectors. Recall drops any chunk whose dimension differs from the live query, so both are permanently unreachable — and nothing re-derives them.
  2. The Gmail response reshape lost its caller in the TinyCortex engine migration, so the sync stores raw provider JSON rather than one record per message.
  3. The tree's active signature names a different provider than the namespace store's embedder, so the two disagree about what a stored vector even is.
  4. Auto-recall is global-only. RecallOpts::default() resolves to the global namespace, so the per-turn context never searches connector namespaces at all.

Steps to reproduce:

  1. Connect Gmail and let the sync ingest a thread.
  2. Ask the agent about something only that thread contains.
  3. The agent answers as if the memory does not exist. A direct namespace query shows the row present but not retrievable.

Solution (optional)

Treat deficient vector_chunks rows as their own durable work-list and sweep them (startup, sync completion, login, embedder switch), writing a vector back only when its width matches the active embedder. Restore the reshape via an executor wrapper, carry the embedding provider into the tree config, and fan auto-recall out to the connector namespaces.

Acceptance criteria

  • Repro gone — a synced Gmail thread is retrievable through auto-recall on a live instance.
  • Sweep is restart-safe — the pending set is re-derived every pass, with no queue to lose.
  • No short-vector writes — a degraded provider cannot stamp a vector that stays pending forever.
  • Regression safety — coverage for the sweep, the reshape, the signature, and the namespace fan-out.
  • Diff coverage ≥ 80% — the fix PR meets the changed-lines coverage gate.

Related

Fix: #5258, which depends on tinyhumansai/tinycortex#132 landing first.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions