feat(qa): HNSW filtered retrieval + temporal decay regression tests - #3
feat(qa): HNSW filtered retrieval + temporal decay regression tests#3mfethe1 wants to merge 1 commit into
Conversation
…on tests - Migration 013: IVFFlat → HNSW index (m=16, ef_construction=200) - Adds composite indexes for agent_id + created_at, memory_type + created_at - Enables pre-filtered ANN search instead of post-filtering - Sets hnsw.ef_search=100 on connection pool init for quality recall - 15 regression tests for temporal decay scoring: - Recency ranking, access boost diminishing returns - Deduplication threshold, edge cases - Full ranking order simulation - All tests passing QA lane: Lenny | Coordination: NATS Rosie's research (Proposal A + B) operationalized into testable code.
🤖 Augment PR SummarySummary: This PR upgrades memU’s retrieval stack to support higher-quality filtered ANN search and adds regression coverage for temporal decay scoring. Changes:
Technical Notes: The migration assumes a pgvector version with HNSW support (and ideally filtered search support); index rebuild and current deployment outage may affect rollout sequencing. 🤖 Was this summary useful? React with 👍 or 👎 |
![augmentcode[bot]](https://avatars.githubusercontent.com/in/1027498?s=60&v=4){kind=small}
| pool = await asyncpg.create_pool(DATABASE_URL, min_size=2, max_size=10) | ||
| async def _init_hnsw_search(conn): | ||
| """Set HNSW ef_search for quality filtered retrieval (migration 013).""" | ||
| await conn.execute("SET hnsw.ef_search = 100") |
There was a problem hiding this comment.
SET hnsw.ef_search = 100 will raise unrecognized configuration parameter on clusters running older pgvector / without HNSW, and because it runs during pool init (before migrations) it can prevent the service from starting (and thus prevent applying migration 013). Consider making startup resilient when the parameter isn’t available.
Severity: high
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| -- Step 2: Create HNSW index on embedding column | ||
| -- m=16, ef_construction=200 are good defaults for our ~3000 memory corpus. | ||
| -- For larger corpora (>100k), increase ef_construction to 256-512. | ||
| CREATE INDEX idx_memories_embedding_hnsw |
There was a problem hiding this comment.
This migration drops/rebuilds the vector index with a plain CREATE INDEX, which takes a strong lock on memories and can block reads/writes during creation on production-sized tables. Please make sure rollout expectations/downtime match this (or that migrations are run in a way that won’t impact availability).
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| changes are merged. They validate: | ||
| 1. Temporal decay scoring ranks recent memories higher | ||
| 2. Access count boosts work with diminishing returns | ||
| 3. Agent-scoped filtering returns only matching memories |
There was a problem hiding this comment.
The module docstring says this suite validates agent-scoped filtering, but there isn’t currently a test that exercises retrieval with an agent_id (or similar) filter. This could give a false sense of coverage for the filtered retrieval upgrade.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| Use a fixed timestamp to avoid sub-second drift between calls. | ||
| """ | ||
| ts = datetime(2026, 1, 1, tzinfo=timezone.utc) | ||
| a = compute_final_score(similarity=0.75, created_at=ts, access_count=3) |
There was a problem hiding this comment.
compute_final_score() calls datetime.now() internally, so the two calls in this determinism test can diverge slightly under slow/loaded CI, and abs=1e-9 may be flaky. Consider freezing time/injecting now for tests or relaxing the tolerance.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4976b06d53
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| pool = await asyncpg.create_pool(DATABASE_URL, min_size=2, max_size=10) | ||
| async def _init_hnsw_search(conn): | ||
| """Set HNSW ef_search for quality filtered retrieval (migration 013).""" | ||
| await conn.execute("SET hnsw.ef_search = 100") |
There was a problem hiding this comment.
create_pool(..., init=_init_hnsw_search) now executes SET hnsw.ef_search = 100 on every new connection, and this happens before run_migrations(pool) is called. In environments where that GUC is unavailable (for example, pgvector not installed yet in the DB or a version that does not expose hnsw.ef_search), pool creation will raise and the API will fail to start instead of degrading gracefully as the migration path currently does.
Useful? React with 👍 / 👎.
1 participant
What
Operationalizes Rosie's Proposal A (Temporal Overlay) and Proposal B (Filtered Retrieval Upgrade) from the 2026-03-07 research rotation.
Changes
Migration 013: IVFFlat → HNSW
Connection Pool
hnsw.ef_search = 100on every pool connection for quality recall15 Regression Tests
Test Results
Blockers
Coordination
NATS event published to
agent.coordination. Team assignments: