Repository navigation
🎯 feat: add RAG_DISTANCE_THRESHOLD to drop weak retrieval matches - #288
Conversation
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a3f599cb55
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| """ | ||
| if RAG_DISTANCE_THRESHOLD is None: | ||
| return documents | ||
| return [(doc, score) for doc, score in documents if score <= RAG_DISTANCE_THRESHOLD] |
There was a problem hiding this comment.
The new filter keeps entries with score <= RAG_DISTANCE_THRESHOLD, which assumes scores are distances (lower is better). That assumption does not hold for the Atlas path: AtlasMongoVector.similarity_search_with_score_by_vector forwards Atlas vectorSearchScore values (similarity scores where higher is better). In VECTOR_DB_TYPE=atlas-mongo deployments that enable RAG_DISTANCE_THRESHOLD (for example 0.5), this logic removes the best matches and keeps weaker ones in both /query and /query_multiple.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Good catch — fixed in fd61df7 by scoping the filter to pgvector only. Atlas (VECTOR_DB_TYPE=atlas-mongo) returns similarity scores (higher = better), so applying score <= threshold would keep the worst matches. The filter now short-circuits for Atlas and logs a one-time startup warning when the env var is set, so operators see their setting is being ignored. Happy to add a dedicated Atlas-aware min-similarity knob in a follow-up if there is demand.
|
@codex review |
danny-avila
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fd61df7a7d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| _distance_threshold_raw = get_env_variable("RAG_DISTANCE_THRESHOLD", None) | ||
| RAG_DISTANCE_THRESHOLD = ( | ||
| float(_distance_threshold_raw) | ||
| if _distance_threshold_raw not in (None, "") |
There was a problem hiding this comment.
RAG_DISTANCE_THRESHOLD is parsed as float unconditionally during config import, so any non-numeric value will raise ValueError and prevent startup even when VECTOR_DB_TYPE=atlas-mongo (where this setting is documented/implemented as ignored). This creates an avoidable outage path for Atlas deployments that carry a stale or templated non-numeric value for this env var; parsing should be conditional on pgvector (or validated with a safe fallback) before import-time initialization fails.
Useful? React with 👍 / 👎.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bace969f04
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| RAG_DISTANCE_THRESHOLD is not None | ||
| and VECTOR_DB_TYPE == VectorDBType.ATLAS_MONGO | ||
| ): |
There was a problem hiding this comment.
This condition can never fire in Atlas mode because RAG_DISTANCE_THRESHOLD is only parsed in app/config.py when VECTOR_DB_TYPE == PGVECTOR, so in atlas-mongo it remains None even if the operator sets RAG_DISTANCE_THRESHOLD. As a result, Atlas deployments silently ignore the setting instead of logging the promised startup warning, which can mislead operators into believing weak-match filtering is active and mask avoidable token-cost increases.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Good catch — fixed in 636f35c. Checking the raw env (os.getenv("RAG_DISTANCE_THRESHOLD")) in the Atlas warning branch instead of the parsed value, so operators who set the threshold under atlas-mongo actually see the startup warning that tells them their setting is being ignored.
Applies a post-search filter on /query and /query_multiple that discards any (document, score) pairs whose vector distance exceeds the configured threshold. The score returned by similarity_search_with_score_by_vector is a distance (lower = more similar), so this prunes the less-relevant tail that top-k search would otherwise surface. Unset by default — behavior unchanged for existing deployments. Closes #109
Atlas Vector Search returns similarity scores (higher = better), the opposite of pgvector's distances (lower = better), so the naive `score <= threshold` comparison would keep the weakest matches and drop the strongest under VECTOR_DB_TYPE=atlas-mongo. Skip the filter on Atlas and emit a one-time startup warning so operators notice their setting is being ignored. A dedicated Atlas-aware min-similarity knob can be added if there is demand.
Same class of bug codex caught on #287's EMBEDDINGS_DIMENSIONS: the float() cast ran unconditionally at import, so a stale or templated non-numeric value would prevent startup for atlas-mongo deployments where the setting is already documented as ignored. Gate the parse on VECTOR_DB_TYPE=pgvector so Atlas boots cleanly regardless of the env var value.
The parsed value is deliberately None under atlas-mongo (we skip the
float() cast so a stale non-numeric value can't break startup), so
checking it for the warning branch guaranteed the warning never fires.
Inspect os.getenv("RAG_DISTANCE_THRESHOLD") directly so Atlas operators
actually see that their configured threshold is being ignored instead
of assuming weak-match filtering is silently active.
bace969 to
636f35c
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. Bravo. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Summary
Adds an optional
RAG_DISTANCE_THRESHOLDenv var. When set, results returned by/queryand/query_multipleare filtered by their vector distance after the top-ksearch — any match with distance greater than the threshold is dropped. The score returned bysimilarity_search_with_score_by_vectoris a distance (lower = more similar), so this prunes the weakly-related tail that top-kalone would still surface._apply_distance_thresholdindocument_routes.py, applied to both single-file and multi-file query paths.Closes #109.
Motivation
From the issue reporter:
The right threshold depends on the embedding model and distance strategy, so the README calls out that operators should inspect their actual scores before choosing one rather than baking a default.
Test plan
RAG_DISTANCE_THRESHOLD=0.5; issue a query and confirm only chunks with distance ≤ 0.5 are returned.build-and-test) should pass.