Repository navigation
📐 feat: add EMBEDDINGS_DIMENSIONS env for text-embedding-3 models - #287
Conversation
Passes the dimensions parameter through to OpenAI / Azure OpenAI embeddings clients when set, letting users shrink text-embedding-3-* vectors below the model's native size (1536 / 3072). Lower dimensionality reduces pgvector storage and similarity-search cost at some retrieval-quality trade-off. Only the OpenAI and Azure providers honor the parameter; other providers ignore it. Closes #67
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8627a15250
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| _embeddings_dimensions_raw = get_env_variable("EMBEDDINGS_DIMENSIONS", None) | ||
| EMBEDDINGS_DIMENSIONS = ( | ||
| int(_embeddings_dimensions_raw) | ||
| if _embeddings_dimensions_raw not in (None, "") |
There was a problem hiding this comment.
EMBEDDINGS_DIMENSIONS is converted to int unconditionally during module import, before provider branching, so deployments using non-OpenAI/Azure providers can still fail at startup if this env var is present but non-numeric (for example, a leftover value from another environment). That contradicts the intended “silently ignore for other providers” behavior and turns an unrelated optional setting into a hard boot failure.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in e95f0de. Moved the int() cast inside a provider gate so it only runs for OpenAI / Azure; other providers short-circuit and leave EMBEDDINGS_DIMENSIONS=None even if the env var is set to a stale or non-numeric value.
Parsing the env var unconditionally at import turned an unrelated, stale value (e.g. EMBEDDINGS_DIMENSIONS=foo left over from a prior OpenAI deployment) into a hard boot failure under bedrock / hf / ollama / vertexai even though those providers ignore the value. Gate the int cast on a provider that actually honors it.
|
@codex review |
|
Codex Review: Didn't find any major issues. Breezy! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
danny-avila
changed the title
Same class of bug codex caught on #287's EMBEDDINGS_DIMENSIONS: the float() cast ran unconditionally at import, so a stale or templated non-numeric value would prevent startup for atlas-mongo deployments where the setting is already documented as ignored. Gate the parse on VECTOR_DB_TYPE=pgvector so Atlas boots cleanly regardless of the env var value.
Same class of bug codex caught on #287's EMBEDDINGS_DIMENSIONS: the float() cast ran unconditionally at import, so a stale or templated non-numeric value would prevent startup for atlas-mongo deployments where the setting is already documented as ignored. Gate the parse on VECTOR_DB_TYPE=pgvector so Atlas boots cleanly regardless of the env var value.
* ✨ feat: add RAG_DISTANCE_THRESHOLD to drop weak retrieval matches Applies a post-search filter on /query and /query_multiple that discards any (document, score) pairs whose vector distance exceeds the configured threshold. The score returned by similarity_search_with_score_by_vector is a distance (lower = more similar), so this prunes the less-relevant tail that top-k search would otherwise surface. Unset by default — behavior unchanged for existing deployments. Closes #109 * 🐛 fix: scope RAG_DISTANCE_THRESHOLD to pgvector only Atlas Vector Search returns similarity scores (higher = better), the opposite of pgvector's distances (lower = better), so the naive `score <= threshold` comparison would keep the weakest matches and drop the strongest under VECTOR_DB_TYPE=atlas-mongo. Skip the filter on Atlas and emit a one-time startup warning so operators notice their setting is being ignored. A dedicated Atlas-aware min-similarity knob can be added if there is demand. * 🐛 fix: only parse RAG_DISTANCE_THRESHOLD under pgvector Same class of bug codex caught on #287's EMBEDDINGS_DIMENSIONS: the float() cast ran unconditionally at import, so a stale or templated non-numeric value would prevent startup for atlas-mongo deployments where the setting is already documented as ignored. Gate the parse on VECTOR_DB_TYPE=pgvector so Atlas boots cleanly regardless of the env var value. * 🐛 fix: use raw env for Atlas RAG_DISTANCE_THRESHOLD warning The parsed value is deliberately None under atlas-mongo (we skip the float() cast so a stale non-numeric value can't break startup), so checking it for the warning branch guaranteed the warning never fires. Inspect os.getenv("RAG_DISTANCE_THRESHOLD") directly so Atlas operators actually see that their configured threshold is being ignored instead of assuming weak-match filtering is silently active.
1 participant
Summary
Adds a new
EMBEDDINGS_DIMENSIONSenvironment variable that lets users request a reduced output vector size fromtext-embedding-3-small/text-embedding-3-large. OpenAI and Azure both support thedimensionsparameter on these models; lower dimensionality reduces pgvector storage and makes similarity search faster, at some retrieval-quality trade-off.init_embeddingsfor theopenaiandazureproviders only (other providers don't accept the parameter and silently ignore it).Closes #67.
Caveat documented in README
Cannot be changed on an existing collection — all vectors in a
pgvectorcolumn must share the same dimensionality. Changing it requires dropping and re-embedding the collection.Test plan
EMBEDDINGS_PROVIDER=openai,EMBEDDINGS_MODEL=text-embedding-3-small,EMBEDDINGS_DIMENSIONS=512on a fresh collection; verify stored vectors have 512 dimensions and retrieval works.EMBEDDINGS_DIMENSIONS; verify behavior matches pre-change (native dimensionality).build-and-test) should pass.