Add Mixedbread (mxbai) model support #89
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Added new dependencies: `ck-models`, `hf-hub`, `tokenizers`, `ort`, `once_cell`, `ndarray`, and `num_cpus` to `ck-embed`. - Updated `Cargo.lock` to reflect new versions and dependencies. - Enhanced model resolution logic in `ck-engine` and `ck-embed` to support new models and improve error handling. - Refactored embedding and reranking model selection to utilize a unified registry approach. - Updated CLI help messages to include new model options. This commit lays the groundwork for better model management and integration across the project.
- Introduced `test_mixedbread_index_and_search` to validate indexing and semantic search functionality using the Mixedbread model. - Added `test_switch_model_to_mixedbread` to ensure model switching works correctly and updates the manifest as expected. - Both tests require the Mixedbread models to be downloaded, and are marked as ignored unless the environment variable `CK_MIXEDBREAD_MODELS_READY` is set. - Created temporary files with semantic content for testing purposes.
…documentation - Introduced first-class support for Mixedbread embedding and reranking models, including `mxbai-xsmall` and `mxbai`. - Updated README with usage instructions and model comparisons for Mixedbread. - Enhanced documentation to include specifications, pros, and cons of the Mixedbread model. - Improved FAQ and limitations sections to reflect new model options and requirements.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add Mixedbread (mxbai) model support
Why
Adds first-class support for Mixedbread models to provide users with an alternative embedding model optimized for local inference. Mixedbread models offer better performance characteristics (faster indexing, larger context windows) while maintaining high-quality semantic understanding, making them ideal for code search workloads.
What
mxbai-xsmall(mixedbread-ai/mxbai-embed-xsmall-v1)mxbai(mixedbread-ai/mxbai-rerank-xsmall-v1)--model mxbai-xsmalland--rerank-model mxbaiflagsPerformance Benchmarks
Hint: The Benchmark was made on a small repo. Don't take this for granted.
Takeaways
.ckdirectory cached to avoid reindexing; switching back and forth between models requires either--switch-modelor cleaning.ck, as done here.Testing
test_mixedbread.rs)Documentation
TESTING_MIXEDBREAD.md,LOCAL_TESTING.md)