-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
huggingface: use AutoModel instead of SentenceTransformer #1250
Conversation
Switch SentenceTransformer for AutoModel in order to set trust_remote_code needed to use the encode method with embeddings models like jinai-v2 Signed-off-by: Lucas Hänke de Cansino <[email protected]>
the failure looks genuine @l4b4r4b4b4 - is that compatible with sentence formers? does it requires the models to be pulled manually? Maybe it makes sense to have a separate backend for this instead |
We need to make sure that the |
@mudler it was my understanding its compatible with transformers since its a component from HugginsFace's transformer library. Could also be implemented as fallback option in case normal |
ah and no you don't have to download anything manually. Simply set the yml in So don't think a seperate backend is actually needed 🤷♂️ |
I do agree 100% with you here, lets keep to one if it's possible, but it's weird - the test failed complaining that could not find the model, maybe it misses some option then? |
@l4b4r4b4b4 friendly ping, are you looking into the failures, or shall I help here? |
@mudler That would be great! Since in my local installation it successfully compiles and does inference with jinai model. However I am not sure why it is not able to do that on build test. Maybe it calls for GPU inference by default? 🤔 |
seems in some cases automodel indeed requires quite few additional steps here: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2#usage-huggingface-transformers
The tests-linux pipeline runs only on CPU, no GPU is involved here |
Closing as #1308 was merged,thanks @l4b4r4b4b4 ! |
Switch SentenceTransformer for AutoModel in order to set trust_remote_code needed to use the encode method with embeddings models like jinai-v2
Description
This PR fixes #
Notes for Reviewers
Signed commits