-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ability to get a list of loaded models and unload a model by request #3378
Comments
https://github.com/mudler/LocalAI/blob/master/core/http/endpoints/localai/backend_monitor.go These endpoints already show which backends are loaded and allow them to be unloaded? |
Thanks for pointing out /backend/shutdown, but it works only when model ends with .bin https://github.com/mudler/LocalAI/blob/master/core/services/backend_monitor.go#L42 About /backend/monitor endpoint - it doesn't show which models are currently loaded, it just shows some metrics and only if model is set in request (ending with ".bin" too).
With "model": "gemma-2-27b-it-Q5_K_S.gguf":
|
Thanks for the updated comment! That sounds like a big bug to me - I'll see if I can investigate this soon. |
Seems that it shows status of a particular requested model https://github.com/mudler/LocalAI/blob/master/core/http/endpoints/localai/backend_monitor.go#L23C34-L23C39 |
No description provided.
The text was updated successfully, but these errors were encountered: