Mistral 3 large #17699
-
|
Hello, Mistral opensourced Mistral 3 large (among other models). Its size is quite a challenge, as i guess it will even in Q4 need at least 4-8 H100s (on the other hand 1-3 B300 should do?) but it's currently a very unique model. Any Chance for llama.cpp support? I mean, Nvidia has already forseen a row for llama.cpp in the compatibility table https://developer.nvidia.com/blog/nvidia-accelerated-mistral-3-open-models-deliver-efficiency-accuracy-at-any-scale/ 😉 Thanks a lot Cheers |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
It seems like it's essentially a DeepSeekV3 architecture, with different tensor names and vision support. I don't think it would necessarily be hard to add. vLLM implementation: vllm-project/vllm#29757 |
Beta Was this translation helpful? Give feedback.
It seems like it's essentially a DeepSeekV3 architecture, with different tensor names and vision support. I don't think it would necessarily be hard to add.
vLLM implementation: vllm-project/vllm#29757