Skip to content

Bug: Server not support mmproj #7988

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lld1995 opened this issue Jun 18, 2024 · 3 comments
Closed

Bug: Server not support mmproj #7988

lld1995 opened this issue Jun 18, 2024 · 3 comments

Comments

@lld1995
Copy link

lld1995 commented Jun 18, 2024

What happened?

I use Yi-VL-6B,and i run it with ./server -m /data/model_data/Yi-VL-6B-GGUF/ggml-model-f16.gguf --n-gpu-layers 4096 -n 384 -c 2048 --host 0.0.0.0 --port 8897 --mmproj /data/model_data/Yi-VL-6B-GGUF/mmproj-model-f16.gguf

But when i upload image on web, it shows
image

Name and Version

/data/llama.cpp/build/bin$ ./server --version
version: 3137 (dcf7527)
built with cc (Ubuntu 10.5.0-1ubuntu1~20.04) 10.5.0 for x86_64-linux-gnu

What operating system are you seeing the problem on?

No response

Relevant log output

No response

@lld1995 lld1995 added bug-unconfirmed high severity Used to report high severity bugs in llama.cpp (Malfunctioning hinder important workflow) labels Jun 18, 2024
@ngxson
Copy link
Collaborator

ngxson commented Jun 18, 2024

For now, server does not support vision model

Edit: related to #8010

@ngxson ngxson removed the high severity Used to report high severity bugs in llama.cpp (Malfunctioning hinder important workflow) label Jun 18, 2024
@lld1995
Copy link
Author

lld1995 commented Jun 18, 2024

https://github.com/trzy/llava-cpp-server

The project is reliable? It has some unpredictable issue?

@ngxson
Copy link
Collaborator

ngxson commented Jun 19, 2024

@lld1995 The mentioned project is not maintained by llama.cpp, so I cannot answer your question.

@lld1995 lld1995 closed this as completed Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants