update reference for go-llama.cpp

[bavarialogy]

Hi there,

thanks for this awesome bot! To be able to run more recent models, I'd need to update the deps. Can you either tell me how to update that in my local dir or maybe even maintain it in the repo?

Thanks!

[thedmdim]

hello! i updated submodule go-llama.cpp so if it's you asked for you can just git pull or just clone the repo again :)

[bavarialogy]

Hi,
thanks for taking such quick action! Actually I was hoping to get a very recent llama.cpp built in so I'd be able to put Mixtral-8x7b as a model. But I also just realized that in order to achieve that, we'll have to wait for go-skynet/go-llama.cpp#314 or rather go-skynet/go-llama.cpp#318 to be resolved first.

[thedmdim]

As I understood there we need 799a1cb13b0b1b560 commit of llama.cpp so I'd recommend you to try:

cd go-llama.cpp/llama.cpp
git checkout 799a1cb13b0b1b560
cd ../..

[bavarialogy]

Tried that yesterday, so far to no avail.. Here's what I had to do:

1. All you mentioned
2. Pull submodules
3. Checkout latest commit of those
4. Update go.mod because go started complaining there is no version string for go-llama in the file (inserted "latest" for dirty test)
5. Remove models folder because it's always copied to the build context (ok, my bad)

I double-checked that I pulled the latest llama.cpp but somehow I still get the error when loading the model. Can't tell where the bit sits that obviously builds an older llama.cpp version..

[thedmdim]

All you tried is right, i'd done same. The only I can suggest is to cd to go-llama-cpp/llama.cpp and look git status. As for today the latest commit of llama.cpp is dc68f00, if git status points to another commit try git checkout dc68f00

[bavarialogy]

Thanks once again - even if I checkout master for both submodules, every time I build it remains the same. I encountered that in the Dockerfile, the submodules are init'ed again, so my changes might even be useless.. But once I try to omit the "git submodule update --init --recursive", the build fails to find go-llama.cpp. Have you got an Idea how I could work around that?