make BUILD_TYPE=cublas build FAILING on Linux

[Free-Radical]

LocalAI version:

1.18.0

Environment, CPU architecture, OS, and Version:

Linux 6.2.0-20-generic #20-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr 6 07:48:48 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug

make for local Linux machine (vs Dockers) fails

To Reproduce

$ make BUILD_TYPE=cublas build

Expected behavior

localAI server running

Logs

$ make BUILD_TYPE=cublas build --debug
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Updating makefiles....
Updating goal targets....
File 'build' does not exist.
File 'prepare' does not exist.
File 'prepare-sources' does not exist.
File 'replace' does not exist.
Must remake target 'replace'.
go mod edit -replace github.com/go-skynet/go-llama.cpp=/home/khan/Documents/Github/LocalAI/go-llama
make: go: No such file or directory
make: *** [Makefile:175: replace] Error 127

Additional context

PS docker image runs fine , trying to run locally for better performance , especially with GPU support
PPS GREAT PROJECT, THX!!!

[bailboy91]

I am experiencing a similar issue I believe. Be sure in the MakeFile you have the correct path to cuda. the default is incorrect for ubuntu using the nvidia-cuda-toolkit.

localai default: CUDA_LIBPATH?=/usr/local/cuda/lib64/
Ubuntu default: CUDA_LIBPATH?=/usr/lib/cuda/lib64/

After changing to the default ubuntu location i get this as a warning and then errors about long long int.
go-bert/bert.cpp/bert.cpp:610:89: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 5 has type ‘int64_t’ {aka ‘long int’} [-Wformat=]

[d-a-v-e-g]

@bailboy91 Any chance the errors below your warning look like this:

github.com/go-skynet/go-llama.cpp

/usr/bin/ld: go-llama/libbinding.a(ggml-cuda.o): in function convert_fp16_to_fp32_cuda(void const*, float*, int, CUstream_st*)': tmpxft_00002630_00000000-6_ggml-cuda.cudafe1.cpp:(.text+0x9a): undefined reference to __cudaPushCallConfiguration'
...

[bailboy91]

@napolitanica

In this instance i wish i did have the same error as that. I have repeating lines like this:

go-bert/bert.cpp/bert.cpp:610:89: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 5 has type ‘int64_t’ {aka ‘long int’} [-Wformat=]
  610 |                 fprintf(stderr, "%s: tensor '%s' has wrong shape in model file: got [%lld, %lld], expected [%lld, %lld]\n",
      |                                                                                      ~~~^
      |                                                                                         |
      |                                                                                         long long int
      |                                                                                      %ld
  611 |                         __func__, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]);
      |                                                ~~~~~~~~~~~~~                             
      |                                                            |
      |                                                            int64_t {aka long int}
go-bert/bert.cpp/bert.cpp:610:95: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 6 has type ‘int64_t’ {aka ‘long int’} [-Wformat=]
  610 |                 fprintf(stderr, "%s: tensor '%s' has wrong shape in model file: got [%lld, %lld], expected [%lld, %lld]\n",
      |                                                                                            ~~~^
      |                                                                                               |
      |                                                                                               long long int
      |                                                                                            %ld

[bailboy91]

Well, Now i feel silly.

A fresh git clone and now i get a different error..

 File 'libgpt4all.a' does not exist.
Must remake target 'libgpt4all.a'.
ar src libgpt4all.a llmodel.o binding.o

[Free-Radical]

Is this going to be addressed/fixed???

[mudler]

LocalAI version:

1.18.0

Environment, CPU architecture, OS, and Version:

Linux 6.2.0-20-generic #20-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr 6 07:48:48 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug

make for local Linux machine (vs Dockers) fails

To Reproduce

$ make BUILD_TYPE=cublas build

Expected behavior

localAI server running

Logs

$ make BUILD_TYPE=cublas build --debug GNU Make 4.3 Built for x86_64-pc-linux-gnu Copyright (C) 1988-2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Reading makefiles... Updating makefiles.... Updating goal targets.... File 'build' does not exist. File 'prepare' does not exist. File 'prepare-sources' does not exist. File 'replace' does not exist. Must remake target 'replace'. go mod edit -replace github.com/go-skynet/go-llama.cpp=/home/khan/Documents/Github/LocalAI/go-llama make: go: No such file or directory make: *** [Makefile:175: replace] Error 127

Additional context

PS docker image runs fine , trying to run locally for better performance , especially with GPU support PPS GREAT PROJECT, THX!!!

You seem to be missing go installed locally.

[Free-Radical]

ok THX, installed GO per web site's instructions. Also ensured conda , python etc are installed correctly, (paths etc) now new errors, full text file attached

$ make clean
$ make BUILD_TYPE=cublas build

errors in file start with:

# github.com/go-skynet/go-bert.cpp
In file included from gobert.cpp:6:
go-bert/bert.cpp/bert.cpp: In function ‘bert_ctx* bert_load_from_file(const char*)’:
go-bert/bert.cpp/bert.cpp:610:89: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 5 has type ‘int64_t’ {aka ‘long int’} [-Wformat=]
  610 |                 fprintf(stderr, "%s: tensor '%s' has wrong shape in model file: got [%lld, %lld], expected [%lld, %lld]\n",
      |                                                           
[tmp.txt](https://github.com/go-skynet/LocalAI/files/11752968/tmp.txt)
                           ~~~^

[mudler]

ok THX, installed GO per web site's instructions. Also ensured conda , python etc are installed correctly, (paths etc) now new errors, full text file attached

$ make clean
$ make BUILD_TYPE=cublas build

errors in file start with:

# github.com/go-skynet/go-bert.cpp
In file included from gobert.cpp:6:
go-bert/bert.cpp/bert.cpp: In function ‘bert_ctx* bert_load_from_file(const char*)’:
go-bert/bert.cpp/bert.cpp:610:89: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 5 has type ‘int64_t’ {aka ‘long int’} [-Wformat=]
  610 |                 fprintf(stderr, "%s: tensor '%s' has wrong shape in model file: got [%lld, %lld], expected [%lld, %lld]\n",
      |                                                           
[tmp.txt](https://github.com/go-skynet/LocalAI/files/11752968/tmp.txt)
                           ~~~^

that is a warning. There are no errors in the log you provided ?