Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Embeddings with langchain fail: "rpc error: code = Unavailable desc = error reading from server: EOF" #2120

Open
Connor2573 opened this issue Apr 24, 2024 · 4 comments
Open

Embeddings with langchain fail: "rpc error: code = Unavailable desc = error reading from server: EOF" #2120

Connor2573 opened this issue Apr 24, 2024 · 4 comments
Labels
bug Something isn't working unconfirmed

Comments

@Connor2573
Copy link

Thanks so much for your help and this amazing software!

LocalAI version:
d65214a commit on 4/24/2024

Environment, CPU architecture, OS, and Version:
Linux server 5.15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug
Using the chroma.from_documents() function with either the LocalAIEmbeddings or OpenAIEmbeddings from langchain as the embedding function causes:
ERR Server error error="rpc error: code = Unavailable desc = error reading from server: EOF" ip=192.168.X.XXX latency=2.232229359s method=POST status=500 url=/embeddings

To Reproduce
Run local-ai compiled with CUBLAS

Expected behavior
I expected this to create a vector database and properly return the embeddings.

Logs
embedding_log.txt

Additional context
I have only managed to get the bert-embeddings backend to work properly, making me think I might have some sort of installation issue. The sentence-transformers backend produces incorrect embeddings for me and I get an error mentioned in another issue when I try to run a model in gguf format.
Additionally, sending a curl request or a basic request through OpenAI's python package to the server works perfectly fine. It is only when using specifically the from_documents method that it fails. I feel it has to have something to do with the two warning messages in the provided logs.
@Connor2573 Connor2573 added bug Something isn't working unconfirmed labels Apr 24, 2024
@localai-bot
Copy link
Contributor

Hello Connor2573,

Thank you for reporting the issue and providing such detailed information. We appreciate your support for the LocalAI project.

Based on the details you provided, here's a possible interpretation of the problem:

  • It seems that the server generates an "EOF" (End Of File) error while reading data, hence the "rpc error: code = Unavailable desc = error reading from server: EOF" message. The error might be originating from a specific part of the server-side code when handling the embedding requests from the from_documents() function in langchain.
  • The fact that other functionalities of LocalAI are working fine, as confirmed by the successful curl requests and OpenAI python package interactions, suggests that this might be a targeted issue with the mentioned function.
  • Given that you've mentioned only the bert-embeddings backend works correctly, it could indicate a problem with the implementations for other backends (sentence-transformers, OpenAIEmbeddings, LocalAIEmbeddings). However, this is just an assumption, and a thorough inspection of the code is necessary to confirm this hypothesis.

The next steps could be:

  1. Upgrade LocalAI to the latest version, in case there's a patch or update that addresses this issue.
  2. Check the server logs for any more detailed messages that could provide further clues on the root cause of the "EOF" error. Make sure to monitor the server status as well for any abnormal activity.
  3. If possible, isolate the specific part of your code that's responsible for handling embedding requests, and go through it line by line to identify potential problems, especially focusing on the areas where the two warning messages mentioned in the logs occur.
  4. If the issue persists, consider opening a pull request to the LocalAI repository with a potential fix or a more detailed analysis of the issue. This could be a great opportunity to contribute to the project and help the community!

Please ensure to use the issue tracker on Github to report any further findings or additional bugs. Let's collaborate to solve this issue and make LocalAI better! Don't hesitate to reach out if you need further help.

I'm a bot, an experiment of @mudler.

Best regards,
The LocalAI Team

(AI-generated response based on the given scenario, not a direct input from a team member of LocalAI)

@Connor2573
Copy link
Author

Managed to get this working but reducing the length of the documents I was feeding to it

@Connor2573
Copy link
Author

The issue seems to now be that the embeddings themselves are all basically the same. Even though when I run the same code using the same model directly it works out fine. I have done this texting with the aio docker image.

@rabner
Copy link

rabner commented Sep 20, 2024

Similar issue on AIO CPU, but when invoking image generation from Nextcloud:

localai | 7:26PM ERR Server error error="rpc error: code = Unavailable desc = error reading from server: EOF" ip=192.168.0.110 latency=3.358938752s method=POST status=500 url=/v1/images/generations

Log 

localai     | 7:26PM DBG Request received: {"model":"","language":"","translate":false,"n":0,"top_p":null,"top_k":null,"temperature":null,"max_tokens":null,"echo":false,"batch":0,"ignore_eos":false,"repeat_penalty":0,"repeat_last_n":0,"n_keep":0,"frequency_penalty":0,"presence_penalty":0,"tfz":null,"typical_p":null,"seed":null,"negative_prompt":"","rope_freq_base":0,"rope_freq_scale":0,"negative_prompt_scale":0,"use_fast_tokenizer":false,"clip_skip":0,"tokenizer":"","file":"","size":"1024x1024","prompt":"cute cat sitting on kitchen table","instruction":"","input":null,"stop":null,"messages":null,"functions":null,"function_call":null,"stream":false,"mode":0,"step":0,"grammar":"","grammar_json_functions":null,"backend":"","model_base_name":""}
localai     | 7:26PM DBG Loading model: stablediffusion
localai     | 7:26PM DBG guessDefaultsFromFile: not a GGUF file
localai     | 7:26PM DBG Parameter Config: &{PredictionOptions:{Model:stablediffusion_assets Language: Translate:false N:0 TopP:0xc000af9c40 TopK:0xc000af9c48 Temperature:0xc000af9c50 Maxtokens:0xc000af9c80 Echo:false Batch:0 IgnoreEOS:false RepeatPenalty:0 RepeatLastN:0 Keep:0 FrequencyPenalty:0 PresencePenalty:0 TFZ:0xc000af9c78 TypicalP:0xc000af9c70 Seed:0xc000af9c98 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:stablediffusion F16:0xc000af9c38 Threads:0xc000af9c30 Debug:0xc0005b6430 Roles:map[] Embeddings:0xc000af9c91 Backend:stablediffusion TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions: UseTokenizerTemplate:false JoinChatMessagesByCharacter:<nil>} PromptStrings:[cute cat sitting on kitchen table] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: ResponseFormat: ResponseFormatMap:map[] FunctionsConfig:{DisableNoAction:false GrammarConfig:{ParallelCalls:false DisableParallelNewLines:false MixedMode:false NoMixedFreeString:false NoGrammar:false Prefix: ExpectStringsAfterJSON:false PropOrder: SchemaType:} NoActionFunctionName: NoActionDescriptionName: ResponseRegex:[] JSONRegexMatch:[] ReplaceFunctionResults:[] ReplaceLLMResult:[] CaptureLLMResult:[] FunctionNameKey: FunctionArgumentsKey:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0xc000af9c68 MirostatTAU:0xc000af9c60 Mirostat:0xc000af9c58 NGPULayers:0xc000af9c88 MMap:0xc000af9c90 MMlock:0xc000af9c91 LowVRAM:0xc000af9c91 Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] TrimSuffix:[] ContextSize:0xc000af9c28 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: FlashAttention:false NoKVOffloading:false RopeScaling: ModelType: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{CUDA:false PipelineType: SchedulerType: EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder: ControlNet:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} TTSConfig:{Voice: VallE:{AudioPath:}} CUDA:false DownloadFiles:[{Filename:stablediffusion_assets/AutoencoderKL-256-256-fp16-opt.param SHA256:18ca4b66685e21406bcf64c484b3b680b4949900415536d599cc876579c85c82 URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/AutoencoderKL-256-256-fp16-opt.param} {Filename:stablediffusion_assets/AutoencoderKL-512-512-fp16-opt.param SHA256:cf45f63aacf3dbbab0f59ed92a6f2c14d9a1801314631cd3abe91e3c85639a20 URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/AutoencoderKL-512-512-fp16-opt.param} {Filename:stablediffusion_assets/AutoencoderKL-base-fp16.param SHA256:0254a056dce61b0c27dc9ec1b78b53bcf55315c540f55f051eb841aa992701ba URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/AutoencoderKL-base-fp16.param} {Filename:stablediffusion_assets/AutoencoderKL-encoder-512-512-fp16.bin SHA256:ddcb79a9951b9f91e05e087739ed69da2c1c4ae30ba4168cce350b49d617c9fa URI:https://github.com/EdVince/Stable-Diffusion-NCNN/releases/download/naifu/AutoencoderKL-encoder-512-512-fp16.bin} {Filename:stablediffusion_assets/AutoencoderKL-fp16.bin SHA256:f02e71f80e70252734724bbfaed5c4ddd3a8ed7e61bb2175ff5f53099f0e35dd URI:https://github.com/EdVince/Stable-Diffusion-NCNN/releases/download/naifu/AutoencoderKL-fp16.bin} {Filename:stablediffusion_assets/FrozenCLIPEmbedder-fp16.bin SHA256:1c9a12f4e1dd1b295a388045f7f28a2352a4d70c3dc96a542189a3dd7051fdd6 URI:https://github.com/EdVince/Stable-Diffusion-NCNN/releases/download/naifu/FrozenCLIPEmbedder-fp16.bin} {Filename:stablediffusion_assets/FrozenCLIPEmbedder-fp16.param SHA256:471afbe678dd1fd3fe764ef9c6eccaccb0a7d7e601f27b462aa926b20eb368c9 URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/FrozenCLIPEmbedder-fp16.param} {Filename:stablediffusion_assets/log_sigmas.bin SHA256:a2089f8aa4c61f9c200feaec541ab3f5c94233b28deb6d5e8bcd974fa79b68ac URI:https://github.com/EdVince/Stable-Diffusion-NCNN/raw/main/x86/linux/assets/log_sigmas.bin} {Filename:stablediffusion_assets/UNetModel-256-256-MHA-fp16-opt.param SHA256:a58c380229f09491776df837b7aa7adffc0a87821dc4708b34535da2e36e3da1 URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/UNetModel-256-256-MHA-fp16-opt.param} {Filename:stablediffusion_assets/UNetModel-512-512-MHA-fp16-opt.param SHA256:f12034067062827bd7f43d1d21888d1f03905401acf6c6eea22be23c259636fa URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/UNetModel-512-512-MHA-fp16-opt.param} {Filename:stablediffusion_assets/UNetModel-base-MHA-fp16.param SHA256:696f6975de49f4325b53ce32aff81861a6d6c07cd9ce3f0aae2cc405350af38d URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/UNetModel-base-MHA-fp16.param} {Filename:stablediffusion_assets/UNetModel-MHA-fp16.bin SHA256:d618918d011bfc1f644c0f2a33bf84931bd53b28a98492b0a8ed6f3a818852c3 URI:https://github.com/EdVince/Stable-Diffusion-NCNN/releases/download/naifu/UNetModel-MHA-fp16.bin} {Filename:stablediffusion_assets/vocab.txt SHA256:e30e57b6f1e47616982ef898d8922be24e535b4fa3d0110477b3a6f02ebbae7d URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/vocab.txt}] Description:Stable Diffusion in NCNN with c++, supported txt2img and img2img
localai     |  Usage:curl http://localhost:8080/v1/images/generations \
localai     |   -H "Content-Type: application/json" \
localai     |   -d '{
localai     |     "prompt": "<positive prompt>|<negative prompt>",
localai     |     "step": 25,
localai     |     "size": "512x512"
localai     |   }'}
localai     | 7:26PM INF Loading model 'stablediffusion_assets' with backend stablediffusion
localai     | 7:26PM DBG Model already loaded in memory: stablediffusion_assets
localai     | 7:26PM DBG Checking model availability (stablediffusion_assets)
localai     | 7:26PM WRN GRPC Model not responding: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:45259: connect: connection refused"
localai     | 7:26PM WRN Deleting the process in order to recreate it
localai     | 7:26PM DBG GRPC Process is not responding: stablediffusion_assets
localai     | 7:26PM ERR (deleteProcess) error while deleting grpc process stablediffusion_assets error="process does not exist"
localai     | 7:26PM DBG Loading model in memory from file: /build/models/stablediffusion_assets
localai     | 7:26PM DBG Loading Model stablediffusion_assets with gRPC (file: /build/models/stablediffusion_assets) (backend: stablediffusion): {backendString:stablediffusion model:stablediffusion_assets threads:4 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc00024c488 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh openvoice:/build/backend/python/openvoice/run.sh parler-tts:/build/backend/python/parler-tts/run.sh rerankers:/build/backend/python/rerankers/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false parallelRequests:false}
localai     | 7:26PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/stablediffusion
localai     | 7:26PM DBG GRPC Service for stablediffusion_assets will be running at: '127.0.0.1:42087'
localai     | 7:26PM DBG GRPC Service state dir: /tmp/go-processmanager3967400667
localai     | 7:26PM DBG GRPC Service Started
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 2024/09/20 19:26:15 gRPC Server listening at 127.0.0.1:42087
localai     | 7:26PM DBG GRPC Service Ready
localai     | 7:26PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:stablediffusion_assets ContextSize:1024 Seed:532999100 NBatch:512 F16Memory:false MLock:false MMap:true VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:99999999 MainGPU: TensorSplit: Threads:4 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/build/models/stablediffusion_assets Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type: FlashAttention:false NoKVOffload:false}
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stdout ----------------[start generation upscaled image]------------------
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stdout positive_prompt: cute cat sitting on kitchen table
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stdout output_png_path: /tmp/generated/images/b642248722688.png
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stdout negative_prompt: 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stdout step: 15
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stdout seed: -1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stdout ----------------[prompt]------------------
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stdout ----------------[diffusion]---------------
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr fopen /build/models/stablediffusion_assets/UNetModel-fp16.param failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr fopen /build/models/stablediffusion_assets/UNetModel-fp16.bin failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name in0 failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name in1 failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name in2 failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name c_in failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name c_out failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name outout failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name in0 failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name in1 failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name in2 failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name c_in failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name c_out failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr find_blob_index_by_name outout failed
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr Try
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr SIGSEGV: segmentation violation
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr PC=0x8369f0 m=5 sigcode=1 addr=0x0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr signal arrived during cgo execution
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr goroutine 34 gp=0xc0003a8000 m=5 mp=0xc000100008 [syscall]:
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.cgocall(0x81f7e0, 0xc000215748)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc000215720 sp=0xc0002156e8 pc=0x43e9ab
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr github.com/mudler/go-stable-diffusion._Cfunc_generate_image_upscaled(0x400, 0x400, 0xf, 0xffffffff, 0x715f20000b70, 0x38f71f0, 0x715f20000bd0, 0x715f20000ba0)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   _cgo_gotypes.go:106 +0x4b fp=0xc000215748 sp=0xc000215720 pc=0x81e14b
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr github.com/mudler/go-stable-diffusion.GenerateImageUpscaled(0x400, 0x400, 0xf, 0xffffffffffffffff, {0xc000304060?, 0xc0001b8320?}, {0x0, 0x0}, {0xc000304090, 0x27}, ...)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /build/sources/go-stable-diffusion/stablediffusion.go:34 +0xf0 fp=0xc0002157a8 sp=0xc000215748 pc=0x81e610
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr github.com/mudler/LocalAI/pkg/stablediffusion.GenerateImage(0xc0001b8320?, 0xc0001bc3c0?, 0x4508f8?, 0x7160356c0788?, 0x100f660?, {0xc000304060?, 0x80?}, {0x0?, 0xc00019e918?}, {0xc000304090, ...}, ...)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /build/pkg/stablediffusion/generate.go:12 +0x7d fp=0xc000215830 sp=0xc0002157a8 pc=0x81e73d
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr github.com/mudler/LocalAI/pkg/stablediffusion.(*StableDiffusion).GenerateImage(...)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /build/pkg/stablediffusion/stablediffusion.go:19
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr main.(*Image).GenerateImage(0x0?, 0xc00019e938?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /build/backend/go/image/stablediffusion/stablediffusion.go:24 +0x65 fp=0xc0002158a8 sp=0xc000215830 pc=0x81eb65
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr github.com/mudler/LocalAI/pkg/grpc.(*server).GenerateImage(0xc000038df0, {0x1011320?, 0xc00019e990?}, 0xc0001b8320)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /build/pkg/grpc/server.go:71 +0xe7 fp=0xc000215950 sp=0xc0002158a8 pc=0x813bc7
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr github.com/mudler/LocalAI/pkg/grpc/proto._Backend_GenerateImage_Handler({0x1011320, 0xc000038df0}, {0x10e7348, 0xc000183680}, 0xc000306180, 0x0)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /build/pkg/grpc/proto/backend_grpc.pb.go:440 +0x1a6 fp=0xc0002159a0 sp=0xc000215950 pc=0x80fc06
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc.(*Server).processUnaryRPC(0xc000204600, {0x10e7348, 0xc0001835f0}, {0x10ea800, 0xc0002bd800}, 0xc0003a0000, 0xc000250a50, 0x148c960, 0x0)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/server.go:1379 +0xdf8 fp=0xc000215da0 sp=0xc0002159a0 pc=0x7f5558
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc.(*Server).handleStream(0xc000204600, {0x10ea800, 0xc0002bd800}, 0xc0003a0000)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/server.go:1790 +0xe8b fp=0xc000215f78 sp=0xc000215da0 pc=0x7fa42b
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc.(*Server).serveStreams.func2.1()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/server.go:1029 +0x8b fp=0xc000215fe0 sp=0xc000215f78 pc=0x7f360b
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goexit({})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000215fe8 sp=0xc000215fe0 pc=0x4a6ca1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 13
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/server.go:1040 +0x125
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr goroutine 1 gp=0xc0000061c0 m=nil [IO wait]:
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gopark(0xc000048a08?, 0x0?, 0xc0?, 0x61?, 0xc00023bb38?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00023bb00 sp=0xc00023bae0 pc=0x474fee
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.netpollblock(0xc00023bb98?, 0x43e146?, 0x0?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc00023bb38 sp=0xc00023bb00 pc=0x46ddf7
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr internal/poll.runtime_pollWait(0x7160360096d0, 0x72)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc00023bb58 sp=0xc00023bb38 pc=0x4a1b65
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr internal/poll.(*pollDesc).wait(0x3?, 0x1?, 0x0)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00023bb80 sp=0xc00023bb58 pc=0x50c367
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr internal/poll.(*pollDesc).waitRead(...)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr internal/poll.(*FD).Accept(0xc000200400)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc00023bc28 sp=0xc00023bb80 pc=0x51170c
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr net.(*netFD).accept(0xc000200400)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc00023bce0 sp=0xc00023bc28 pc=0x62f889
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr net.(*TCPListener).accept(0xc0000a0400)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc00023bd08 sp=0xc00023bce0 pc=0x646a7e
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr net.(*TCPListener).Accept(0xc0000a0400)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/net/tcpsock.go:327 +0x30 fp=0xc00023bd38 sp=0xc00023bd08 pc=0x645c70
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc.(*Server).Serve(0xc000204600, {0x10e6a18, 0xc0000a0400})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/server.go:885 +0x49e fp=0xc00023be98 sp=0xc00023bd38 pc=0x7f20be
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr github.com/mudler/LocalAI/pkg/grpc.StartServer({0x7ffc79849981?, 0xc000024100?}, {0x10ecde0, 0xc000038d30})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /build/pkg/grpc/server.go:238 +0x170 fp=0xc00023bf20 sp=0xc00023be98 pc=0x816070
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr main.main()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /build/backend/go/image/stablediffusion/main.go:18 +0x85 fp=0xc00023bf50 sp=0xc00023bf20 pc=0x81e9e5
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.main()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:271 +0x29d fp=0xc00023bfe0 sp=0xc00023bf50 pc=0x474bbd
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goexit({})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00023bfe8 sp=0xc00023bfe0 pc=0x4a6ca1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr goroutine 2 gp=0xc000006c40 m=nil [force gc (idle)]:
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000084fa8 sp=0xc000084f88 pc=0x474fee
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goparkunlock(...)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:408
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.forcegchelper()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:326 +0xb3 fp=0xc000084fe0 sp=0xc000084fa8 pc=0x474e73
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goexit({})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000084fe8 sp=0xc000084fe0 pc=0x4a6ca1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr created by runtime.init.6 in goroutine 1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:314 +0x1a
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr goroutine 3 gp=0xc000007180 m=nil [GC sweep wait]:
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000085780 sp=0xc000085760 pc=0x474fee
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goparkunlock(...)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:408
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.bgsweep(0xc0000aa000)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/mgcsweep.go:278 +0x94 fp=0xc0000857c8 sp=0xc000085780 pc=0x4605f4
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gcenable.gowrap1()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/mgc.go:203 +0x25 fp=0xc0000857e0 sp=0xc0000857c8 pc=0x454f45
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goexit({})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000857e8 sp=0xc0000857e0 pc=0x4a6ca1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr created by runtime.gcenable in goroutine 1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/mgc.go:203 +0x66
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr goroutine 4 gp=0xc000007340 m=nil [GC scavenge wait]:
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gopark(0xc0000aa000?, 0x10df9d8?, 0x1?, 0x0?, 0xc000007340?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000085f78 sp=0xc000085f58 pc=0x474fee
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goparkunlock(...)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:408
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.(*scavengerState).park(0x14d46c0)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000085fa8 sp=0xc000085f78 pc=0x45dfe9
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.bgscavenge(0xc0000aa000)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/mgcscavenge.go:653 +0x3c fp=0xc000085fc8 sp=0xc000085fa8 pc=0x45e57c
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gcenable.gowrap2()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc000085fe0 sp=0xc000085fc8 pc=0x454ee5
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goexit({})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x4a6ca1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr created by runtime.gcenable in goroutine 1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/mgc.go:204 +0xa5
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr goroutine 5 gp=0xc000007c00 m=nil [finalizer wait]:
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gopark(0xc000084648?, 0x448625?, 0xa8?, 0x1?, 0x102bbe0?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000084620 sp=0xc000084600 pc=0x474fee
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.runfinq()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/mfinal.go:194 +0x107 fp=0xc0000847e0 sp=0xc000084620 pc=0x453f87
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goexit({})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000847e8 sp=0xc0000847e0 pc=0x4a6ca1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr created by runtime.createfing in goroutine 1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/mfinal.go:164 +0x3d
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr goroutine 13 gp=0xc0000f48c0 m=nil [IO wait]:
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gopark(0xc0001ca270?, 0xc0000f48c0?, 0x8?, 0xa?, 0xb?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000095a08 sp=0xc0000959e8 pc=0x474fee
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.netpollblock(0x4f23d8?, 0x43e146?, 0x0?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc000095a40 sp=0xc000095a08 pc=0x46ddf7
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr internal/poll.runtime_pollWait(0x7160360095d8, 0x72)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000095a60 sp=0xc000095a40 pc=0x4a1b65
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr internal/poll.(*pollDesc).wait(0xc000306100?, 0xc0002c6000?, 0x0)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000095a88 sp=0xc000095a60 pc=0x50c367
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr internal/poll.(*pollDesc).waitRead(...)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr internal/poll.(*FD).Read(0xc000306100, {0xc0002c6000, 0x8000, 0x8000})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc000095b20 sp=0xc000095a88 pc=0x50d65a
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr net.(*netFD).Read(0xc000306100, {0xc0002c6000?, 0x1060100000000?, 0x8?})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/net/fd_posix.gohp\",\:55 +0x25 fp=0xc000095b68 sp=0xc000095b20 pc=0x62d8a5
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr net.(*conn).Read(0xc000196040, {0xc0002c6000?, 0x800010601?, 0xc000000000?})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/net/net.go:185 +0x45 fp=0xc000095bb0 sp=0xc000095b68 pc=0x63dc85
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr net.(*TCPConn).Read(0x0?, {0xc0002c6000?, 0xc000095c08?, 0x4949cd?})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   <autogenerated>:1 +0x25 fp=0xc000095be0 sp=0xc000095bb0 pc=0x64fba5
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr bufio.(*Reader).Read(0xc0002c4060, {0xc0000f0740, 0x9, 0x14d4ee0?})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/bufio/bufio.go:241 +0x197 fp=0xc000095c18 sp=0xc000095be0 pc=0x5a5f57
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr io.ReadAtLeast({0x10e4860, 0xc0002c4060}, {0xc0000f0740, 0x9, 0x9}, 0x9)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/io/io.go:335 +0x90 fp=0xc000095c60 sp=0xc000095c18 pc=0x4ec330
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr io.ReadFull(...)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/io/io.go:354
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr golang.org/x/net/http2.readFrameHeader({0xc0000f0740, 0x9, 0xc000122018?}, {0x10e4860?, 0xc0002c4060?})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:237 +0x65 fp=0xc000095cb0 sp=0xc000095c60 pc=0x75e745
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr golang.org/x/net/http2.(*Framer).ReadFrame(0xc0000f0700)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/golang.org/x/[email protected]/http2/frame.go:501 +0x85 fp=0xc000095d58 sp=0xc000095cb0 pc=0x75ee85
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc0002bd800, {0x10e7348, 0xc000251dd0}, 0xc000251e00)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:644 +0x15e fp=0xc000095e88 sp=0xc000095d58 pc=0x787dfe
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc.(*Server).serveStreams(0xc000204600, {0x10e72a0?, 0x15352a0?}, {0x10ea800, 0xc0002bd800}, {0x10ea2f8?, 0xc000196040?})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/server.go:1023 +0x3b6 fp=0xc000095f70 sp=0xc000095e88 pc=0x7f3396
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc.(*Server).handleRawConn.func1()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/server.go:959 +0x56 fp=0xc000095fe0 sp=0xc000095f70 pc=0x7f2b36
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goexit({})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000095fe8 sp=0xc000095fe0 pc=0x4a6ca1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr created by google.golang.org/grpc.(*Server).handleRawConn in goroutine 23
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/server.go:958 +0x1c6
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr goroutine 11 gp=0xc0000f4a80 m=nil [select]:
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gopark(0xc000389ea0?, 0x2?, 0x1e?, 0x0?, 0xc000389e74?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000389d20 sp=0xc000389d00 pc=0x474fee
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.selectgo(0xc000389ea0, 0xc000389e70, 0x78ea56?, 0x0, 0xc0002ce000?, 0x1)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000389e40 sp=0xc000389d20 pc=0x486365
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0000b8b90, 0x1)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/internal/transport/controlbuf.go:418 +0x113 fp=0xc000389ed0 sp=0xc000389e40 pc=0x76df73
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000380000)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/internal/transport/controlbuf.go:552 +0x86 fp=0xc000389f30 sp=0xc000389ed0 pc=0x76e6e6
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc/internal/transport.NewServerTransport.func2()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:334 +0xc7 fp=0xc000389fe0 sp=0xc000389f30 pc=0x784ee7
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goexit({})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000389fe8 sp=0xc000389fe0 pc=0x4a6ca1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport in goroutine 23
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:332 +0x193e
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr goroutine 12 gp=0xc0000f4e00 m=nil [select]:
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.gopark(0xc000087740?, 0x4?, 0xf0?, 0x75?, 0xc0000876c0?)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000087558 sp=0xc000087538 pc=0x474fee
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.selectgo(0xc000087740, 0xc0000876b8, 0xc000002180?, 0x0, 0xc000250cc0?, 0x1)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000087678 sp=0xc000087558 pc=0x486365
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc0002bd800)
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:1168 +0x205 fp=0xc0000877c8 sp=0xc000087678 pc=0x78c0e5
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr google.golang.org/grpc/internal/transport.NewServerTransport.gowrap1()
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:355 +0x25 fp=0xc0000877e0 sp=0xc0000877c8 pc=0x784de5
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr runtime.goexit({})
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000877e8 sp=0xc0000877e0 pc=0x4a6ca1
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr created by google.golang.org/grpc/internal/transport.NewServerTransport in goroutine 23
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr   /root/go/pkg/mod/google.golang.org/[email protected]/internal/transport/http2_server.go:355 +0x1985
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr 
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rax    0x0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rbx    0x715f391ef040
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rcx    0x0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rdx    0x0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rdi    0x0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rsi    0x4
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rbp    0x715f391ede10
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rsp    0x715f391eda00
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr r8     0x0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr r9     0x0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr r10    0x0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr r11    0x829bc0d32c6ce50
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr r12    0x715f391ef088
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr r13    0x715f391edb00
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr r14    0x715f391eda60
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr r15    0x715f391edbf0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rip    0x8369f0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr rflags 0x10246
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr cs     0x33
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr fs     0x0
localai     | 7:26PM DBG GRPC(stablediffusion_assets-127.0.0.1:42087): stderr gs     0x0
localai     | 7:26PM ERR Server error error="rpc error: code = Unavailable desc = error reading from server: EOF" ip=192.168.0.110 latency=3.358938752s method=POST status=500 url=/v1/images/generations

When I do it from LocalAI UI logs shows that image is generated, but nothing shows on screen and four dots keep moving...

localai | 8:17PM DBG Response: {"created":1726859868,"id":"ca117b81-592b-4755-a01c-d1d409917ce9","data":[{"embedding":null,"index":0,"url":"http://localai.xxx/generated-images/b641806093114.png"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

Log 

localai     | 8:15PM DBG Request received: {"model":"stablediffusion","language":"","translate":false,"n":1,"top_p":null,"top_k":null,"temperature":null,"max_tokens":null,"echo":false,"batch":0,"ignore_eos":false,"repeat_penalty":0,"repeat_last_n":0,"n_keep":0,"frequency_penalty":0,"presence_penalty":0,"tfz":null,"typical_p":null,"seed":null,"negative_prompt":"","rope_freq_base":0,"rope_freq_scale":0,"negative_prompt_scale":0,"use_fast_tokenizer":false,"clip_skip":0,"tokenizer":"","file":"","size":"512x512","prompt":"cute cat sitting on kitchen table","instruction":"","input":null,"stop":null,"messages":null,"functions":null,"function_call":null,"stream":false,"mode":0,"step":0,"grammar":"","grammar_json_functions":null,"backend":"","model_base_name":""}
localai     | 8:15PM DBG Loading model: stablediffusion
localai     | 8:15PM DBG guessDefaultsFromFile: not a GGUF file
localai     | 8:15PM DBG Parameter Config: &{PredictionOptions:{Model:stablediffusion_assets Language: Translate:false N:0 TopP:0xc000af9c40 TopK:0xc000af9c48 Temperature:0xc000af9c50 Maxtokens:0xc000af9c80 Echo:false Batch:0 IgnoreEOS:false RepeatPenalty:0 RepeatLastN:0 Keep:0 FrequencyPenalty:0 PresencePenalty:0 TFZ:0xc000af9c78 TypicalP:0xc000af9c70 Seed:0xc000af9c98 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:stablediffusion F16:0xc000af9c38 Threads:0xc000af9c30 Debug:0xc001262e00 Roles:map[] Embeddings:0xc000af9c91 Backend:stablediffusion TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions: UseTokenizerTemplate:false JoinChatMessagesByCharacter:<nil>} PromptStrings:[cute cat sitting on kitchen table] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: ResponseFormat: ResponseFormatMap:map[] FunctionsConfig:{DisableNoAction:false GrammarConfig:{ParallelCalls:false DisableParallelNewLines:false MixedMode:false NoMixedFreeString:false NoGrammar:false Prefix: ExpectStringsAfterJSON:false PropOrder: SchemaType:} NoActionFunctionName: NoActionDescriptionName: ResponseRegex:[] JSONRegexMatch:[] ReplaceFunctionResults:[] ReplaceLLMResult:[] CaptureLLMResult:[] FunctionNameKey: FunctionArgumentsKey:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0xc000af9c68 MirostatTAU:0xc000af9c60 Mirostat:0xc000af9c58 NGPULayers:0xc000af9c88 MMap:0xc000af9c90 MMlock:0xc000af9c91 LowVRAM:0xc000af9c91 Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] TrimSuffix:[] ContextSize:0xc000af9c28 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: FlashAttention:false NoKVOffloading:false RopeScaling: ModelType: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{CUDA:false PipelineType: SchedulerType: EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder: ControlNet:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} TTSConfig:{Voice: VallE:{AudioPath:}} CUDA:false DownloadFiles:[{Filename:stablediffusion_assets/AutoencoderKL-256-256-fp16-opt.param SHA256:18ca4b66685e21406bcf64c484b3b680b4949900415536d599cc876579c85c82 URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/AutoencoderKL-256-256-fp16-opt.param} {Filename:stablediffusion_assets/AutoencoderKL-512-512-fp16-opt.param SHA256:cf45f63aacf3dbbab0f59ed92a6f2c14d9a1801314631cd3abe91e3c85639a20 URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/AutoencoderKL-512-512-fp16-opt.param} {Filename:stablediffusion_assets/AutoencoderKL-base-fp16.param SHA256:0254a056dce61b0c27dc9ec1b78b53bcf55315c540f55f051eb841aa992701ba URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/AutoencoderKL-base-fp16.param} {Filename:stablediffusion_assets/AutoencoderKL-encoder-512-512-fp16.bin SHA256:ddcb79a9951b9f91e05e087739ed69da2c1c4ae30ba4168cce350b49d617c9fa URI:https://github.com/EdVince/Stable-Diffusion-NCNN/releases/download/naifu/AutoencoderKL-encoder-512-512-fp16.bin} {Filename:stablediffusion_assets/AutoencoderKL-fp16.bin SHA256:f02e71f80e70252734724bbfaed5c4ddd3a8ed7e61bb2175ff5f53099f0e35dd URI:https://github.com/EdVince/Stable-Diffusion-NCNN/releases/download/naifu/AutoencoderKL-fp16.bin} {Filename:stablediffusion_assets/FrozenCLIPEmbedder-fp16.bin SHA256:1c9a12f4e1dd1b295a388045f7f28a2352a4d70c3dc96a542189a3dd7051fdd6 URI:https://github.com/EdVince/Stable-Diffusion-NCNN/releases/download/naifu/FrozenCLIPEmbedder-fp16.bin} {Filename:stablediffusion_assets/FrozenCLIPEmbedder-fp16.param SHA256:471afbe678dd1fd3fe764ef9c6eccaccb0a7d7e601f27b462aa926b20eb368c9 URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/FrozenCLIPEmbedder-fp16.param} {Filename:stablediffusion_assets/log_sigmas.bin SHA256:a2089f8aa4c61f9c200feaec541ab3f5c94233b28deb6d5e8bcd974fa79b68ac URI:https://github.com/EdVince/Stable-Diffusion-NCNN/raw/main/x86/linux/assets/log_sigmas.bin} {Filename:stablediffusion_assets/UNetModel-256-256-MHA-fp16-opt.param SHA256:a58c380229f09491776df837b7aa7adffc0a87821dc4708b34535da2e36e3da1 URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/UNetModel-256-256-MHA-fp16-opt.param} {Filename:stablediffusion_assets/UNetModel-512-512-MHA-fp16-opt.param SHA256:f12034067062827bd7f43d1d21888d1f03905401acf6c6eea22be23c259636fa URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/UNetModel-512-512-MHA-fp16-opt.param} {Filename:stablediffusion_assets/UNetModel-base-MHA-fp16.param SHA256:696f6975de49f4325b53ce32aff81861a6d6c07cd9ce3f0aae2cc405350af38d URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/UNetModel-base-MHA-fp16.param} {Filename:stablediffusion_assets/UNetModel-MHA-fp16.bin SHA256:d618918d011bfc1f644c0f2a33bf84931bd53b28a98492b0a8ed6f3a818852c3 URI:https://github.com/EdVince/Stable-Diffusion-NCNN/releases/download/naifu/UNetModel-MHA-fp16.bin} {Filename:stablediffusion_assets/vocab.txt SHA256:e30e57b6f1e47616982ef898d8922be24e535b4fa3d0110477b3a6f02ebbae7d URI:https://raw.githubusercontent.com/EdVince/Stable-Diffusion-NCNN/main/x86/linux/assets/vocab.txt}] Description:Stable Diffusion in NCNN with c++, supported txt2img and img2img
localai     |  Usage:curl http://localhost:8080/v1/images/generations \
localai     |   -H "Content-Type: application/json" \
localai     |   -d '{
localai     |     "prompt": "<positive prompt>|<negative prompt>",
localai     |     "step": 25,
localai     |     "size": "512x512"
localai     |   }'}
localai     | 8:15PM INF Loading model 'stablediffusion_assets' with backend stablediffusion
localai     | 8:15PM DBG Model already loaded in memory: stablediffusion_assets
localai     | 8:15PM DBG Checking model availability (stablediffusion_assets)
localai     | 8:15PM WRN GRPC Model not responding: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:42087: connect: connection refused"
localai     | 8:15PM WRN Deleting the process in order to recreate it
localai     | 8:15PM DBG GRPC Process is not responding: stablediffusion_assets
localai     | 8:15PM ERR (deleteProcess) error while deleting grpc process stablediffusion_assets error="process does not exist"
localai     | 8:15PM DBG Loading model in memory from file: /build/models/stablediffusion_assets
localai     | 8:15PM DBG Loading Model stablediffusion_assets with gRPC (file: /build/models/stablediffusion_assets) (backend: stablediffusion): {backendString:stablediffusion model:stablediffusion_assets threads:4 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc00024c488 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh openvoice:/build/backend/python/openvoice/run.sh parler-tts:/build/backend/python/parler-tts/run.sh rerankers:/build/backend/python/rerankers/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false parallelRequests:false}
localai     | 8:15PM DBG Loading GRPC Process: /tmp/localai/backend_data/backend-assets/grpc/stablediffusion
localai     | 8:15PM DBG GRPC Service for stablediffusion_assets will be running at: '127.0.0.1:45723'
localai     | 8:15PM DBG GRPC Service state dir: /tmp/go-processmanager3374170917
localai     | 8:15PM DBG GRPC Service Started
localai     | 8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stderr 2024/09/20 20:15:54 gRPC Server listening at 127.0.0.1:45723
localai     | 8:15PM DBG GRPC Service Ready
localai     | 8:15PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:stablediffusion_assets ContextSize:1024 Seed:440928848 NBatch:512 F16Memory:false MLock:false MMap:true VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:99999999 MainGPU: TensorSplit: Threads:4 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/build/models/stablediffusion_assets Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 TensorParallelSize:0 MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type: FlashAttention:false NoKVOffload:false}
localai     | 8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout ----------------[start stable diffusion]--------------------
localai     | 8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout ----------------[init]--------------------
localai     | 8:16PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stderr fopen /build/models/stablediffusion_assets/AutoencoderKL-encoder-512-512-fp16.param failed
localai     | 8:16PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stderr network graph not ready
localai     | 8:16PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout  3.54G / 3.63G
localai     | 8:16PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout ----------------[prompt]------------------ 3.54G / 3.63G
localai     | 8:16PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout ----------------[diffusion]---------------
localai     | 8:16PM INF Success ip=127.0.0.1 latency="37.853µs" method=GET status=200 url=/readyz
localai     | 8:17PM INF Success ip=127.0.0.1 latency="56.534µs" method=GET status=200 url=/readyz
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step: 1/15        6370.46ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step: 2/15        6375.90ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step: 3/15        6357.02ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step: 4/15        6348.03ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step: 5/15        6243.35ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step: 6/15        6328.86ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step: 7/15        6327.57ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step: 8/15        6393.32ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step: 9/15        6187.91ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step:10/15        6330.75ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step:11/15        6283.45ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step:12/15        6295.68ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step:13/15        6214.24ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step:14/15        6364.48ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout step:15/15        6285.27ms
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout ----------------[diffusion]--------------- 6.41G / 6.41G
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout ----------------[decode]------------------ 8.01G / 8.01G
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout ----------------[save]--------------------
localai     | 8:17PM DBG GRPC(stablediffusion_assets-127.0.0.1:45723): stdout ----------------[close]-------------------
localai     | 8:17PM DBG Response: {"created":1726859868,"id":"ca117b81-592b-4755-a01c-d1d409917ce9","data":[{"embedding":null,"index":0,"url":"http://localai.xxx/generated-images/b641806093114.png"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

Now when I connect with LocalAI via IP:port image shows below 'prompt' box. Hmm nginx proxy #3608?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working unconfirmed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Connor2573 @rabner @localai-bot