stablediffusion and assets

[himpierre]

Hello Community!

I'm trying to use stablediffusion to generate some images via nextcloud openai integration. When I try to do that the docker logs the follwoing.

8:15PM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:stablediffusion_assets ContextSize:0 Seed:0 NBatch:0 F16Memory:false MLock:false MMap:false VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:0 MainGPU: TensorSplit: Threads:0 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/stablediffusion_assets Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType: SchedulerType: CUDA:false CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0}
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stdout ----------------[start generation upscaled image]------------------
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stdout positive_prompt: self-constructed dirndl 3d craft sheet 
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stdout output_png_path: /tmp/b643903930003.png
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stdout negative_prompt: 
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stdout step: 15
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stdout seed: 0
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stdout ----------------[prompt]------------------
[127.0.0.1]:57394  200  -  GET      /readyz
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stderr fopen /models/stablediffusion_assets/UNetModel-fp16.param failed
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stderr fopen /models/stablediffusion_assets/UNetModel-fp16.bin failed
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stdout ----------------[diffusion]---------------
8:15PM DBG GRPC(stablediffusion_assets-127.0.0.1:41875): stderr find_blob_index_by_name in0 failed

Do I need the assets at some point during the installation. Cannot find much in the documentaion. I suppose this works for someone?

cheers, t.

[sbe-arg]

I don't think this works for anyone anymore.
I get exactly the same error

[sbe-arg]

@mudler do you have any idea what we can/should do about this?

[sbe-arg]

These assets work https://github.com/fengwang/Stable-Diffusion-NCNN?tab=readme-ov-file @himpierre if you can also confirm on your end ill open a pr

[sbe-arg]

I'll be adding the assets here https://github.com/sbe-arg/Stable-Diffusion-NCNN_nextcloud_assets
and then we need to link them here https://github.com/mudler/LocalAI/blob/master/aio/cpu/image-gen.yaml
I have a temporal test here https://raw.githubusercontent.com/sbe-arg/LocalAI/stablediffusion/path/aio/cpu/image-gen.yaml
Opened pr

had to nuke the models volume on my docker setup... (I have a very ephemeral setup) but its working now

[sbe-arg]

now I'm having other problem

9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stdout ----------------[start generation upscaled image]------------------
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stdout positive_prompt: a dog
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stdout output_png_path: /tmp/generated/images/b642847695003.png
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stdout negative_prompt: 
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stdout step: 15
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stdout seed: 1855247344
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stdout ----------------[prompt]------------------
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stdout ----------------[diffusion]---------------
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stderr ModelBin read weight_data failed 135
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stderr layer load_model 9 pnnx_fold_v_8.1 failed
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stderr SIGSEGV: segmentation violation
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stderr PC=0xea7320 m=4 sigcode=1
9:55AM DBG GRPC(stablediffusion_assets-127.0.0.1:45997): stderr signal arrived during cgo execution

[sbe-arg]

anyone reading this, should use https://apps.nextcloud.com/apps/text2image_stablediffusion