Need some assistance with Text to Image failing on fresh AIO GPU docker for windows Local-AI Instance

[ThatFrostyGuy]

Hey everyone! I am receiving an error after building a new image using "docker run -p 8080:8080 --gpus all --name local-ai -ti localai/localai:latest-aio-gpu-nvidia-cuda-12"

When attempting to generate any images using the default stable diffusion model, I receive the following error:

could not load model (no success): Unexpected err=RepositoryNotFoundError('401 Client Error. (Request ID: Root=1-66de0c53-15c3eb6052783e1c1e555e4d;71959834-58b2-4e87-be6c-cffa8a049f20)\n\nRepository Not Found for url: https://huggingface.co/api/models/runwayml/stable-diffusion-v1-5/revision/main.\nPlease make sure you specified the correct repo_id and repo_type.\nIf you are trying to access a private or gated repo, make sure you are authenticated.\nInvalid username or password.'), type(err)=

I have attempted to load different AIO images, and now always get this error. I am thinking it is because the https://huggingface.co/api/models/runwayml/stable-diffusion-v1-5/revision/main URL does not lead anywhere any longer.

This error happens when I attempt to run any of the models in the default LocalAI gallery in the Image Generation repository. I must be doing something wrong on my end since I cant find any other people with this error.

Any advice for a noobie such as myself?

[gnoumph]

Hello, not sure your doing something wrong, like you I suspect the invalid https://huggingface.co/api/models/runwayml/stable-diffusion-v1-5/revision/main URL.

Got the same issue with dreamshaper model, using local-ai:latest-gpu-nvidia-cuda-12 on Linux.

INF Loading model 'DreamShaper_8_pruned.safetensors' with backend diffusers
ERR Server error error="could not load model (no success): Unexpected err=RepositoryNotFoundError('401 Client Error. (Request ID: Root=XXX;XXX)\\n\\nRepository Not Found for url: https://huggingface.co/api/models/runwayml/stable-diffusion-v1-5/revision/main.\\nPlease make sure you specified the correct `repo_id` and `repo_type`.\\nIf you are trying to access a private or gated repo, make sure you are authenticated.\\nInvalid username or password.'), type(err)=<class 'huggingface_hub.utils._errors.RepositoryNotFoundError'>" ip=172.18.0.1 latency=2.278204232s method=POST status=500 url=/v1/images/generations

Everything works a few days ago, and I didn't change anything. I just tried to reinstall LocalAI and all the models, with the same result.

[robdeweese]

same for me, but text generation doesnt work either

[gnoumph]

Got an issue with text generation too, I had to force llama backend in config files:

._gallery_model.yaml

# ...
config_file: |
  backend: llama
# ...

model.yaml

backend: llama
# ...

See:

• https://localai.io/advanced/#full-config-model-file-reference
• https://localai.io/model-compatibility/
• https://localai.io/features/text-generation/#llamacpp

[gnoumph]

As far as I understand, first time you generate an image with backend diffusers (the default one), LocalAI needs to download a model, the one supposed to be at https://huggingface.co/api/models/runwayml/stable-diffusion-v1-5/revision/main.
But this URL is not valid anymore, so download fails.

As a workaround, I think we can use stablediffusion-cpp backend instead diffusers (see https://localai.io/features/image-generation/#stablediffusion-cpp).
I tried on my Docker instance, but got this error:

grpc process not found: /tmp/localai/backend_data/backend-assets/grpc/stablediffusion-cpp. some backends(stablediffusion, tts) require LocalAI compiled with GO_TAGS

Strange, because:

Note: In order to use the images/generation endpoint with the stablediffusion C++ backend, you need to build LocalAI with GO_TAGS=stablediffusion. If you are using the container images, it is already enabled.

I am using a container image, so I don't understand, but nevermind. Let's try to rebuild everything.
I add GO_TAGS=stablediffusion and REBUILD=true to my docker-compose.yml file, wait A LOT that everything was built, and... same "grpc process not found" error -_-'

So here I am, I don't know what to do next... If someone has an idea! Maybe the best is to wait for an official fix :/

---

EDIT: I found a dirty workaround.

# Connect to your container:
docker exec -it <container name or id> bash

# Replace "runwayml" occurrences with "stable-diffusion-v1-5" in all files in "/build" directory:
find /build/ -type f -print0 | xargs -0 sed -i 's/runwayml/stable-diffusion-v1-5/g'

[gnoumph]

You can connect to your container with:

docker exec -it <container name or id> bash

Then you just have to execute the previous command.

I can't give you more detailed explanations right now, but if you need I'll do it ASAP ;)

Hope it helps!

[ThatFrostyGuy]

Woa that worked!!!!! Nice thinking!

[gnoumph]

Perfect :) Keep in mind that's a really dirty workaround. You'll have to redo the process everytime the container is recreated.

Now we just have to wait for a real fix!

[robdeweese]

worked for me, thanks

[jakommo]

Thanks for sharing the workaround. Did a bit more digging and the issue relates to #3431 and #3480 (and probably also huggingface/diffusers#9322)