Allow models to run without all text encoder(s)

[stduhpf]

For now only Flux and SD3.x.

Just puts a warning instead of crashing when text encoders are missing, and then proceed without it.

TODOs (maybe in follow up PRs):

• Re-enable gpu prompt processing if t5 isn't actually used
• Support unet models (SDXL ?)
• Default embeddings (non-zero) to mimic empty prompts?

Comparisons:

• Using clip_l/clip_g q8_0 and t5xxl q4_k.
• 5 steps
• 1024x1024
• default seed
• cfg 1
• default guidance
• tiled vae
• prompt: 'Illustration of a cute cat holding a sign saying "You do not need all text encoders!"'

SD3.5 Large Turbo (iq4_nl):

With t5_xxl:

	with clip_l	without clip_l
with clip_g
without clip_g

Without t5_xxl:

	with clip_l	without clip_l
with clip_g
without clip_g

Flux Schnell (iq4_nl imatrix):

	with clip_l	without clip_l
with T5_xxl
without T5_xxl

[rmatif]

Thanks to this, one can now run Flux on an 8GB Android phone

[Green-Sky]

@rmatif is your comment for this pr specifically? ... kind of sounds not related.

BTW, did you try one of the flux 8B "lite" prunes?
https://huggingface.co/Green-Sky/flux.1-lite-8B-GGUF/tree/main/lora-experiments
Here with hyper-sd lora merged, for lower step count.

[stduhpf]

@Green-Sky I think @rmatif meant that with this PR it's possible to drop T5, which makes Flux fit in only 8GB of system memory.

[rmatif]

@Green-Sky I think @rmatif meant that with this PR it's possible to drop T5, which makes Flux fit in only 8GB of system memory.

This is exactly what I meant, sorry if I wasn't clear. With this PR, we can drop the heavy T5, so we can squeeze Flux into just an 8GB phone.

@Green-Sky I just tested Flux.1-lite and the q4_k version can also fit now into those kinds of devices, although you can't run inference on resolutions larger than 512x512 due to the compute buffer, but I bet q3_k will do just fine.

[leejet]

Thank you for your contribution.

[LostRuins]

haha i love this thing