Skip to content

Commit 5b8d16a

Browse files
leejetleejet
committed
docs: reorganize README.md
1 parent 3d854f7 commit 5b8d16a

File tree

8 files changed

+136
-132
lines changed

8 files changed

+136
-132
lines changed

‎README.md

+8-132
Original file line numberDiff line numberDiff line change
@@ -202,26 +202,6 @@ arguments:
202202
-v, --verbose print extra info
203203
```
204204
205-
#### Quantization
206-
207-
You can specify the model weight type using the `--type` parameter. The weights are automatically converted when loading the model.
208-
209-
- `f16` for 16-bit floating-point
210-
- `f32` for 32-bit floating-point
211-
- `q8_0` for 8-bit integer quantization
212-
- `q5_0` or `q5_1` for 5-bit integer quantization
213-
- `q4_0` or `q4_1` for 4-bit integer quantization
214-
215-
#### Convert to GGUF
216-
217-
You can also convert weights in the formats `ckpt/safetensors/diffusers` to gguf and perform quantization in advance, avoiding the need for quantization every time you load them.
218-
219-
For example:
220-
221-
```sh
222-
./bin/sd -M convert -m ../models/v1-5-pruned-emaonly.safetensors -o ../models/v1-5-pruned-emaonly.q8_0.gguf -v --type q8_0
223-
```
224-
225205
#### txt2img example
226206
227207
```sh
@@ -250,119 +230,15 @@ Using formats of different precisions will yield results of varying quality.
250230
<img src="./assets/img2img_output.png" width="256x">
251231
</p>
252232

253-
#### with LoRA
254-
255-
- You can specify the directory where the lora weights are stored via `--lora-model-dir`. If not specified, the default is the current working directory.
256-
257-
- LoRA is specified via prompt, just like [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#lora).
258-
259-
Here's a simple example:
260-
261-
```
262-
./bin/sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat<lora:marblesh:1>" --lora-model-dir ../models
263-
```
264-
265-
`../models/marblesh.safetensors` or `../models/marblesh.ckpt` will be applied to the model
266-
267-
#### LCM/LCM-LoRA
268-
269-
- Download LCM-LoRA form https://huggingface.co/latent-consistency/lcm-lora-sdv1-5
270-
- Specify LCM-LoRA by adding `<lora:lcm-lora-sdv1-5:1>` to prompt
271-
- It's advisable to set `--cfg-scale` to `1.0` instead of the default `7.0`. For `--steps`, a range of `2-8` steps is recommended. For `--sampling-method`, `lcm`/`euler_a` is recommended.
272-
273-
Here's a simple example:
274-
275-
```
276-
./bin/sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat<lora:lcm-lora-sdv1-5:1>" --steps 4 --lora-model-dir ../models -v --cfg-scale 1
277-
```
278-
279-
| without LCM-LoRA (--cfg-scale 7) | with LCM-LoRA (--cfg-scale 1) |
280-
| ---- |---- |
281-
| ![](./assets/without_lcm.png) |![](./assets/with_lcm.png) |
282-
283-
#### Using TAESD to faster decoding
284-
285-
You can use TAESD to accelerate the decoding of latent images by following these steps:
286-
287-
- Download the model [weights](https://huggingface.co/madebyollin/taesd/blob/main/diffusion_pytorch_model.safetensors).
288-
289-
Or curl
290-
291-
```bash
292-
curl -L -O https://huggingface.co/madebyollin/taesd/blob/main/diffusion_pytorch_model.safetensors
293-
```
294-
295-
- Specify the model path using the `--taesd PATH` parameter. example:
296-
297-
```bash
298-
sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat" --taesd ../models/diffusion_pytorch_model.safetensors
299-
```
300-
301-
#### Using ESRGAN to upscale results
302-
303-
You can use ESRGAN to upscale the generated images. At the moment, only the [RealESRGAN_x4plus_anime_6B.pth](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth) model is supported. Support for more models of this architecture will be added soon.
304-
305-
- Specify the model path using the `--upscale-model PATH` parameter. example:
306-
307-
```bash
308-
sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat" --upscale-model ../models/RealESRGAN_x4plus_anime_6B.pth
309-
```
310-
311-
#### Using PhotoMaker to personalize image generation
312-
313-
You can use [PhotoMaker](https://github.com/TencentARC/PhotoMaker) to personalize generated images with your own ID.
314-
315-
**NOTE**, currently PhotoMaker **ONLY** works with **SDXL** (any SDXL model files will work).
316-
317-
Download PhotoMaker model file (in safetensor format) [here](https://huggingface.co/bssrdf/PhotoMaker). The official release of the model file (in .bin format) does not work with ```stablediffusion.cpp```.
318-
319-
- Specify the PhotoMaker model path using the `--stacked-id-embd-dir PATH` parameter.
320-
- Specify the input images path using the `--input-id-images-dir PATH` parameter.
321-
- input images **must** have the same width and height for preprocessing (to be improved)
322-
323-
In prompt, make sure you have a class word followed by the trigger word ```"img"``` (hard-coded for now). The class word could be one of ```"man, woman, girl, boy"```. If input ID images contain asian faces, add ```Asian``` before the class
324-
word.
325-
326-
Another PhotoMaker specific parameter:
327-
328-
- ```--style-ratio (0-100)%```: default is 20 and 10-20 typically gets good results. Lower ratio means more faithfully following input ID (not necessarily better quality).
329-
330-
Other parameters recommended for running Photomaker:
331-
332-
- ```--cfg-scale 5.0```
333-
- ```-H 1024```
334-
- ```-W 1024```
335-
336-
If on low memory GPUs (<= 8GB), recommend running with ```--vae-on-cpu``` option to get artifact free images.
337-
338-
Example:
339-
340-
```bash
341-
bin/sd -m ../models/sdxlUnstableDiffusers_v11.safetensors --vae ../models/sdxl_vae.safetensors --stacked-id-embd-dir ../models/photomaker-v1.safetensors --input-id-images-dir ../assets/examples/scarletthead_woman -p "a girl img, retro futurism, retro game art style but extremely beautiful, intricate details, masterpiece, best quality, space-themed, cosmic, celestial, stars, galaxies, nebulas, planets, science fiction, highly detailed" -n "realistic, photo-realistic, worst quality, greyscale, bad anatomy, bad hands, error, text" --cfg-scale 5.0 --sampling-method euler -H 1024 -W 1024 --style-ratio 10 --vae-on-cpu -o output.png
342-
```
343-
344-
### Docker
345-
346-
#### Building using Docker
347-
348-
```shell
349-
docker build -t sd .
350-
```
351-
352-
#### Run
353-
354-
```shell
355-
docker run -v /path/to/models:/models -v /path/to/output/:/output sd [args...]
356-
# For example
357-
# docker run -v ./models:/models -v ./build:/output sd -m /models/sd-v1-4.ckpt -p "a lovely cat" -v -o /output/output.png
358-
```
359-
360-
## Memory Requirements
233+
## More Guides
361234

362-
| precision | f32 | f16 |q8_0 |q5_0 |q5_1 |q4_0 |q4_1 |
363-
| ---- | ---- |---- |---- |---- |---- |---- |---- |
364-
| **Memory** (txt2img - 512 x 512) | ~2.8G | ~2.3G | ~2.1G | ~2.0G | ~2.0G | ~2.0G | ~2.0G |
365-
| **Memory** (txt2img - 512 x 512) *with Flash Attention* | ~2.4G | ~1.9G | ~1.6G | ~1.5G | ~1.5G | ~1.5G | ~1.5G |
235+
- [LoRA](./docs/lora.md)
236+
- [LCM/LCM-LoRA](./docs/lcm.md)
237+
- [Using PhotoMaker to personalize image generation](./docs/photo_maker.md)
238+
- [Using ESRGAN to upscale results](./docs/esrgan.md)
239+
- [Using TAESD to faster decoding](./docs/taesd.md)
240+
- [Docker](./docs/docker.md)
241+
- [Quantization and GGUF](./docs/quantization_and_gguf.md)
366242

367243
## Bindings
368244

‎docs/docker.md

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
## Docker
2+
3+
### Building using Docker
4+
5+
```shell
6+
docker build -t sd .
7+
```
8+
9+
### Run
10+
11+
```shell
12+
docker run -v /path/to/models:/models -v /path/to/output/:/output sd [args...]
13+
# For example
14+
# docker run -v ./models:/models -v ./build:/output sd -m /models/sd-v1-4.ckpt -p "a lovely cat" -v -o /output/output.png
15+
```

‎docs/esrgan.md

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
## Using ESRGAN to upscale results
2+
3+
You can use ESRGAN to upscale the generated images. At the moment, only the [RealESRGAN_x4plus_anime_6B.pth](https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.2.4/RealESRGAN_x4plus_anime_6B.pth) model is supported. Support for more models of this architecture will be added soon.
4+
5+
- Specify the model path using the `--upscale-model PATH` parameter. example:
6+
7+
```bash
8+
sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat" --upscale-model ../models/RealESRGAN_x4plus_anime_6B.pth
9+
```

‎docs/lcm.md

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
## LCM/LCM-LoRA
2+
3+
- Download LCM-LoRA form https://huggingface.co/latent-consistency/lcm-lora-sdv1-5
4+
- Specify LCM-LoRA by adding `<lora:lcm-lora-sdv1-5:1>` to prompt
5+
- It's advisable to set `--cfg-scale` to `1.0` instead of the default `7.0`. For `--steps`, a range of `2-8` steps is recommended. For `--sampling-method`, `lcm`/`euler_a` is recommended.
6+
7+
Here's a simple example:
8+
9+
```
10+
./bin/sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat<lora:lcm-lora-sdv1-5:1>" --steps 4 --lora-model-dir ../models -v --cfg-scale 1
11+
```
12+
13+
| without LCM-LoRA (--cfg-scale 7) | with LCM-LoRA (--cfg-scale 1) |
14+
| ---- |---- |
15+
| ![](../assets/without_lcm.png) |![](../assets/with_lcm.png) |

‎docs/lora.md

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
## LoRA
2+
3+
- You can specify the directory where the lora weights are stored via `--lora-model-dir`. If not specified, the default is the current working directory.
4+
5+
- LoRA is specified via prompt, just like [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#lora).
6+
7+
Here's a simple example:
8+
9+
```
10+
./bin/sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat<lora:marblesh:1>" --lora-model-dir ../models
11+
```
12+
13+
`../models/marblesh.safetensors` or `../models/marblesh.ckpt` will be applied to the model

‎docs/photo_maker.md

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
## Using PhotoMaker to personalize image generation
2+
3+
You can use [PhotoMaker](https://github.com/TencentARC/PhotoMaker) to personalize generated images with your own ID.
4+
5+
**NOTE**, currently PhotoMaker **ONLY** works with **SDXL** (any SDXL model files will work).
6+
7+
Download PhotoMaker model file (in safetensor format) [here](https://huggingface.co/bssrdf/PhotoMaker). The official release of the model file (in .bin format) does not work with ```stablediffusion.cpp```.
8+
9+
- Specify the PhotoMaker model path using the `--stacked-id-embd-dir PATH` parameter.
10+
- Specify the input images path using the `--input-id-images-dir PATH` parameter.
11+
- input images **must** have the same width and height for preprocessing (to be improved)
12+
13+
In prompt, make sure you have a class word followed by the trigger word ```"img"``` (hard-coded for now). The class word could be one of ```"man, woman, girl, boy"```. If input ID images contain asian faces, add ```Asian``` before the class
14+
word.
15+
16+
Another PhotoMaker specific parameter:
17+
18+
- ```--style-ratio (0-100)%```: default is 20 and 10-20 typically gets good results. Lower ratio means more faithfully following input ID (not necessarily better quality).
19+
20+
Other parameters recommended for running Photomaker:
21+
22+
- ```--cfg-scale 5.0```
23+
- ```-H 1024```
24+
- ```-W 1024```
25+
26+
If on low memory GPUs (<= 8GB), recommend running with ```--vae-on-cpu``` option to get artifact free images.
27+
28+
Example:
29+
30+
```bash
31+
bin/sd -m ../models/sdxlUnstableDiffusers_v11.safetensors --vae ../models/sdxl_vae.safetensors --stacked-id-embd-dir ../models/photomaker-v1.safetensors --input-id-images-dir ../assets/examples/scarletthead_woman -p "a girl img, retro futurism, retro game art style but extremely beautiful, intricate details, masterpiece, best quality, space-themed, cosmic, celestial, stars, galaxies, nebulas, planets, science fiction, highly detailed" -n "realistic, photo-realistic, worst quality, greyscale, bad anatomy, bad hands, error, text" --cfg-scale 5.0 --sampling-method euler -H 1024 -W 1024 --style-ratio 10 --vae-on-cpu -o output.png
32+
```

‎docs/quantization_and_gguf.md

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
## Quantization
2+
3+
You can specify the model weight type using the `--type` parameter. The weights are automatically converted when loading the model.
4+
5+
- `f16` for 16-bit floating-point
6+
- `f32` for 32-bit floating-point
7+
- `q8_0` for 8-bit integer quantization
8+
- `q5_0` or `q5_1` for 5-bit integer quantization
9+
- `q4_0` or `q4_1` for 4-bit integer quantization
10+
11+
12+
### Memory Requirements of Stable Diffusion 1.x
13+
14+
| precision | f32 | f16 |q8_0 |q5_0 |q5_1 |q4_0 |q4_1 |
15+
| ---- | ---- |---- |---- |---- |---- |---- |---- |
16+
| **Memory** (txt2img - 512 x 512) | ~2.8G | ~2.3G | ~2.1G | ~2.0G | ~2.0G | ~2.0G | ~2.0G |
17+
| **Memory** (txt2img - 512 x 512) *with Flash Attention* | ~2.4G | ~1.9G | ~1.6G | ~1.5G | ~1.5G | ~1.5G | ~1.5G |
18+
19+
## Convert to GGUF
20+
21+
You can also convert weights in the formats `ckpt/safetensors/diffusers` to gguf and perform quantization in advance, avoiding the need for quantization every time you load them.
22+
23+
For example:
24+
25+
```sh
26+
./bin/sd -M convert -m ../models/v1-5-pruned-emaonly.safetensors -o ../models/v1-5-pruned-emaonly.q8_0.gguf -v --type q8_0
27+
```

‎docs/taesd.md

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
## Using TAESD to faster decoding
2+
3+
You can use TAESD to accelerate the decoding of latent images by following these steps:
4+
5+
- Download the model [weights](https://huggingface.co/madebyollin/taesd/blob/main/diffusion_pytorch_model.safetensors).
6+
7+
Or curl
8+
9+
```bash
10+
curl -L -O https://huggingface.co/madebyollin/taesd/blob/main/diffusion_pytorch_model.safetensors
11+
```
12+
13+
- Specify the model path using the `--taesd PATH` parameter. example:
14+
15+
```bash
16+
sd -m ../models/v1-5-pruned-emaonly.safetensors -p "a lovely cat" --taesd ../models/diffusion_pytorch_model.safetensors
17+
```

0 commit comments

Comments
 (0)