init #2852

DerekLiu35 · 2025-05-13T20:33:02Z

Preparing the Article

Add an entry to _blog.yml.
Add a thumbnail. There are no requirements here, but there is a template if it's helpful.
Check you use a short title and blog path.
Upload any additional assets (such as images) to the Documentation Images repo. This is to reduce bloat in the GitHub base repo when cloning and pulling. Try to have small images to avoid a slow or expensive user experience.
Add metadata (such as authors) to your md file. You can also specify guest or org for the authors.
Ensure the publication date is correct.
Preview the content. A quick way is to paste the markdown content in https://huggingface.co/new-blog. Do not click publish, this is just a way to do an early check.

DerekLiu35 · 2025-05-13T20:34:05Z

@SunMarc

sayakpaul

Thanks for working on this.

diffusers-quantization.md

sayakpaul · 2025-05-15T04:44:05Z

diffusers-quantization.md

+
+**BF16:**
+
+![Baroque, Futurist, and Noir style images generated with BF16 precision](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/quantization-backends-diffusers/combined_flux-dev_bf16_combined.png)


Let's also provide an actual caption for the figure.

diffusers-quantization.md

sayakpaul · 2025-05-15T04:46:00Z

diffusers-quantization.md

+**BnB 4-bit:**
+
+![Baroque, Futurist, and Noir style images generated with BnB 4-bit](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/quantization-backends-diffusers/combined_flux-dev_bnb_4bit_combined.png)
+
+**BnB 8-bit:**
+![Baroque, Futurist, and Noir style images generated with BnB 8-bit](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/quantization-backends-diffusers/combined_flux-dev_bnb_8bit_combined.png)


Can we combine three images here?

BF16

4bit

8bit

Along with the caption?

sayakpaul · 2025-05-15T04:46:28Z

diffusers-quantization.md

+**BnB 8-bit:**
+![Baroque, Futurist, and Noir style images generated with BnB 8-bit](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/quantization-backends-diffusers/combined_flux-dev_bnb_8bit_combined.png)
+
+| BnB Precision | Memory after loading | Peak memory | Inference time |


Let's include the Bf16 numbers too.

sayakpaul · 2025-05-15T04:55:53Z

diffusers-quantization.md

+| Q8_0           | 21.502 GB            | 25.973 GB   | 15 seconds     |
+| Q2_k           | 13.264 GB            | 17.752 GB   | 26 seconds     |
+
+**Example (Flux-dev with GGUF Q4_1)**


I don't think we have to be exhaustive about showing snippets for every configuration unless they vary significantly from one another.

sayakpaul · 2025-05-15T04:57:14Z

diffusers-quantization.md

+
+For more information check out the [GGUF docs](https://huggingface.co/docs/diffusers/quantization/gguf).
+
+### FP8 Layerwise Casting (`enable_layerwise_casting`)


Could also write that can be combined with group_offloading:
https://huggingface.co/docs/diffusers/en/optimization/memory#group-offloading

sayakpaul · 2025-05-15T04:57:47Z

diffusers-quantization.md

+
+We created a setup where you can provide a prompt, and we generate results using both the original, high-precision model (e.g., Flux-dev in BF16) and several quantized versions (BnB 4-bit, BnB 8-bit). The generated images are then presented to you and your challenge is to identify which ones came from the quantized models.
+
+Try it out [here](https://huggingface.co/spaces/derekl35/flux-quant)!


Let's embed the space inside the blog post.

sayakpaul · 2025-05-15T04:58:42Z

diffusers-quantization.md

+Here's a quick guide to choosing a quantization backend:
+
+*   **Easiest Memory Savings (NVIDIA):** Start with `bitsandbytes` 4/8-bit.
+*   **Prioritize Inference Speed:** `torchao` + `torch.compile` offers the best performance potential.


GGUF also supports torch.compile(). So does bitsandbytes. I think we should mention that.

sayakpaul · 2025-05-15T04:59:12Z

diffusers-quantization.md

+*   **Simplicity (Hopper/Ada):** Explore FP8 Layerwise Casting (`enable_layerwise_casting`).
+*   **For Using Existing GGUF Models:** Use GGUF loading (`from_single_file`).
+
+Quantization significantly lowers the barrier to entry for using large diffusion models. Experiment with these backends to find the best balance of memory, speed, and quality for your needs.


Should we hint the readers that they can expect a follow-up blog around training with quantization?

Co-authored-by: Sayak Paul <[email protected]>

DerekLiu35 · 2025-05-15T16:12:52Z

diffusers-quantization.md

+### bitsandbytes (BnB)
+
+[`bitsandbytes`](https://github.com/bitsandbytes-foundation/bitsandbytes) is a popular and user-friendly library for 8-bit and 4-bit quantization, widely used for LLMs and QLoRA fine-tuning. We can use it for transformer-based diffusion and flow models, too.
+


Not sure if this is what you meant by combining the images

ChunTeLee · 2025-05-15T19:56:19Z

Exploring Quantization Backends in Diffusers thumbnail

Hey there, here is the thumbnail suggestion! cc @sayakpaul

sayakpaul · 2025-05-16T03:15:25Z

@ChunTeLee possible to reduce the size of the middle object a bit so that "exploring" and "quantization" words are clear?

init

c65f422

DerekLiu35 marked this pull request as ready for review May 13, 2025 20:33

SunMarc requested a review from sayakpaul May 14, 2025 17:17

sayakpaul reviewed May 15, 2025

View reviewed changes

DerekLiu35 and others added 3 commits May 15, 2025 09:55

Apply suggestions from code review

f094eab

Co-authored-by: Sayak Paul <[email protected]>

turn images into tables

d825e78

apply more suggestion from code review

43c3ca6

DerekLiu35 commented May 15, 2025

View reviewed changes

update thumbnail

7ac1eda

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

init #2852

init #2852

DerekLiu35 commented May 13, 2025

DerekLiu35 commented May 13, 2025

sayakpaul left a comment

sayakpaul May 15, 2025

sayakpaul May 15, 2025

sayakpaul May 15, 2025

sayakpaul May 15, 2025

sayakpaul May 15, 2025

sayakpaul May 15, 2025

sayakpaul May 15, 2025

sayakpaul May 15, 2025

DerekLiu35 May 15, 2025

ChunTeLee commented May 15, 2025 •

edited

Loading

sayakpaul commented May 16, 2025


		BF16:

		![Baroque, Futurist, and Noir style images generated with BF16 precision](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/quantization-backends-diffusers/combined_flux-dev_bf16_combined.png)


		For more information check out the [GGUF docs](https://huggingface.co/docs/diffusers/quantization/gguf).

		### FP8 Layerwise Casting (`enable_layerwise_casting`)


		We created a setup where you can provide a prompt, and we generate results using both the original, high-precision model (e.g., Flux-dev in BF16) and several quantized versions (BnB 4-bit, BnB 8-bit). The generated images are then presented to you and your challenge is to identify which ones came from the quantized models.

		Try it out [here](https://huggingface.co/spaces/derekl35/flux-quant)!

		### bitsandbytes (BnB)

		[`bitsandbytes`](https://github.com/bitsandbytes-foundation/bitsandbytes) is a popular and user-friendly library for 8-bit and 4-bit quantization, widely used for LLMs and QLoRA fine-tuning. We can use it for transformer-based diffusion and flow models, too.

init #2852

Are you sure you want to change the base?

init #2852

Conversation

DerekLiu35 commented May 13, 2025

Preparing the Article

DerekLiu35 commented May 13, 2025

sayakpaul left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChunTeLee commented May 15, 2025 • edited Loading

sayakpaul commented May 16, 2025

ChunTeLee commented May 15, 2025 •

edited

Loading