How to use save_pretrained correctly

[nitinmukesh]

Hello,

I need some understanding on how to use save_pretrained

1. This code is creating the bf16 versions, however I noticed only text_encoder size is reduced to half but transformer and vae size is same as original.
   If I look at following repo the size of transformer is reduced to half Sana

Repo
https://huggingface.co/Lightricks/LTX-Video-0.9.5/tree/main

import torch
from diffusers import LTXConditionPipeline
import os

output_dir = "Lightricks-bf16/LTX-Video-0.9.5"
pipe = LTXConditionPipeline.from_pretrained(
    "Lightricks/LTX-Video-0.9.5",
    torch_dtype=torch.bfloat16
)
pipe.text_encoder = pipe.text_encoder.to(torch.bfloat16)
pipe.vae = pipe.vae.to(torch.bfloat16)
pipe.transformer = pipe.transformer.to(torch.bfloat16)
pipe.save_pretrained(
    output_dir,
    safe_serialization=True,
    variant="bf16"
)
print("Model saved successfully!")

└───LTX-Video-0.9.5
│ model_index.json
│
├───scheduler
│ scheduler_config.json
│
├───text_encoder ---------This is reduced from original ~18 GB to 8.87 GB
│ config.json
│ model.bf16-00001-of-00002.safetensors
│ model.bf16-00002-of-00002.safetensors
│ model.safetensors.index.bf16.json
│
├───tokenizer
│ added_tokens.json
│ special_tokens_map.json
│ spiece.model
│ tokenizer_config.json
│
├───transformer ---------------Same size as original
│ config.json
│ diffusion_pytorch_model.bf16.safetensors (3.58 GB)
│
└───vae ---------------Same size as original
│ config.json
│ diffusion_pytorch_model.bf16.safetensors (2.32 GB)

[asomoza]

Hi!, this is because the transformer model its already in bf16 while the one you're comparing it to (sana) its in FP32, you can easily check this in the HUB by clicking in the layers symbol:

[nitinmukesh]

Thank you asomoza.
I think file naming conventions (fp16, bf16, etc..) should be enforced.

[asomoza]

we try to suggest it but sadly we don't have a nice way of enforcing it, this is mostly a community effort and most people don't even bother to do it and let the guessing to the users, also most of the time they just publish one variant which is the case here but I don't know why did they left the text encoder in full precision.

The best method to get this going with the model authors is that the users of those models open issues in the repo asking them if they can change them.