Exporting Keras Llama Checkpoint to HF

[salrowili]

Hi,

As the Llama3 is popular model, it would be great if we can have a script that export Llama Keras checkpoint to HF. The code is a already exist for Gemma : https://github.com/keras-team/keras-hub/blob/8ca2076d651532f7270ffa2beba44d377b100bbf/tools/gemma/export_gemma_to_hf.py

Something similar to Llama would be a great!
Thanks

[Gopi-Uppari]

Hi,

Noted it.

We can use the keras_hub library to easily integrate Keras models with Hugging Face. Here iss how to export a Keras Llama model to Hugging Face:

1. Load a pre-trained Llama model from the Hugging Face Hub using keras_hub.
2. Save the fine-tuned model locally.
3. Upload the model to the Hugging Face Hub for easy sharing and future use.

For more details, please check the reference provided.

Thank you.

[salrowili]

Hi @Gopi-Uppari ,
Thank you for taking care of this.

Yes, i have used Keras to load my custom load using "keras_hub.models.Llama3CausalLM.from_preset("./local_folder") and it work pretty well. However, the main reason to ask for this feature because the built-in generate method with keras_hub give me little options regarding things like terminators, top_k, Sample, temperature. i had to hard coded it.

[Gopi-Uppari]

Hi @salrowili,

Okay, could you please confirm if this issue is resolved for you with the above comment ? Please feel free to close the issue if it is resolved ?

Thank you.

[salrowili]

Hi @Gopi-Uppari ,
The issue has not been resolved because we still need to have a script that convert Keras checkpoint to HF format for Llama. The solution you have suggested and what "keras_hub.models.Llama3CausalLM.from_preset" do is the opposite . It load HF model into Keras compatible checkpoint.

[sachinprasadhs]

@salrowili , If you you are asking about the weight files for Llama in HF, here is the link https://huggingface.co/keras/llama3_8b_en_int8/tree/main

[salrowili]

Hi @sachinprasadhs ,
Thank you for trying to help. To illustrate the issue that i am facing take this example.
I have used keras_hub to finetune llama3 8b on my custom SFT data. Then, i want to convert the finetuned llm to HF so i can use it in production with HF library. Thus, what i am looking for is a script that can convert the finetuned Keras checkpoint to HF format. A similar script exists for Gemma 2 [ tools/gemma/export_gemma_to_hf.py ] but not for llama.

[sachinprasadhs]

Understood, Thanks for clarifying, as of now we don't have conversion script to convert from Keras to HuggingFace for Llama, instead we have a conversion script to convert HuggingFace checkpoint to Keras checkpoint here https://github.com/keras-team/keras-hub/blob/master/tools/checkpoint_conversion/convert_llama_checkpoints.py
You can refer this above script and create one for Keras to HuggingFace.