Training Loras and fine tuning

[newsand]

Can i train LoRas for Bonsai image models (or even for text models) using Kohya or do i need some special software for running your ternary models? it would be very nice to have a guide for that..

[khosravipasha]

Yeah let me ask the team to come up with a nice guide for LoRas

In the meantime:

1. Yes, should be able to train LoRa with out of the box methods since the model weights will be frozen for LoRAs. Might need to use the "unpacked" versions though instead since current support is inference only for the packed versions: https://huggingface.co/collections/prism-ml/bonsai-auxiliary

2. After training the LoRa need to convert/pack into to a format llama.cpp or mlx likes to be able to do inference with the LoRA.

[khosravipasha]

There are also few discussions in lora channel in our discord: https://discord.com/invite/prismml

[Bullish-Design]

Yeah let me ask the team to come up with a nice guide for LoRas

In the meantime:

1. Yes, should be able to train LoRa with out of the box methods since the model weights will be frozen for LoRAs. Might need to use the "unpacked" versions though instead since current support is inference only for the packed versions: https://huggingface.co/collections/prism-ml/bonsai-auxiliary

2. After training the LoRa need to convert/pack into to a format llama.cpp or mlx likes to be able to do inference with the LoRA.

Would be very appreciative of a LoRA guide for these! The idea of a fine tuned binary model for specific tasks seems like it'd be a game changer.