LoRa Weights Application

[alexcalovi98]

Hello @DenisovAV

I've tried to fine tune a Gemma 2b model using the Colab, everything works fine there.

When I try to apply LoRA Weights in my application, it seems that they don't apply.

I don't know if it's a bug or something wrong with my code. I've tried on iOS emulator and physical device (iPhone 14 Pro).

Thank you very much

[DenisovAV]

Hi, let me check on my side, if ecerything will be fine I'll come back and ask more details

[DenisovAV]

I was able to reproduce the issue, thank you for highlighting, I'll let you know when fix it

[alexcalovi98]

Thank you very much @DenisovAV

[DenisovAV]

I found the reason, I forgot to change model type, I tuned not IT model, so model type should be ModelType.general, not ModelType.gemmaIt, please try

final model = await _gemma.createModel(
  modelType: ModelType.general,
  preferredBackend: PreferredBackend.gpu,
);

[alexcalovi98]

I've tried but no differences. There is something very strange.

I saw theInferenceModel.swift file and the loraPath field is not passed to the session options.
I tried to pull the library and edit the line but it doesn't work.

Are you trying on an Android device? It's possibile that is a MediaPipe bug on iOS?

I cannot try on Android because I don't have a device with a good enough hardware to run AI locally

[DenisovAV]

Yes, I tried Android.

Thank you, I see now. Will fix it tomorrow, thank you for highlighting

[alexcalovi98]

By GPT-4.5 (released on 27 February 2025)

We can try to open an issue to Media Pipe?

[DenisovAV]

Hi, no, chatGPT is liar here :) it's already supported, but I forgot to fix it, I checked only on Android, It's my fault, I'm going to publish a new version today

[DenisovAV]

Please check 0.8.4 version

[alexcalovi98]

Thanks @DenisovAV, is the same fix that I applied locally but is still not working.

I bought a Pixel 7a to test AI on Android to check if there are differences. Do you confirm that it works on iOS?

[DenisovAV]

Yes, I checked with my example model and weights from my article, it works. How did you check? Can you provide your case and lora weights or dataset (if it's public), I'll check as well