Multiple adapters with core small language model architecture drafter

[Aisuko]

I submit a proposal of "Multiple adapters with core small language model architecture". It can be seen as a solution for solving multi-modal real word issues.

(Q)LoRA
https://www.kaggle.com/code/aisuko/fine-tuning-phi-2-with-qlora
VeRA(trl)
https://www.kaggle.com/code/aisuko/producing-adapter-with-vera

The proposal of the architecture see Google Doc

Related issues

• [feature] Support multiple adapters for core small language models kimchima#15