Description
The LoRA config format (lora_parameters in the YAML, e.g. mlx_lm/examples/lora_config.yaml)
requires a key named scale. Many LoRA tutorials and other frameworks (e.g. PEFT) use
alpha for the equivalent concept, so it's an easy mistake to make. When alpha is
passed instead of scale, the failure is a bare KeyError: 'scale' raised deep in
mlx_lm/tuner/utils.py (to_lora), with no message pointing at the actual problem.
To reproduce
- Create a config with:
lora_parameters:
keys: ["self_attn.q_proj", "self_attn.v_proj"]
rank: 8
alpha: 16
dropout: 0.0
Description
The LoRA config format (
lora_parametersin the YAML, e.g.mlx_lm/examples/lora_config.yaml)requires a key named
scale. Many LoRA tutorials and other frameworks (e.g. PEFT) usealphafor the equivalent concept, so it's an easy mistake to make. Whenalphaispassed instead of
scale, the failure is a bareKeyError: 'scale'raised deep inmlx_lm/tuner/utils.py(to_lora), with no message pointing at the actual problem.To reproduce