LoRA Fails to Match Full Fine-Tuning for Multi-Step Procedural Tasks, New Study Finds
A new study from ArXiv cs.AI finds that LoRA, a popular parameter-efficient fine-tuning method, fails to internalize multi-step procedures with conditional branching, uniformly underperforming full fine-tuning across all tested ranks (16–128) on a travel booking task.