New Research Reveals Why Fine-Tuning Causes Hallucinations in LLMs
Fine-tuning large language models can increase hallucinations by degrading pre-trained knowledge. Researchers propose a self-distillation method to mitigate this issue. This could reshape how we approach model training.