Attention-Guided Layer Selection Improves LLM Factuality on TruthfulQA
Researchers introduced three new attention-guided strategies—Attention-JSD, Attention-Entropy-Max, and Attention-Entropy-Min—to improve the factual accuracy of large language models by leveraging self-attention mechanisms for better layer selection during contrastive decoding.
