UniMatrix: Sparse Retrieval Meets Structured Recurrence in Language Models
Researchers introduce UniMatrix, a Universal Transformer variant that combines sparse retrieval with structured recurrence for efficient language modeling. The model achieves strong performance on associative recall tasks while maintaining computational efficiency.