7 stories tagged Long Context
Researchers introduced RBS-Attention, a training-free sparse-prefill method that reduces the prefill time for long-context large language models by using two complementary selection branches to avoid missing relevant tokens.
A new study from ArXiv cs.CL reveals that four seemingly minor architectural decisions, each adopted by at least one of the Olmo, Llama, or Qwen model families, have a compoundingly negative effect on long-context extensibility. Any one choice alone has a minor impact, but combining three or more can cause significant performance drops.
Researchers created a new test to measure AI's ability to connect clues scattered across long documents. This could improve how AI understands complex real-world information.
Researchers developed a technique called ProxyCoT to help AI models reason better with long texts. This method trains models to use shorter, relevant parts of long documents to solve complex problems.
Scientists have discovered a fundamental flaw in a widely used AI technique called RoPE, which helps models understand long texts. As texts get longer, RoPE loses its ability to focus on relevant information, making AI responses less reliable.
Researchers have developed a new AI memory system called Lossless Context Management (LCM) that handles long texts better than Claude Code. This could make AI assistants more reliable for tasks requiring large amounts of information.
NVIDIA's new Nemotron 3 Nano Omni model supports long-context multimodal intelligence across documents, audio, and video. It is designed for developers to build advanced AI agents.