28 stories tagged Performance
GardenBench is a new benchmark that evaluates Large Language Models (LLMs) daily, capturing performance changes—including accuracy, cost, and response time—that static benchmarks miss. It uses 11 test suites to compare models across providers and versions.
A new arXiv study pinpoints the damage from halving routed experts during prefill in MoE models: the F1 loss on short-answer QA is mostly answer formatting, concentrated in just a few prompt tokens like the chat-template tokens before the answer.
OpenAI has enhanced prompt caching in GPT-6, reducing latency and costs with better cache hit rates and new diagnostic tools. Users can now manage caching more effectively with explicit breakpoints and controls.
A new ArXiv paper introduces a formal framework for routing queries to the most cost-effective large language model, balancing performance and expense. The adaptive approach could reduce inference costs by sending simple queries to cheaper models and complex ones to more capable ones.
OpenAI transformed Habitat from a Python library into a globally distributed storage platform. It now handles over 22 million requests per second for ChatGPT's 1 billion users.
Anthropic released Claude Fable 5.1 and Mythos 5.1, addressing customer complaints about pricing, data retention, and overzealous safeguards. The company claims Fable 5.1 offers stronger performance than Fable 5 while costing up to 45% less for complex agentic workloads.
Checkly used AI agents to rewrite a critical Node.js service processing 92 million messages daily into Go, achieving significant performance and reliability improvements. The project demonstrates that AI can handle large-scale codebase refactoring.
OpenAI's custom inference chip, Jalapeño, achieves up to 2.5x higher throughput and 30% lower latency than the best available GPUs, setting new industry benchmarks for speed and power efficiency in AI inference.
Researchers introduced KVBoost, a chunk-level key-value cache reuse system for HuggingFace-compatible decoder models that enables reuse regardless of content position, reducing prefill latency in large language models.
A new study from ArXiv cs.CL reveals that four seemingly minor architectural decisions, each adopted by at least one of the Olmo, Llama, or Qwen model families, have a compoundingly negative effect on long-context extensibility. Any one choice alone has a minor impact, but combining three or more can cause significant performance drops.
Researchers introduced Nova, an automated end-to-end JIT compiler for deep learning that maps high-level mathematical operations directly to hardware. It promises to improve AI model performance and energy efficiency by providing whole-graph visibility and granular control over hardware and memory.
Researchers found that small tweaks in how questions are asked to AI models can drastically change their performance scores. This means current leaderboards might not be as reliable as they seem. The study introduces two new metrics to measure this effect, helping developers build more consistent AI systems.
Google released Litert.js, a tool that lets websites run AI models directly in your browser. This could make AI faster and more private for everyday users.
Role-Model is a new routing protocol and runtime that helps AI applications intelligently decide whether to use local or cloud-based models. It assigns domains and roles to models, enriches requests with task types, and offers a benchmark to compare speed, quality, and cost.
Nous Research announced Hermes MoA (Mixture of Agents) virtual models, which scored 8% higher than Opus 4.8 and 11% higher than GPT 5.5 on key AI benchmarks. This breakthrough could lead to better performance in everyday tasks like writing and problem-solving.
Larger AI models consistently outperform smaller ones in reasoning tasks. Researchers developed a new tool to study why this happens, revealing key differences in problem-solving approaches.
Researchers argue that after a benchmark's accuracy saturates, the focus should shift to six other key dimensions of AI performance: construct validity (shortcuts), out-of-distribution generalizability, efficiency, reliability, model vs. scaffold importance, and human–AI collaboration uplift.
AI agents have made huge strides in both performance and safety over the past two years. The best agent now completes nearly 90% of tasks and makes harmful mistakes just 2.5% of the time, down from 26%.
New research shows that adding memory to AI models can actually make them perform worse and more likely to agree with harmful ideas. This challenges the assumption that memory always improves AI.
A new tool ranks local AI models by performance and hardware needs, making it easier to pick the right one. This could save you time and effort when setting up AI tools on your computer.
OpenAI has introduced MRC, a new networking protocol designed to improve the performance and reliability of large-scale AI training. This could make AI models faster and more efficient to train.
Researchers have introduced Token Arena, a continuous benchmark that evaluates AI systems at the endpoint level. It measures five key factors to give a more realistic comparison of AI performance.
Gemma 4 31B demonstrated superior efficiency and speed in a game development contest, completing the task in a fraction of the time compared to Qwen 3.6 27B. The results highlight significant performance differences between the two models.
A 200-person Chinese team released DeepSeek V4, surpassing models from larger labs. The model achieves state-of-the-art performance on key benchmarks, challenging the dominance of well-funded Western AI labs.