17 stories tagged Performance
Researchers found that small tweaks in how questions are asked to AI models can drastically change their performance scores. This means current leaderboards might not be as reliable as they seem. The study introduces two new metrics to measure this effect, helping developers build more consistent AI systems.
Google released Litert.js, a tool that lets websites run AI models directly in your browser. This could make AI faster and more private for everyday users.
Role-Model is a new routing protocol and runtime that helps AI applications intelligently decide whether to use local or cloud-based models. It assigns domains and roles to models, enriches requests with task types, and offers a benchmark to compare speed, quality, and cost.
Nous Research announced Hermes MoA (Mixture of Agents) virtual models, which scored 8% higher than Opus 4.8 and 11% higher than GPT 5.5 on key AI benchmarks. This breakthrough could lead to better performance in everyday tasks like writing and problem-solving.
Larger AI models consistently outperform smaller ones in reasoning tasks. Researchers developed a new tool to study why this happens, revealing key differences in problem-solving approaches.
Researchers argue that after a benchmark's accuracy saturates, the focus should shift to six other key dimensions of AI performance: construct validity (shortcuts), out-of-distribution generalizability, efficiency, reliability, model vs. scaffold importance, and human–AI collaboration uplift.
AI agents have made huge strides in both performance and safety over the past two years. The best agent now completes nearly 90% of tasks and makes harmful mistakes just 2.5% of the time, down from 26%.
New research shows that adding memory to AI models can actually make them perform worse and more likely to agree with harmful ideas. This challenges the assumption that memory always improves AI.
A new tool ranks local AI models by performance and hardware needs, making it easier to pick the right one. This could save you time and effort when setting up AI tools on your computer.
OpenAI has introduced MRC, a new networking protocol designed to improve the performance and reliability of large-scale AI training. This could make AI models faster and more efficient to train.
Researchers have introduced Token Arena, a continuous benchmark that evaluates AI systems at the endpoint level. It measures five key factors to give a more realistic comparison of AI performance.
Gemma 4 31B demonstrated superior efficiency and speed in a game development contest, completing the task in a fraction of the time compared to Qwen 3.6 27B. The results highlight significant performance differences between the two models.
A 200-person Chinese team released DeepSeek V4, surpassing models from larger labs. The model achieves state-of-the-art performance on key benchmarks, challenging the dominance of well-funded Western AI labs.
Alibaba has released Qwen3.6-27B, an open-source model with 27 billion parameters that excels in coding tasks, surpassing its larger predecessor. This model demonstrates significant advancements in agentic coding capabilities.
DeepSeek has previewed two new AI models, DeepSeek V4 and DeepSeek V4+, that outperform its previous V3.2 model and significantly close the gap with leading models. The improvements come from architectural enhancements that boost efficiency and performance.
A C++ LLM inference engine from scratch has output tokens costing 5x more. The article explores the reasons behind this increased cost.
AMD's AI director claims Claude Code has become dumber and lazier since its update. This statement raises concerns about the model's performance.