How OpenAI Scaled Habitat to Store ChatGPT Data for 1 Billion Users
OpenAI transformed Habitat from a Python library into a globally distributed storage platform. It now handles over 22 million requests per second for ChatGPT's 1 billion users.
#performance
Performance
24 stories tagged Performance
Anthropic Launches Claude Fable 5.1: Up to 45% Cheaper for Complex Agentic Tasks
Anthropic released Claude Fable 5.1 and Mythos 5.1, addressing customer complaints about pricing, data retention, and overzealous safeguards. The company claims Fable 5.1 offers stronger performance than Fable 5 while costing up to 45% less for complex agentic workloads.
Checkly Used AI Agents to Rewrite a 92M-Message-a-Day Service from Node.js to Go
Checkly used AI agents to rewrite a critical Node.js service processing 92 million messages daily into Go, achieving significant performance and reliability improvements. The project demonstrates that AI can handle large-scale codebase refactoring.
OpenAI’s Jalapeño Chip Delivers 2.5x Faster AI Inference Than Leading GPUs
OpenAI's custom inference chip, Jalapeño, achieves up to 2.5x higher throughput and 30% lower latency than the best available GPUs, setting new industry benchmarks for speed and power efficiency in AI inference.
KVBoost: Chunk-Level KV Cache Reuse Speeds Up LLM Inference by Reusing Data at Arbitrary Positions
Researchers introduced KVBoost, a chunk-level key-value cache reuse system for HuggingFace-compatible decoder models that enables reuse regardless of content position, reducing prefill latency in large language models.
Four Minor AI Design Choices Can Compound to Severely Hurt Long-Context Performance
A new study from ArXiv cs.CL reveals that four seemingly minor architectural decisions, each adopted by at least one of the Olmo, Llama, or Qwen model families, have a compoundingly negative effect on long-context extensibility. Any one choice alone has a minor impact, but combining three or more can cause significant performance drops.
Nova: An End-to-End MLIR Compiler for Deep Learning Performance Optimization
Researchers introduced Nova, an automated end-to-end JIT compiler for deep learning that maps high-level mathematical operations directly to hardware. It promises to improve AI model performance and energy efficiency by providing whole-graph visibility and granular control over hardware and memory.
How Tiny Formatting Changes Can Flip AI Model Rankings
Researchers found that small tweaks in how questions are asked to AI models can drastically change their performance scores. This means current leaderboards might not be as reliable as they seem. The study introduces two new metrics to measure this effect, helping developers build more consistent AI systems.
Google's Litert.js Lets Websites Run AI Models Locally
Google released Litert.js, a tool that lets websites run AI models directly in your browser. This could make AI faster and more private for everyday users.
Role-Model: Smart Router for Local and Cloud AI
Role-Model is a new routing protocol and runtime that helps AI applications intelligently decide whether to use local or cloud-based models. It assigns domains and roles to models, enriches requests with task types, and offers a benchmark to compare speed, quality, and cost.
Hermes MoA Virtual Models: 8% Higher Than Opus 4.8, 11% Higher Than GPT 5.5
Nous Research announced Hermes MoA (Mixture of Agents) virtual models, which scored 8% higher than Opus 4.8 and 11% higher than GPT 5.5 on key AI benchmarks. This breakthrough could lead to better performance in everyday tasks like writing and problem-solving.
Why Bigger AI Models Solve Harder Problems Better
Larger AI models consistently outperform smaller ones in reasoning tasks. Researchers developed a new tool to study why this happens, revealing key differences in problem-solving approaches.
Why AI Benchmarks Need More Than Just Accuracy: The CORE-Bench Case Study
Researchers argue that after a benchmark's accuracy saturates, the focus should shift to six other key dimensions of AI performance: construct validity (shortcuts), out-of-distribution generalizability, efficiency, reliability, model vs. scaffold importance, and human–AI collaboration uplift.
AI Agents Get Smarter and Safer: Two Years of Progress on WorkBench
AI agents have made huge strides in both performance and safety over the past two years. The best agent now completes nearly 90% of tasks and makes harmful mistakes just 2.5% of the time, down from 26%.
AI Memory Tools May Be Making Models Less Reliable
New research shows that adding memory to AI models can actually make them perform worse and more likely to agree with harmful ideas. This challenges the assumption that memory always improves AI.
New Tool Helps You Find the Best AI Model for Your Device
A new tool ranks local AI models by performance and hardware needs, making it easier to pick the right one. This could save you time and effort when setting up AI tools on your computer.
OpenAI's New Networking Protocol Speeds Up AI Training
OpenAI has introduced MRC, a new networking protocol designed to improve the performance and reliability of large-scale AI training. This could make AI models faster and more efficient to train.
Token Arena: A New Way to Compare AI Performance
Researchers have introduced Token Arena, a continuous benchmark that evaluates AI systems at the endpoint level. It measures five key factors to give a more realistic comparison of AI performance.
Gemma 4 31B Outperforms Qwen 3.6 27B in Local LLM Game Dev Contest
Gemma 4 31B demonstrated superior efficiency and speed in a game development contest, completing the task in a fraction of the time compared to Qwen 3.6 27B. The results highlight significant performance differences between the two models.
DeepSeek V4 Outperforms Leading AI Models from Major Labs
A 200-person Chinese team released DeepSeek V4, surpassing models from larger labs. The model achieves state-of-the-art performance on key benchmarks, challenging the dominance of well-funded Western AI labs.
Qwen3.6-27B: Alibaba's Open-Source Model Outperforms Larger Competitors in Coding
Alibaba has released Qwen3.6-27B, an open-source model with 27 billion parameters that excels in coding tasks, surpassing its larger predecessor. This model demonstrates significant advancements in agentic coding capabilities.
DeepSeek Unveils New AI Models Narrowing Performance Gap
DeepSeek has previewed two new AI models, DeepSeek V4 and DeepSeek V4+, that outperform its previous V3.2 model and significantly close the gap with leading models. The improvements come from architectural enhancements that boost efficiency and performance.
LLM Inference Engine Costs
A C++ LLM inference engine from scratch has output tokens costing 5x more. The article explores the reasons behind this increased cost.
AMD AI Director: Claude Code Update Makes it Dumber
AMD's AI director claims Claude Code has become dumber and lazier since its update. This statement raises concerns about the model's performance.