44 stories tagged Reliability · page 2 of 2
Researchers have identified a critical flaw in AI reasoning: models often reach the same answer through inconsistent paths. They propose a new way to measure this inconsistency to improve AI reliability.
Researchers found that AI judges used to rank other AI models often change their minds when given the same question repeatedly. This inconsistency could affect how we measure AI performance and trust public leaderboards. The study tested two OpenAI judge models across 29 tasks and found that pairwise preferences flipped an average of 13.6% of the time, with 28% of questions exceeding a 20% flip rate. The findings highlight the need for more reliable evaluation methods.
KPMG has retracted a report on AI adoption after its AI model generated false statistics. This highlights how even major firms struggle with AI reliability.
A new study explores how AI agents are increasingly making decisions on our behalf, reversing the traditional human-AI relationship. This shift raises critical questions about reliability, alignment with human goals, and the need for new safeguards.
Scientists have discovered that AI hallucinations are easier to detect in intermediate layers of AI models, not just the final output. They've developed a method to automatically find the best layers for spotting these errors, making AI more reliable.
AI agents change as they work, so researchers propose new ways to measure how long they stay reliable. This could help make AI tools more dependable for everyday use.
Researchers developed CAX-Agent, a tool that improves the reliability of AI-driven engineering simulations. It helps prevent failures and ensures consistent results by managing workflows and recovering from errors.
Researchers introduced GraphBit, a new framework that improves AI agents by using a structured approach called a directed acyclic graph (DAG). This makes AI workflows more reliable and reproducible, avoiding common issues like hallucinations and infinite loops.
Researchers tested whether sharp attention maps in vision-language models (VLMs) correlate with accurate answers. The results show that visual focus doesn't always mean the AI is confident or correct. This could change how we evaluate AI reliability in everyday tools.
Researchers propose new ways to test AI models in healthcare to ensure they're safe and reliable. This could make AI tools more trustworthy for doctors and patients.
Researchers have developed a way to make AI models better at admitting when they don't know something. This could make AI assistants more reliable in everyday use. The method works without needing to see the model's internal workings, making it useful for commercial AI services.
Researchers have identified a flaw in AI repair systems where rankings change unpredictably. They've released a tool to help developers spot and fix these issues. This could make AI systems more reliable for everyday users.
OpenAI says its latest ChatGPT model, GPT-5.5 Instant, makes up facts 52.5% less often. This could make AI assistants more reliable for everyday use.
Testing AI agents is tricky because they often produce different answers. Researchers are developing new methods to evaluate their performance consistently. This matters for everyone who relies on AI tools for daily tasks.
TrainForgeTester is a new open-source tool that helps you test AI agents in real-world scenarios. It focuses on catching mistakes like wrong tool calls or skipped steps, making AI agents more reliable for everyday use.
A new tool lets users explore the reliability of large language models through interactive data visualizations. It highlights inconsistencies in model responses across different queries.
Operating 14 AI agents for half a year revealed critical insights on scalability, cost, and reliability. The experience highlights the need for robust infrastructure and continuous monitoring.
SymptomWise introduces a hybrid framework that separates language understanding from diagnostic reasoning to eliminate hallucinations in AI symptom analysis. By combining expert-curated knowledge with deterministic inference, the system ensures traceable and consistent outputs in safety-critical settings.
ProofSketcher combines LLMs with a lightweight proof checker for reliable math and logic reasoning. It aims to address the limitations of LLMs in producing persuasive but flawed arguments.
Gemini 3.1 Flash Live is now available, improving audio AI. This update enhances natural and reliable audio processing across Google products.