17 stories tagged Uncertainty
Researchers introduced Evidence Chain Evaluation (ECE), a selective fact-checking framework that allows AI models to abstain from verdicts when supporting evidence is weak, sparse, or inconsistent, improving reliability over forced true/false systems.
Researchers introduced Safe-Psych, a sequential benchmark that evaluates how well large language models handle evolving diagnostic uncertainty in clinical psychiatry. Unlike existing medical benchmarks that assume complete information upfront, Safe-Psych tests whether models request clarification or abstain when data is insufficient.
Researchers developed YUKTI, an AI system that handles uncertainty better by showing when its decisions might be wrong. This could help doctors, managers, and others make safer choices with real-world consequences.
Scientists developed a new method for AI to better distinguish between different types of uncertainty — whether it's missing information or just inherent randomness. This could make LLMs more reliable in real-world tasks where mistakes matter.
Researchers propose a new way for AI to understand and communicate uncertainty, which could help chatbots ask better questions. This could make AI assistants more helpful in everyday situations where information is unclear.
Researchers found that large language models (LLMs) often overestimate their confidence when analyzing medical data. The study suggests new methods to help AI recognize what it doesn't know, improving reliability in healthcare applications.
Researchers have identified a critical flaw in AI reasoning: models often reach the same answer through inconsistent paths. They propose a new way to measure this inconsistency to improve AI reliability.
Researchers developed a method called ACTION-RATING to help AI models ask for clarification when they're unsure. This could make AI assistants more reliable by reducing mistakes from guessing.
Researchers developed a way to train AI models to handle real-world problems with incomplete or ambiguous information. This could improve AI's ability to make practical decisions in everyday scenarios.
Researchers found that AI models fail in two distinct ways, leaving identifiable patterns in their reasoning. Understanding these patterns could help improve AI reliability in the future.
Researchers introduced a new framework called CHAL for AI debates. It focuses on areas where truth is uncertain, helping AI models refine their reasoning through structured discussions. This could make AI more useful in real-world, ambiguous situations.
Large language models (LLMs) often fail to adjust their responses based on how certain the information they retrieve is. This could have serious implications in fields like medicine and finance where accuracy is critical.
Researchers have developed a way to make AI models better at admitting when they don't know something. This could make AI assistants more reliable in everyday use. The method works without needing to see the model's internal workings, making it useful for commercial AI services.
Researchers created a new framework called CLEAR to test how well AI handles ambiguous medical questions. They found that AI models often give unreliable answers when faced with real-world uncertainties.
Researchers are studying new methods for AI to update its beliefs more flexibly. These methods could help AI systems adapt to new information more naturally, like humans do.
Researchers propose a new approach to sequential clinical diagnosis using uncertainty-guided latent diagnostic trajectory learning. This method addresses the challenge of learning effective diagnostic trajectories under uncertainty.
SELFDOUBT is a new framework for uncertainty quantification in reasoning language models. It addresses the difficulty of deploying uncertainty estimation in practice, particularly for proprietary APIs.