11 stories tagged Risk
A new study from MIT and the University of Toronto shows that more advanced AI models can increase systemic risk in financial markets because they make similar decisions, creating dangerous correlations that cannot be diversified away.
A federal judge ruled the Trump administration illegally labeled Anthropic a supply-chain risk, marking a significant legal victory for the AI company. This decision comes as Anthropic continues to fight another Pentagon lawsuit in Washington.
Researchers propose a monotonic evaluation framework for wildfire risk systems that measures whether higher predicted risk scores consistently correspond to increased operational load — such as number of fires, intervention time, and deployed resources — rather than relying on traditional metrics like F1-score or IoU.
Researchers from ArXiv cs.CL have introduced a session-layer framework to detect Conversational Risk Accumulation (CRA) in multi-turn LLM systems. Unlike existing guardrails that evaluate each prompt-response in isolation, the CRA Framework tracks semantic drift, fragmented assembly of prohibited instructions, and sensitivity build-up over a dialogue. This could make future AI conversations significantly safer.
Researchers propose CPSAINT, a seven-layer integrity decomposition paired with the FRIESA-K risk functional, to quantify residual risk in agentic AI. The framework bridges the gap between describing failure mechanisms and producing transferable risk estimates, enabling safer deployment of autonomous AI agents.
In a surprising blog post, Microsoft CEO Satya Nadella warned enterprises about the dangers of relying on proprietary AI models from companies like Anthropic and OpenAI, urging businesses to maintain control over their AI systems to avoid vendor lock-in and unforeseen risks.
Researchers found that AI models can still make irrational decisions even when trained to align with human values. This 'rational value risk' means models may not always choose the best possible action, even if they understand what's valuable.
A provocative new opinion piece argues that the rapid pace of AI development poses greater existential risks than slowing down progress on medical breakthroughs like cancer cures. The author frames the debate as a stark trade-off between technological acceleration and societal safety.
Researchers have developed a machine-learned comorbidity index that better predicts patient outcomes. Unlike older methods, it captures complex relationships between diseases and health risks.
Researchers introduced a new AI safety framework called Risk-Aware Causal Gating (RACG) that helps AI models make safer decisions by evaluating potential risks. This approach could prevent costly errors in AI-driven systems by deciding when to act, defer, or abstain from actions.
Researchers propose NuHF Claw, a risk-constrained AI agent framework designed to support human operators in digital nuclear control rooms. The system aims to mitigate cognitive risks while maintaining human authority in safety-critical environments.