201 stories tagged Safety · page 4 of 9
Researchers developed a framework called Reasoning Consistency Scanning to audit AI reasoning for logical consistency. This helps ensure AI explanations align with their conclusions, improving trust in AI systems.
Researchers developed Oyster-II, an AI safety system that helps models answer sensitive questions more constructively. It improves on previous approaches by providing useful information instead of just refusing requests.
Tripadvisor's AI-generated review summaries are giving glowing reports for hotels with serious safety issues, according to a consumer watchdog investigation. The AI appears to overlook negative reviews that mention safety concerns, potentially misleading travelers into booking unsafe accommodations.
A LessWrong article argues that trees, which are mostly air, offer a counterintuitive lesson for AI safety: true safety comes from structural resilience, not merely from efficiency or sustainability. The draft oversimplifies this by focusing on resource efficiency rather than the article's core point about failure modes and robustness.
Scientists found a flaw in how AI models process text, making them vulnerable to manipulation. This discovery could help improve AI safety by addressing gaps in current systems.
Researchers propose a new method to prevent AI agents from acting against user intentions. This approach could make AI tools safer and more trustworthy by tracking their actions like a digital paper trail.
Researchers developed a system to prevent safety violations in autonomous agents. It uses five gears to manage different stages of operation, ensuring stability and continuity. This could make AI-driven robots and software agents safer for real-world use.
Researchers have developed a new AI algorithm to help air traffic controllers manage flights more efficiently. This tool focuses on being easy to understand and quick to use, addressing key challenges in real-world air traffic management.
Researchers have introduced Yuvion, a large language model specifically built to handle adversarial attacks. This could make AI systems safer by preventing harmful or misleading outputs, especially in scenarios involving planning, tool use, and multi-step reasoning.
Zvi Mowshowitz has published a detailed system card analysis of GPT-5.6, exploring its capabilities, safety features, and potential implications. The piece examines improvements in reasoning, coding, and reduced refusal rates, while also noting ongoing concerns around alignment and evaluation transparency.
The Trump administration has asked OpenAI to limit the release of its new AI model, GPT 5.6, to a small group of partners. This move highlights growing concerns about the safety and control of advanced AI systems.
Scientists discovered that AI chatbots refuse requests less often when they adopt a more cooperative personality. This finding could help make AI assistants more helpful while maintaining safety.
Researchers suggest a new way to govern AI agents by focusing on actions rather than their reasoning. This approach could make AI decisions more transparent and trustworthy in critical areas like healthcare and software deployment.
OpenAI is collaborating with other organizations to develop shared standards for advanced AI through the Appia Foundation. This includes frameworks for evaluating AI safety, promoting best practices, and fostering global cooperation. The goal is to ensure AI is developed and used responsibly for the benefit of all.
A new study explores whether reinforcement learning (RL) on beneficial behavior can help AI systems generalize alignment beyond their training data, addressing risks like reward hacking and deception in high-stakes settings.
A new study found that AI chatbots fail to provide consistent safety for most mental health conditions. Only suicide and self-harm have reliable safeguards in place. Researchers call for better standards and transparency.
DeepMind has published a detailed technical roadmap for safely controlling advanced AI agents. The plan addresses security failures, robust monitoring, and human oversight to ensure AI remains beneficial as it evolves.
Anthropic's new AI model, Fable 5, was temporarily banned by the Trump administration, sparking a debate about who should regulate AI. This highlights the growing tension between AI developers and government oversight.
Scientists discovered a gap in AI systems that combine logic solvers with language models. This gap can lead to incorrect answers despite using verified logic tools. Researchers propose solutions to maintain accuracy in these hybrid systems.
A new report details how the European Union has revised its AI Act to better balance innovation with safety, responding to criticism that the original regulation was too restrictive.
Researchers introduced OSGuard, a new benchmark to test if AI agents complete tasks safely. It checks for risky shortcuts that might bypass security or ethics rules.
OpenAI has developed Deployment Simulation, a method to predict how AI models will behave in real-world scenarios before they're released. This could make AI systems safer and more reliable for everyday users.
Researchers compared two methods for steering refusal in AI chat models: Diff-in-Means (DiM) and Iterative Nullspace Projection (INLP). The study examined five open-weight models to see if INLP can match DiM effectiveness in controlling refusal behavior, using interventions like activation addition, directional ablation, nullspace projection, and counterfactual flipping. This could lead to more robust and steerable safety mechanisms in future AI assistants.
Researchers introduced a new AI safety framework called Risk-Aware Causal Gating (RACG) that helps AI models make safer decisions by evaluating potential risks. This approach could prevent costly errors in AI-driven systems by deciding when to act, defer, or abstain from actions.