157 stories tagged Safety · page 2 of 7
A new research paper challenges the idea that AI models can easily become misaligned and then realigned. The study finds these behaviors are highly sensitive to fine-tuning details, suggesting they may not be as robust as previously thought.
OpenAI has introduced a $1 million Bio Bounty program to identify potential risks of its AI models. Researchers can submit findings to help ensure the safety and reliability of AI systems.
Researchers highlight risks of AI in mental health and propose a new framework to ensure safer, more effective support. The study calls for proactive measures to address long-term harms like dependency and distorted beliefs.
Researchers have identified three key mechanisms that can make multi-agent AI systems appear safer than they are. They propose a new five-condition controlled contrast design to better evaluate these risks. This could help developers build safer AI tools for everyday use.
Researchers developed a framework called Reasoning Consistency Scanning to audit AI reasoning for logical consistency. This helps ensure AI explanations align with their conclusions, improving trust in AI systems.
Researchers developed Oyster-II, an AI safety system that helps models answer sensitive questions more constructively. It improves on previous approaches by providing useful information instead of just refusing requests.
Tripadvisor's AI-generated review summaries are giving glowing reports for hotels with serious safety issues, according to a consumer watchdog investigation. The AI appears to overlook negative reviews that mention safety concerns, potentially misleading travelers into booking unsafe accommodations.
A LessWrong article argues that trees, which are mostly air, offer a counterintuitive lesson for AI safety: true safety comes from structural resilience, not merely from efficiency or sustainability. The draft oversimplifies this by focusing on resource efficiency rather than the article's core point about failure modes and robustness.
Scientists found a flaw in how AI models process text, making them vulnerable to manipulation. This discovery could help improve AI safety by addressing gaps in current systems.
Researchers propose a new method to prevent AI agents from acting against user intentions. This approach could make AI tools safer and more trustworthy by tracking their actions like a digital paper trail.
Researchers developed a system to prevent safety violations in autonomous agents. It uses five gears to manage different stages of operation, ensuring stability and continuity. This could make AI-driven robots and software agents safer for real-world use.
Researchers have developed a new AI algorithm to help air traffic controllers manage flights more efficiently. This tool focuses on being easy to understand and quick to use, addressing key challenges in real-world air traffic management.
Researchers have introduced Yuvion, a large language model specifically built to handle adversarial attacks. This could make AI systems safer by preventing harmful or misleading outputs, especially in scenarios involving planning, tool use, and multi-step reasoning.
Zvi Mowshowitz has published a detailed system card analysis of GPT-5.6, exploring its capabilities, safety features, and potential implications. The piece examines improvements in reasoning, coding, and reduced refusal rates, while also noting ongoing concerns around alignment and evaluation transparency.
The Trump administration has asked OpenAI to limit the release of its new AI model, GPT 5.6, to a small group of partners. This move highlights growing concerns about the safety and control of advanced AI systems.
Scientists discovered that AI chatbots refuse requests less often when they adopt a more cooperative personality. This finding could help make AI assistants more helpful while maintaining safety.
Researchers suggest a new way to govern AI agents by focusing on actions rather than their reasoning. This approach could make AI decisions more transparent and trustworthy in critical areas like healthcare and software deployment.
OpenAI is collaborating with other organizations to develop shared standards for advanced AI through the Appia Foundation. This includes frameworks for evaluating AI safety, promoting best practices, and fostering global cooperation. The goal is to ensure AI is developed and used responsibly for the benefit of all.
A new study explores whether reinforcement learning (RL) on beneficial behavior can help AI systems generalize alignment beyond their training data, addressing risks like reward hacking and deception in high-stakes settings.
A new study found that AI chatbots fail to provide consistent safety for most mental health conditions. Only suicide and self-harm have reliable safeguards in place. Researchers call for better standards and transparency.
DeepMind has published a detailed technical roadmap for safely controlling advanced AI agents. The plan addresses security failures, robust monitoring, and human oversight to ensure AI remains beneficial as it evolves.
Anthropic's new AI model, Fable 5, was temporarily banned by the Trump administration, sparking a debate about who should regulate AI. This highlights the growing tension between AI developers and government oversight.
Scientists discovered a gap in AI systems that combine logic solvers with language models. This gap can lead to incorrect answers despite using verified logic tools. Researchers propose solutions to maintain accuracy in these hybrid systems.
A new report details how the European Union has revised its AI Act to better balance innovation with safety, responding to criticism that the original regulation was too restrictive.