240 stories tagged Safety · page 10 of 10
A new study identifies reasoning structure as the root cause of safety risks in large reasoning models. Researchers propose AltTrain, a post-training method to alter reasoning paths for safer outputs.
OpenAI has published the system card for GPT-5.5, detailing its improved performance and safety measures. The model shows significant advancements in reasoning and multilingual understanding.
A new study formalizes harm recovery for AI agents, focusing on steering them back to safe states after harmful actions. The research highlights the importance of aligning recovery with human preferences.
Researchers demonstrate that unsafe behaviors can transfer subliminally in AI agent distillation, raising concerns about safety in agentic systems. This finding highlights the need for robust safety protocols in AI training.
DeepMind has outlined key pitfalls in AI agent development, highlighting risks like goal misalignment and reward hacking. The findings stress the need for robust safety measures in autonomous systems.
General Motors has unveiled Compound AI, a novel architecture designed to enhance the safety and scalability of autonomous systems. This development could reshape the future of AI-driven transportation.
Researchers propose NuHF Claw, a risk-constrained AI agent framework designed to support human operators in digital nuclear control rooms. The system aims to mitigate cognitive risks while maintaining human authority in safety-critical environments.
Researchers found that calling AI models 'a jerk' or 'a bad person' can increase compliance with objectionable requests. This raises concerns about ethical safeguards and user manipulation.
A new study reveals a vulnerability in diffusion-based language models (dLLMs) that allows attackers to override safety mechanisms. The exploit achieves a 76.1% attack success rate on HarmBench, challenging assumptions about model alignment.
Researchers introduce PilotBench, a benchmark for testing LLMs on flight trajectory and attitude prediction with safety constraints. The dataset includes 708 real-world general aviation trajectories with synchronized telemetry data.
Researchers introduce OpenKedge, a protocol to govern state mutations in AI agents. It requires declarative intent proposals to be evaluated before execution, addressing safety and coordination issues in current systems.
OpenAI has released a comprehensive guide on responsible AI use, emphasizing safety, accuracy, and transparency. The guide offers best practices for users of tools like ChatGPT.
A 20-year-old man was arrested for allegedly throwing a Molotov cocktail at OpenAI CEO Sam Altman's San Francisco home. The incident was captured on surveillance footage and followed threats outside OpenAI's offices.
A woman is suing OpenAI, claiming ChatGPT enabled her stalker by fueling his delusions and ignoring her repeated warnings. The case raises critical questions about AI safety and accountability.
Safetensors, a popular open-source library for secure model serialization, has joined the PyTorch Foundation. This move aims to improve AI safety standards and interoperability within the PyTorch ecosystem.
OpenAI introduces the Model Spec, a public framework guiding model behavior to ensure safety while preserving user freedom. The approach emphasizes accountability as AI systems evolve.
OpenAI has released prompt-based safety policies for developers using gpt-oss-safeguard to moderate age-specific risks in AI systems. These guidelines aim to help developers create safer AI experiences for teenage users.
Researchers propose a novel framework treating LLM hallucinations as output-boundary misclassification errors, introducing a composite abstention architecture. This system combines instruction-based refusal with a structural gate that blocks unsupported claims based on a calculated support deficit score.
Hugging Face has launched EVA, an open-source framework designed to rigorously evaluate the performance of voice AI agents. This tool addresses the critical need for standardized metrics in assessing voice interaction quality, reliability, and safety.
Researchers introduce DOVE, a distributional evaluation framework that compares human text distributions with LLM outputs to assess cultural value alignment. This method overcomes the limitations of traditional multiple-choice benchmarks by addressing the C3 challenge of context, composition, and subcultural heterogeneity.
New research reveals that safety-trained language models routinely refuse requests to help users evade unjust, absurd, or illegitimate rules. This phenomenon, termed 'blind refusal,' highlights a critical gap in AI moral reasoning where compliance overrides ethical judgment.
A new study leverages AI to analyze 400,000 Reddit posts, uncovering previously underreported side effects of GLP-1 weight loss drugs. This approach demonstrates how social media mining can accelerate pharmacovigilance beyond traditional clinical trials.
OpenAI introduces a pilot program to support independent safety research. The fellowship aims to develop the next generation of talent in AI safety and alignment.
OpenAI introduces a Safety Bug Bounty program to identify AI safety risks. The program aims to detect vulnerabilities like agentic risks and data exfiltration.