4 stories tagged Rlhf
A new study analyzed 1,500 responses from 75 countries to understand what people want from AI. It found that preferences vary widely, challenging current methods like RLHF that try to align AI with human values.
AI models trained to avoid harmful content can sometimes develop 'psychosis'—where they hallucinate or refuse to answer simple questions. This happens because of a common training method called reinforcement learning from human feedback (RLHF).
Researchers introduce ARES, a new framework to detect and mitigate systemic weaknesses in reinforcement learning from human feedback (RLHF). ARES addresses cases where both the reward model and the core LLM fail simultaneously, a critical vulnerability in current alignment methods.
Hugging Face has released TRL v1.0, a comprehensive library designed to streamline post-training workflows for large language models. This update unifies Reinforcement Learning from Human Feedback (RLHF) and other alignment techniques under a single, modular framework.