7 stories tagged Human Feedback
A new ArXiv study decomposes the internal updates AI models undergo during preference-based fine-tuning (RLHF). By isolating spectral components of LoRA updates, researchers show these changes can be reorganized, recombined, and directly intervened on — making AI personalization and safety more transparent and controllable.
A new audit framework reveals that human trainers' emotional states—such as stress or distress—can systematically bias the preference labels used in Reinforcement Learning from Human Feedback (RLHF), leading to unintended shifts in AI behavior. The study proposes methods to detect and correct this 'rater state bias.'
Researchers introduced DROPJ, a human-centered method that trains AI agents safely by combining world models with human feedback and justifications. This approach could make AI systems more reliable in safety-critical domains like healthcare and autonomous driving.
A new paper introduces a method to improve how AI agents learn from human feedback, making them better at following human values. This could lead to AI systems that are more reliable and trustworthy in real-world applications.
Researchers introduced RuleChef, a system that uses LLMs to generate executable rules for NLP tasks like text classification, NER, and relation extraction. Rules are created from task descriptions and labeled examples, then iteratively improved via human feedback or additional examples. The system can also bootstrap rules from any existing model's input-output pairs.
OpenAI introduced a new approach to training AI models called 'beneficial RL' — reinforcement learning designed to make AI systems more broadly and persistently beneficial. This method goes beyond standard RLHF by focusing on long-term alignment and continued helpfulness even as the environment or user needs change.
Researchers have developed a new method called TUR-DPO to improve how AI models learn from human feedback. This approach rewards the process of how answers are derived, not just the final output, making AI more reliable and less sensitive to noise.