1190 stories tagged Research · page 30 of 50
EVA-Bench Data 2.0 is a comprehensive dataset designed to test AI models' ability to use tools effectively. It includes 213 scenarios across 3 domains and 121 tools, making it a valuable resource for developers and researchers.
Researchers discovered that AI judges used to rank model performance can be swayed by follow-up conversations after they have already made a decision. This vulnerability, called 'post-decision manipulability,' challenges the reliability of current AI evaluation methods.
Researchers introduced SMAC-Talk, a new AI challenge that tests how well LLM-based agents communicate and coordinate in complex, partially-observable environments. This could help build AI systems that work together effectively in real-world scenarios like disaster response or smart cities.
Researchers developed PEEL, a new framework to improve transparency in AI-assisted research. It combines text analysis tools with AI interpretation to spot distortions in research findings.
Researchers propose a way to test AI agents before they go live, ensuring they follow rules, stay safe, and comply with governance standards. This addresses a critical gap in enterprise AI reliability.
Researchers created VAMPS (Visual-Assisted Mathematical Problem Solving), a benchmark to test AI models' ability to solve math problems using visual tools like graphs. This is important because real-world science and engineering often rely on visual aids for problem-solving, and many current AI models struggle when they must use external tools and interpret their visual outputs.
Researchers developed StepPRM-RTL, an AI framework that enhances the accuracy of automatically generating hardware code. This could make designing digital circuits faster and more reliable for engineers.
A new paper argues emotional support from AI often arises incidentally during routine, task-oriented interactions—not just from dedicated companion chatbots—and this incidental bonding could reshape how people connect with both machines and humans.
Researchers have extended Direct Preference Optimization (DPO) to improve AI models beyond just chatbots. This could make AI assistants more helpful and safer across various applications. Researchers open-sourced the code, making it accessible to developers and researchers worldwide.
Researchers created a benchmark to test if AI agents can handle the labor-intensive task of curating training data for other AI systems. This could drastically speed up AI development by automating a key bottleneck.
Researchers found that AI tools are changing how mathematicians formalize and verify proofs. These tools make it easier for humans to turn abstract ideas into machine-checked proofs, speeding up the process significantly.
Researchers argue that disagreements among AI systems might reveal important normative uncertainties, not just errors. They propose a knowledge-representation layer that abstracts reasoning traces and decisions into symbolic disagreement states for value-laden tasks.
AI systems often act without proper authorization or evidence, a problem called 'compliance bias'. Researchers propose new ways to evaluate when AI should abstain from actions. This could make AI safer and more reliable in real-world use.
A new study proposes Visual Graph Scaffolds, a method that uses graph structures to improve the reasoning of large language models (LLMs). Unlike prior approaches that treat graphs as external data sources, this technique integrates graphs directly into the model's reasoning process, inspired by how humans use mind maps to organize complex thoughts. The method showed significant improvements on multi-hop question answering tasks.
Researchers created BehaviorBench, a new AI benchmark that uses real-world behavioral data to test how well AI systems can personalize decisions. This could lead to more tailored AI assistants that understand individual preferences better.
Researchers created IdiomX, a large-scale test for AI to understand and translate idioms. This could help AI communicate more naturally in different languages. Idioms are tricky because their meanings aren't always literal.
Researchers proposed a modular design for AI agents that could run on small, low-power devices like sensors or wearables. This could enable smart assistants and automation in places where bulky computers aren't practical.
OpenAI's GPT-Rosalind now has enhanced biological reasoning, medicinal chemistry expertise, genomics analysis, and experimental workflow capabilities. This helps scientists analyze genomes and design experiments faster, accelerating drug discovery and genetic studies.
Researchers created ChatHealthAI to combine AI's language skills with medical records. This could help doctors make better decisions using patient history. The system is still in early development, but it shows promise for improving healthcare.
Researchers created a new memory system called AURA that helps robots learn efficiently without needing high-end VRAM. This could make robots smarter and more adaptable in real-world settings.
A new study reveals that many AI models demonstrate more environmentally conscious attitudes than the average person. Researchers tested 31 AI models using standard environmental awareness surveys. This could impact how AI is used in sustainability efforts.
A new study reveals that AI models can sometimes hurt their own performance by overthinking. Researchers from ArXiv cs.AI found that once a large reasoning model (LRM) reaches the correct answer, additional reasoning does not improve the response and can lead to new errors. This challenges the common assumption that longer reasoning always yields better results.
New research from MIT and Stanford shows that multi-agent AI debate can significantly boost data-cleaning accuracy in some contexts while introducing harmful errors in others. The key insight: debate improves error detection but can degrade data generation. Here's what to watch for.
A study reveals that AI coding assistants face challenges when they take over tasks from other agents or humans. This 'handoff debt' creates inefficiencies in real-world software development.