1190 stories tagged Research · page 9 of 50
A new arXiv study warns that large language models (LLMs) are not yet safe for autonomous clinical decision support, despite passing medical licensing exams. The research highlights critical risks like misdiagnosis and lack of contextual understanding when AI is used without human oversight in real-world patient triage.
Researchers developed an AI system to evaluate AI-written scientific papers, testing it on four leading AI scientist frameworks. The system assesses originality, rigor, clarity, and significance, offering a new way to compare AI research quality.
OpenAI has made significant breakthroughs in solving long-standing problems in mathematics and theoretical computer science. These advances include new solutions in geometry, cryptography, and complexity theory.
Researchers introduced SkillSmith, an AI system that merges text-based knowledge and parametric (weight-space) skills to solve complex problems. This integrated approach could make AI agents more versatile and efficient in real-world tasks.
A systematic evaluation of 41 open-weight language models for zero-shot intent classification provides developers with practical guidance on model selection under compute, latency, and robustness constraints. The study tests models from 135 million to 9 billion parameters across 15 families on eight English datasets.
Researchers introduced a benchmark to evaluate how well large language models (LLMs) perform logical inference over natural-language probability expressions like 'likely' and 'unlikely', aiming to distinguish genuine reasoning from pattern matching.
Researchers introduced MultivationBench, a benchmark that evaluates how well multimodal AI models understand character motivations across sequential visual narratives, addressing a gap in existing static-text or single-image evaluations.
Researchers introduced LayerRAG-Bench, a cross-layer reliability benchmark for agentic retrieval-augmented generation systems. It tests nine models from OpenAI, Anthropic, and Gemini across 8 enterprise domains, 240 tasks, and 9 fault scenarios, finding that schema normalization raises schema-drift success from 0.000 to 0.913.
A new study from MIT and Stanford published in Nature reveals that large language models often produce logically correct answers by mimicking surface-level patterns rather than demonstrating genuine reasoning, raising reliability concerns for critical applications in healthcare, finance, and law.
Researchers introduced HSS-Synth, the first data synthesis pipeline specifically designed for humanities and social sciences (HSS). It covers 14 mainstream fields and aims to generate high-quality, diverse training data for large language models (LLMs) in open-ended domains.
Researchers introduced GoGoTB, an agentic AI framework that uses large language models to automate functional verification of integrated circuits, detecting interface mismatches and tying coverage data to specification requirements to reduce costly silicon respins.
Researchers introduce CaM-Wolf, the first AI agent for social deduction games that integrates both text and visual perception and generation, enabling more human-like social interaction in games like Werewolf.
Researchers introduced BridgeAlign, a new preference-alignment pipeline designed specifically for humanities and social sciences (HSS). Unlike existing methods that focus on objective correctness, BridgeAlign handles nuanced, subjective judgments, making it a first-of-its-kind approach for broad HSS disciplines.
The Bioinformatics Open Source Conference (BOSC) experimented with AI to pre-review abstracts, reducing the workload on human reviewers. This approach could help other conferences manage the surge in submissions enabled by generative AI.
Researchers introduced B1ade, a lightweight AI system that combines a small embedding model and a compact language model. It achieves top performance in retrieval tasks without needing massive training data or expensive hardware.
Researchers introduced AI SciBrief, a system that uses multi-stage prompt chaining to generate high-quality scholarly digests. This method outperforms traditional single-shot prompting in accuracy and reliability.
A University of Cambridge study found that seven large language models (LLMs) consistently failed to match the emotional perception of 3,011 human participants when evaluating sympathy in news headlines, with significant gaps across sociodemographic groups.
Researchers argue that AI evaluation scores should be treated as temporary knowledge claims, not absolute truths. They warn that combining different evaluation methods can create an illusion of higher accuracy than actually exists.
Researchers introduced AHA-Memes, the first large-scale Arabic hateful meme benchmark with fine-grained, multi-label annotations. This dataset enables more precise detection of hate speech in Arabic memes, addressing a critical gap in online safety for Arabic-speaking communities.
A new arXiv study finds that AI models trained with reinforcement learning (RL) develop superior internal representations for mathematical reasoning compared to supervised fine-tuning (SFT) models, explaining their better performance on math problems.
A new ArXiv paper identifies a fundamental epistemic problem in AI evaluation: benchmark inferences do not compose. Even when each link in a chain of reasoning is warranted, the chain itself may not hold, meaning strong benchmark performance does not guarantee real-world reliability.
Researchers introduced TraceCoder, an AI coding tool that records every step of code generation. It makes AI-generated code fully transparent and auditable, addressing key limitations of current systems.
Researchers introduced evidence-ledger adjudication, a workflow that pairs AI-generated claims with evidence packets, assigns support relations, and routes unsupported or contradicted claims back to the author for review. The system was tested on a 2,335-row blind benchmark built from AVeriTeC, CLIMATE-FEVER, and SciFact.
Researchers developed GuideSkill, an AI system that translates clinical practice guidelines into executable functions to improve diagnostic accuracy. The system can refine its skills over time using real case data, offering a more precise approach to medical decision-making.