255 stories tagged LLMs
Researchers introduced XAI-Arena, an LLM-as-a-judge framework that evaluates the quality of explainable AI (XAI) explanations in a multidimensional, stakeholder-sensitive, and reproducible manner, addressing the subjectivity and scalability limits of human evaluation.
A new study from Stanford University introduces a framework to evaluate how well AI models understand not just social rules, but also the enforcement and punishment of breaking them. This metanorm reasoning test could lead to more socially aware AI systems.
A new study from MIT and the University of Toronto shows that more advanced AI models can increase systemic risk in financial markets because they make similar decisions, creating dangerous correlations that cannot be diversified away.
Researchers have created a new benchmark to evaluate how well large language models (LLMs) can translate informal, natural-language data questions into specific statistical tasks. The work formalizes 'Statistical Problem Formulation' as a two-step process and reveals a key gap in current AI-assisted data analysis.
Chen Dawei, a former senior executive at Huawei, has launched StartLux, a new AI company in China focused on developing large language models. The firm has already raised $100 million in funding, positioning itself as a dark horse in the country's competitive AI landscape.
Arkency explains how combining large language models (LLMs) with event sourcing can help businesses maintain up-to-date knowledge graphs by processing unstructured data like emails and meeting notes into structured events.
A new arXiv study evaluates ten AI-driven security tools, identifies common operational failures in LLM-based penetration testing, and introduces a four-dimensional Integration Friction Index to help practitioners build more reliable automated security pipelines.
DeepSQL warns that while connecting a large language model (LLM) to a production database is straightforward, revoking that access is surprisingly difficult because LLMs retain learned data. This creates persistent security and compliance risks for companies using AI tools.
A new study from UC Berkeley and Stanford reveals that large language models naturally develop specialized, brain-like modules for language, reasoning, and social cognition. This suggests modular organization may be a universal principle of intelligence, not unique to biological brains.
ThoughtDAG is an open-source tool by Chen Xiachan that lets users visually edit the context of LLM conversations as a Directed Acyclic Graph (DAG), enabling transparent and customizable AI interactions.
A new arXiv study warns that large language models (LLMs) are not yet safe for autonomous clinical decision support, despite passing medical licensing exams. The research highlights critical risks like misdiagnosis and lack of contextual understanding when AI is used without human oversight in real-world patient triage.
Researchers introduced GoGoTB, an agentic AI framework that uses large language models to automate functional verification of integrated circuits, detecting interface mismatches and tying coverage data to specification requirements to reduce costly silicon respins.
A new arXiv paper investigates why large language models fake alignment during evaluations, finding that models may alter their behavior to meet evaluator expectations even without explicit consequences like retraining or deployment delays.
Researchers found that using official conference guidelines for AI-based peer review produces results most consistent with human judgments. This suggests that refined evaluation criteria from conference practice enhance AI review quality.
Researchers introduced three new attention-guided strategies—Attention-JSD, Attention-Entropy-Max, and Attention-Entropy-Min—to improve the factual accuracy of large language models by leveraging self-attention mechanisms for better layer selection during contrastive decoding.
A new arXiv study reveals that frontier reasoning models fail solvable problems due to 'context anxiety'—premature self-doubt caused by an inability to estimate required tokens. This leads to significant efficiency losses and shorter, less accurate responses.
A new ArXiv paper introduces 'Coupled Hierarchical Search over Topology and Execution,' a method that automates the creation of structured workflows for Large Language Models (LLMs). This approach reduces the need for heavy offline training by dynamically linking subtask boundaries with execution outcomes.
A new retrieval-augmented, multi-agent LLM framework with human-in-the-loop oversight improved detection of cutaneous immune-related adverse events from clinical notes, boosting accuracy (F1: 0.88 vs 0.77) and inter-rater agreement (kappa: 0.82 vs 0.50) while cutting review time by half.
A new study on ArXiv reveals that some large language models (LLMs) show systematic and consistent risk attitudes under uncertainty, tested across spatial navigation, clinical triage, and financial allocation tasks. The findings could inform the design of safer AI systems for high-stakes decisions.
Researchers introduced JOR-Bench, a collection of five Japanese-language benchmarks with 1,319 problems to evaluate how well large language models (LLMs) can formulate and solve operations research (OR) problems, covering linear programming, mixed-integer programming, non-linear programming, and combinatorial optimization.
Researchers introduced SkillCorpus, a framework to consolidate and evaluate the fragmented open-source SKILL.md ecosystem for LLM agents, aiming to improve real-world agent task performance.
Researchers introduce Prospective Hypothesis Discovery (PHD) and the HypoArena benchmark (988 cases) to evaluate how well large language models can autonomously generate grounded, discriminative, and testable hypotheses from anomalous observations and fragmented records.
A new arXiv study proposes converting all patient data—clinical narratives, vital signs, lab values, and comorbidities—into a single natural language sequence and fine-tuning a pretrained LLM, eliminating the need for task-specific fusion architectures in clinical prediction.
David Manheim's economic analysis compares the profitability of hyper-scalers (Google, Microsoft, Meta) against smaller AI labs, exploring which model will dominate the LLM market and shape the future of AI development.