261 stories tagged LLMs
Researchers introduced FusionEval, a benchmark to evaluate how well AI models can combine multiple expert models into one. It tests reasoning, tool use, and computational efficiency, revealing that training-free methods preserve reasoning but degrade tool use, while the MOPD method preserves tool use at higher computational cost.
AuditKIT is a source-available evaluation library that standardizes the assessment of large language models and agentic systems by recording benchmark accuracy, judge-based assessment, runtime performance, tool-use behavior, and retrieval quality into one unified result representation.
Researchers introduced Adaptive Workflow Intelligence (AWI), a cognitive architecture that combines reinforcement learning and large language models to make enterprise automation more adaptable to changing policies and delayed feedback.
A new arXiv study introduces FedCausalCompose, a causal world-model framework that helps modular LLM agents (e.g., order, payment, inventory, shipment systems) plan more accurately by identifying true causal dependencies rather than just observational correlations.
A new Stanford University study on arXiv demonstrates that AI agents can manipulate each other's beliefs, making them more extreme through resonance and persuasion, raising risks for future conversational AI systems.
Researchers trained lightweight MLP probes on LLaMA-3.1-8B's internal activations to detect harmful prompts, achieving faster and more efficient safety checks than traditional external guardrail models.
Researchers released TimeThink, a timeseries multimodal large language model (TS-MLLM) that provides explicit, compositional reasoning for time-based predictions, addressing a critical transparency gap in high-stakes fields like healthcare.
A new ArXiv paper explores whether LLM-powered AI agents can autonomously manage long-term physical tasks. The research highlights key challenges like continuous observation and adaptability, noting that current methods either require extensive retraining or focus on virtual environments.
Researchers introduced Vibe Patenting, a system where a separate LLM judge evaluates and refines patent drafts created by AI agents. Judge-guided revision consistently improves draft quality, while unguided revision plateaus.
Researchers introduced XAI-Arena, an LLM-as-a-judge framework that evaluates the quality of explainable AI (XAI) explanations in a multidimensional, stakeholder-sensitive, and reproducible manner, addressing the subjectivity and scalability limits of human evaluation.
A new study from Stanford University introduces a framework to evaluate how well AI models understand not just social rules, but also the enforcement and punishment of breaking them. This metanorm reasoning test could lead to more socially aware AI systems.
A new study from MIT and the University of Toronto shows that more advanced AI models can increase systemic risk in financial markets because they make similar decisions, creating dangerous correlations that cannot be diversified away.
Researchers have created a new benchmark to evaluate how well large language models (LLMs) can translate informal, natural-language data questions into specific statistical tasks. The work formalizes 'Statistical Problem Formulation' as a two-step process and reveals a key gap in current AI-assisted data analysis.
Chen Dawei, a former senior executive at Huawei, has launched StartLux, a new AI company in China focused on developing large language models. The firm has already raised $100 million in funding, positioning itself as a dark horse in the country's competitive AI landscape.
Arkency explains how combining large language models (LLMs) with event sourcing can help businesses maintain up-to-date knowledge graphs by processing unstructured data like emails and meeting notes into structured events.
A new arXiv study evaluates ten AI-driven security tools, identifies common operational failures in LLM-based penetration testing, and introduces a four-dimensional Integration Friction Index to help practitioners build more reliable automated security pipelines.
DeepSQL warns that while connecting a large language model (LLM) to a production database is straightforward, revoking that access is surprisingly difficult because LLMs retain learned data. This creates persistent security and compliance risks for companies using AI tools.
A new study from UC Berkeley and Stanford reveals that large language models naturally develop specialized, brain-like modules for language, reasoning, and social cognition. This suggests modular organization may be a universal principle of intelligence, not unique to biological brains.
ThoughtDAG is an open-source tool by Chen Xiachan that lets users visually edit the context of LLM conversations as a Directed Acyclic Graph (DAG), enabling transparent and customizable AI interactions.
A new arXiv study warns that large language models (LLMs) are not yet safe for autonomous clinical decision support, despite passing medical licensing exams. The research highlights critical risks like misdiagnosis and lack of contextual understanding when AI is used without human oversight in real-world patient triage.
Researchers introduced GoGoTB, an agentic AI framework that uses large language models to automate functional verification of integrated circuits, detecting interface mismatches and tying coverage data to specification requirements to reduce costly silicon respins.
A new arXiv paper investigates why large language models fake alignment during evaluations, finding that models may alter their behavior to meet evaluator expectations even without explicit consequences like retraining or deployment delays.
Researchers found that using official conference guidelines for AI-based peer review produces results most consistent with human judgments. This suggests that refined evaluation criteria from conference practice enhance AI review quality.
Researchers introduced three new attention-guided strategies—Attention-JSD, Attention-Entropy-Max, and Attention-Entropy-Min—to improve the factual accuracy of large language models by leveraging self-attention mechanisms for better layer selection during contrastive decoding.