7 stories tagged Reproducibility
The UK's Artificial Intelligence Standards Institute (AISI) and EvalEval have partnered to standardize AI evaluations, making benchmark results reproducible and trustworthy for developers and researchers.
A new arXiv paper introduces a governed approach to enterprise analytics that combines language models with deterministic policies to ensure reliable, replayable results. Tested across 440 runs with three 8B models, the system interprets questions and runs pre-approved analytical programs.
Hugging Face reproduced 2,200 papers from ICML 2026 and found that only 60% had fully reproducible results. This massive effort aims to improve transparency and reliability in AI research by identifying which papers are trustworthy and which need more work.
Researchers introduced agrepl, a developer-first CLI framework that enables deterministic replay of AI agent executions by intercepting all external interactions, solving the non-determinism problem in LLM-based agent systems.
Researchers introduced GraphBit, a new framework that improves AI agents by using a structured approach called a directed acyclic graph (DAG). This makes AI workflows more reliable and reproducible, avoiding common issues like hallucinations and infinite loops.
Researchers propose an artifact-based agent framework to improve adaptability and reproducibility in medical image processing workflows. This approach addresses critical needs for real-world clinical deployment.
Researchers developed an AI system that can reproduce social science findings using only paper descriptions and raw data. The method achieves deterministic, cell-level result comparisons without access to original code or outputs.