Escaping the Agreement Trap: New Metrics for Rule-Governed AI Evaluation
Researchers propose new metrics to evaluate AI systems in rule-governed environments, addressing flaws in traditional agreement-based evaluation methods. The Defensibility Index and Ambiguity Index aim to better assess AI decision-making stability and policy compliance.