10 stories tagged Interpretability
Researchers introduced Concept-Targeted Attribution (CTA), a method that traces how AI models recognize concepts internally, even when those concepts aren't expressed in the model's output. The framework uses Cross-Layer Transcoders to build probe-specific circuits that explain why concept representations arise.
A new arXiv paper demonstrates that language models can be trained to be inherently interpretable, explaining their reasoning as they learn, and that this interpretability scales with model capability rather than acting as a trade-off.
Scientists have found a way to make AI reasoning more transparent by treating it like a moving object in space. This could help us understand how AI makes decisions in complex tasks.
Researchers have developed a new method to uncover the underlying causes of AI decisions in cyber-physical systems. This approach provides more robust insights, helping users understand automated decisions, especially in high-risk domains.
Neuronpedia is an open-source platform that makes AI models more transparent by visualizing their inner workings. It helps researchers and developers understand how AI systems make decisions.
Researchers have developed ANDRE, a new AI system that extracts logical rules from data more effectively than previous methods. This could make AI systems more interpretable and reliable in real-world, uncertain situations.
A new arXiv paper introduces perturbation-based attribution analysis to study how different fine-tuning strategies impact LLMs' interpretive behaviors for automated code compliance. The research highlights significant differences across full fine-tuning, LoRA, and quantized LoRA methods.
A new position paper challenges the prevailing view of LLM reasoning as a chain of thought, proposing instead that it should be studied as latent-state trajectory formation. This shift could redefine how we evaluate and interpret AI reasoning.
Researchers introduce ADAG, a new pipeline that automatically describes attribution graphs in language models, eliminating the need for manual circuit tracing. This shift promises to scale interpretability research by replacing ad-hoc human inspection with automated analysis.
Large reasoning models perform well on multi-step tasks but have unstable behavior. Step-Saliency analysis reveals information-flow failures.