Tommaso Dolci, Dina Fettache, Vittorio La Ferla, Amanda Le, Sihem Amer-Yahia, Jasmina Bogojeska, Katja Hose, Noha Ibrahim, Silviu Maniu, Katya Mirylenka, Maxime Peyrard
Preprint submitted as deliverable of the ARMADA project, 2026.
Large language models (LLMs) achieve strong performance across a number of generative AI tasks, yet their internal decision-making remains largely opaque, limiting trust and safe deployment in high-stakes domains such as healthcare, finance, and law. While Explainable AI (XAI) has produced diverse methods for interpreting traditional AI models, the scale and emergent behaviors of LLMs pose a challenge to conventional explainability approaches. Moreover, explanations are increasingly being exploited not only for improving human understanding, but also for enhancing model performance and reliability. This survey re-examines explainability methods for LLMs through the lens of interpretability of the model and model improvement, arguing that the field is evolving towards both passive interpretation and active intervention, i.e., explanations diagnose failures, guide model updates, and improve reasoning. To structure this novel perspective, we introduce a taxonomy distinguishing explainability of the prediction (i.e., uncovering mechanisms behind specific outputs) and explainability for the predictor (i.e., characterizing broader model behavior to improve the model or system). We further review domain-specific applications and key open challenges about XAI for LLMs, including scalability, faithfulness of explanations, and causal validation.
@article{dolci2026survey,
title = {A Survey of New Frontiers in Explainability for Large Language Models: From Interpretation to Intervention},
author = {Dolci, Tommaso and Fettache, Dina and La Ferla, Vittorio and Le, Amanda and Amer-Yahia, Sihem and Bogojeska, Jasmina and Hose, Katja and Ibrahim, Noha and Maniu, Silviu and Mirylenka, Katya and Peyrard, Maxime},
year = 2026,
journal = {Zenodo},
doi = {10.5281/zenodo.23592601},
}