Explainable AI (XAI)

Explainable AI, often shortened to XAI, refers to techniques and models that make a machine learning system's predictions understandable to a human, rather than leaving them as an opaque "black box." It covers two different approaches: using models that are inherently interpretable, such as a shallow decision tree or logistic regression, or applying explanation techniques on top of complex models, such as a deep neural network, whose internal logic cannot be read directly.

The most common post-hoc technique is SHAP values, which attribute a prediction to the contribution of each input feature; other approaches include simpler feature-importance rankings and example-based explanations that show similar past cases. A key distinction is between global explanations, describing how a model behaves overall, and local explanations, describing why it made one specific prediction.

Explainability matters wherever a decision affects people and needs to be justified, credit denial, medical risk scoring, hiring, and it is increasingly a regulatory expectation as part of Responsible AI rather than a nice-to-have. A common pitfall is treating a plausible-sounding explanation as proof the model is reasoning correctly; post-hoc explanations approximate a complex model's behavior and can be misleading if taken as a literal account of its internal logic.