Explaining Predictions of Arbitrary Model with Feature Contributions

Prof. Igor Kononenko
Slovenia, Laboratory for Cognitive Modeling University of Ljubljana
Acquisition of knowledge from data is a quintessential task of machine learning. The data are often noisy, inconsistent, and incomplete, so various preprocessing methods are used before the appropriate machine learning algorithm is applied. The knowledge we extract this way might not be suitable for immediate use and one or more data postprocessing methods could be applied as well.
Data postprocessing includes integration, filtering, evaluation, and explanation of acquired knowledge. We present a sensitivity analysis-based method for explaining prediction models, which can be applied to any type of classification or regression model. This method is based on fundamental concepts from coalitional game theory where predictions are explained using individual feature values. Its advantage over existing general methods is that all subsets of input features are perturbed, so all interactions between features and redundancies are taken into account. We overcome the method’s initial exponential time complexity using a sampling-based approximation. When explaining an additive model, this method is equivalent to commonly used additive model-specific methods. We illustrate the method’s usefulness with examples from artificial and real-world data sets and an empirical analysis of running times. We use the developed method on models generated by several well-known machine learning algorithms. The results demonstrate that this method is efficient and that the explanations are intuitive and useful.
In addition, we evaluated transparency of explanations to human users and found that the results from a controlled experiment with 122 participants suggested that the method’s explanations improved the participants’ understanding of the model. The method was successfully applied to several real-world problems, including a novel breast cancer recurrence prediction data set where results were evaluated by expert oncologists.