Interpretability & Explainability of Black-Box Models

There is increasing need for methods that aid interpretability and explainability in Machine Learning (ML). In this thesis, inspired by recent studies calling for an increase in usage of Sensitivity Analysis (SA) methods, we explore interfaces between SA and ML. In particular, in our first work we use the notion of mean dimension to characterize the internal structure of complex modern neural network architectures. We exploit this notion with an innovative estimation routine which allows to calculate the mean dimension from data and we compare different architectures, comprising LeNet, ResNet and DenseNet. In a second work, we analyze the difference between two trained neural networks using the notion of mean dimension and a new method for gaining interpretability on a dataset level, the so-called Xi-method. The networks arise from two different optimization routines, namely the standard stochastic gradient descent (SGD) and the recent replicated stochastic gradient descent (rSGD). The rSGD algorithm is designed to find flat minima in the loss landscape. The results, exploiting explainability tools, confirm findings of previous studies that suggest that the rSGD method is connected to better generalization. Finally, we go on and ask the question if neural networks ’see’ statistical dependence, and we offer a comparison between specific methods devised to understand the decision of neural networks and global sensitivity measures. Overall, the thesis finds useful tools for investigation and shading light into modern black-box models.