11 research outputs found
Investigating Poor Performance Regions of Black Boxes: LIME-based Exploration in Sepsis Detection
Interpreting machine learning models remains a challenge, hindering their
adoption in clinical settings. This paper proposes leveraging Local
Interpretable Model-Agnostic Explanations (LIME) to provide interpretable
descriptions of black box classification models in high-stakes sepsis
detection. By analyzing misclassified instances, significant features
contributing to suboptimal performance are identified. The analysis reveals
regions where the classifier performs poorly, allowing the calculation of error
rates within these regions. This knowledge is crucial for cautious
decision-making in sepsis detection and other critical applications. The
proposed approach is demonstrated using the eICU dataset, effectively
identifying and visualizing regions where the classifier underperforms. By
enhancing interpretability, our method promotes the adoption of machine
learning models in clinical practice, empowering informed decision-making and
mitigating risks in critical scenarios.Comment: Accepted at the 1st World Conference on eXplainable Artificial
Intelligence - Late-breaking work, Demos and Doctoral Consortium, 202
Interpretability of deep learning models: A survey of results
Deep neural networks have achieved near-human accuracy levels in various types of classification and prediction tasks including images, text, speech, and video data. However, the networks continue to be treated mostly as black-box function approximators, mapping a given input to a classification output. The next step in this human-machine evolutionary process-incorporating these networks into mission critical processes such as medical diagnosis, planning and control-requires a level of trust association with the machine output. Typically, statistical metrics are used to quantify the uncertainty of an output. However, the notion of trust also depends on the visibility that a human has into the working of the machine. In other words, the neural network should provide human-understandable justifications for its output leading to insights about the inner workings. We call such models as interpretable deep networks. Interpretability is not a monolithic notion. In fact, the subjectivity of an interpretation, due to different levels of human understanding, implies that there must be a multitude of dimensions that together constitute interpretability. In addition, the interpretation itself can be provided either in terms of the low-level network parameters, or in terms of input features used by the model. In this paper, we outline some of the dimensions that are useful for model interpretability, and categorize prior work along those dimensions. In the process, we perform a gap analysis of what needs to be done to improve model interpretability
Exceptionally monotone models : the rank correlation model class for exceptional model mining
Exceptional Model Mining strives to find coherent subgroups of the dataset where multiple target attributes interact in an unusual way. One instance of such an investigated form of interaction is Pearson's correlation coefficient between two targets. EMM then finds subgroups with an exceptionally linear relation between the targets. In this paper, we enrich the EMM toolbox by developing the more general rank correlation model class. We find subgroups with an exceptionally monotone relation between the targets. Apart from catering for this richer set of relations, the rank correlation model class does not necessarily require the assumption of target normality, which is implicitly invoked in the Pearson's correlation model class. Furthermore, it is less sensitive to outliers