4 research outputs found
Reliable Multi-class Classification based on Pairwise Epistemic and Aleatoric Uncertainty
International audienceWe propose a method for reliable prediction in multi-class classification, where reliability refers to the possibility of partial abstention in cases of uncertainty. More specifically, we allow for predictions in the form of preorder relations on the set of classes, thereby generalizing the idea of set-valued predictions. Our approach relies on combining learning by pairwise comparison with a recent proposal for modeling uncertainty in classification, in which a distinction is made between reducible (a.k.a. epistemic) uncertainty caused by a lack of information and irreducible (a.k.a. aleatoric) uncertainty due to intrinsic randomness. The problem of combining uncertain pairwise predictions into a most plausible preorder is then formalized as an integer programming problem. Experimentally, we show that our method is able to appropriately balance reliability and precision of predictions
Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods
The notion of uncertainty is of major importance in machine learning and
constitutes a key element of machine learning methodology. In line with the
statistical tradition, uncertainty has long been perceived as almost synonymous
with standard probability and probabilistic predictions. Yet, due to the
steadily increasing relevance of machine learning for practical applications
and related issues such as safety requirements, new problems and challenges
have recently been identified by machine learning scholars, and these problems
may call for new methodological developments. In particular, this includes the
importance of distinguishing between (at least) two different types of
uncertainty, often referred to as aleatoric and epistemic. In this paper, we
provide an introduction to the topic of uncertainty in machine learning as well
as an overview of attempts so far at handling uncertainty in general and
formalizing this distinction in particular.Comment: 59 page