Decision-making based on machine learning systems, especially when this decision-making can affect humanlives, is a subject of maximum interest in the Machine Learning community. It is, therefore, necessary to equipthese systems with a means of estimating uncertainty in the predictions they emit in order to help practition-ers make more informed decisions. In the present work, we introduce the topic of uncertainty estimation, andwe analyze the peculiarities of such estimation when applied to classification systems. We analyze differentmethods that have been designed to provide classification systems based on deep learning with mechanismsfor measuring the uncertainty of their predictions. We will take a look at how this uncertainty can be mod-eled and measured using different approaches, as well as practical considerations of different applications ofuncertainty. Moreover, we review some of the properties that should be borne in mind when developing suchmetrics. All in all, the present survey aims at providing a pragmatic overview of the estimation of uncertaintyin classification systems that can be very useful for both academic research and deep learning practitioners
