13,961 research outputs found
Calibrating Ensembles for Scalable Uncertainty Quantification in Deep Learning-based Medical Segmentation
Uncertainty quantification in automated image analysis is highly desired in
many applications. Typically, machine learning models in classification or
segmentation are only developed to provide binary answers; however, quantifying
the uncertainty of the models can play a critical role for example in active
learning or machine human interaction. Uncertainty quantification is especially
difficult when using deep learning-based models, which are the state-of-the-art
in many imaging applications. The current uncertainty quantification approaches
do not scale well in high-dimensional real-world problems. Scalable solutions
often rely on classical techniques, such as dropout, during inference or
training ensembles of identical models with different random seeds to obtain a
posterior distribution. In this paper, we show that these approaches fail to
approximate the classification probability. On the contrary, we propose a
scalable and intuitive framework to calibrate ensembles of deep learning models
to produce uncertainty quantification measurements that approximate the
classification probability. On unseen test data, we demonstrate improved
calibration, sensitivity (in two out of three cases) and precision when being
compared with the standard approaches. We further motivate the usage of our
method in active learning, creating pseudo-labels to learn from unlabeled
images and human-machine collaboration
Visual Integration of Data and Model Space in Ensemble Learning
Ensembles of classifier models typically deliver superior performance and can
outperform single classifier models given a dataset and classification task at
hand. However, the gain in performance comes together with the lack in
comprehensibility, posing a challenge to understand how each model affects the
classification outputs and where the errors come from. We propose a tight
visual integration of the data and the model space for exploring and combining
classifier models. We introduce a workflow that builds upon the visual
integration and enables the effective exploration of classification outputs and
models. We then present a use case in which we start with an ensemble
automatically selected by a standard ensemble selection algorithm, and show how
we can manipulate models and alternative combinations.Comment: 8 pages, 7 picture
Fair comparison of skin detection approaches on publicly available datasets
Skin detection is the process of discriminating skin and non-skin regions in
a digital image and it is widely used in several applications ranging from hand
gesture analysis to track body parts and face detection. Skin detection is a
challenging problem which has drawn extensive attention from the research
community, nevertheless a fair comparison among approaches is very difficult
due to the lack of a common benchmark and a unified testing protocol. In this
work, we investigate the most recent researches in this field and we propose a
fair comparison among approaches using several different datasets. The major
contributions of this work are an exhaustive literature review of skin color
detection approaches, a framework to evaluate and combine different skin
detector approaches, whose source code is made freely available for future
research, and an extensive experimental comparison among several recent methods
which have also been used to define an ensemble that works well in many
different problems. Experiments are carried out in 10 different datasets
including more than 10000 labelled images: experimental results confirm that
the best method here proposed obtains a very good performance with respect to
other stand-alone approaches, without requiring ad hoc parameter tuning. A
MATLAB version of the framework for testing and of the methods proposed in this
paper will be freely available from https://github.com/LorisNann
- …