47 research outputs found
Greedy Bayesian Posterior Approximation with Deep Ensembles
Ensembles of independently trained neural networks are a state-of-the-art
approach to estimate predictive uncertainty in Deep Learning, and can be
interpreted as an approximation of the posterior distribution via a mixture of
delta functions. The training of ensembles relies on non-convexity of the loss
landscape and random initialization of their individual members, making the
resulting posterior approximation uncontrolled. This paper proposes a novel and
principled method to tackle this limitation, minimizing an -divergence
between the true posterior and a kernel density estimator in a function space.
We analyze this objective from a combinatorial point of view, and show that it
is submodular with respect to mixture components for any . Subsequently, we
consider the problem of ensemble construction, and from the marginal gain of
the total objective, we derive a novel diversity term for training ensembles
greedily. The performance of our approach is demonstrated on computer vision
out-of-distribution detection benchmarks in a range of architectures trained on
multiple datasets. The source code of our method is publicly available at
https://github.com/MIPT-Oulu/greedy_ensembles_training
Adaptive Segmentation of Knee Radiographs for Selecting the Optimal ROI in Texture Analysis
The purposes of this study were to investigate: 1) the effect of placement of
region-of-interest (ROI) for texture analysis of subchondral bone in knee
radiographs, and 2) the ability of several texture descriptors to distinguish
between the knees with and without radiographic osteoarthritis (OA). Bilateral
posterior-anterior knee radiographs were analyzed from the baseline of OAI and
MOST datasets. A fully automatic method to locate the most informative region
from subchondral bone using adaptive segmentation was developed. We used an
oversegmentation strategy for partitioning knee images into the compact regions
that follow natural texture boundaries. LBP, Fractal Dimension (FD), Haralick
features, Shannon entropy, and HOG methods were computed within the standard
ROI and within the proposed adaptive ROIs. Subsequently, we built logistic
regression models to identify and compare the performances of each texture
descriptor and each ROI placement method using 5-fold cross validation setting.
Importantly, we also investigated the generalizability of our approach by
training the models on OAI and testing them on MOST dataset.We used area under
the receiver operating characteristic (ROC) curve (AUC) and average precision
(AP) obtained from the precision-recall (PR) curve to compare the results. We
found that the adaptive ROI improves the classification performance (OA vs.
non-OA) over the commonly used standard ROI (up to 9% percent increase in AUC).
We also observed that, from all texture parameters, LBP yielded the best
performance in all settings with the best AUC of 0.840 [0.825, 0.852] and
associated AP of 0.804 [0.786, 0.820]. Compared to the current state-of-the-art
approaches, our results suggest that the proposed adaptive ROI approach in
texture analysis of subchondral bone can increase the diagnostic performance
for detecting the presence of radiographic OA
Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data
Knee osteoarthritis (OA) is the most common musculoskeletal disease without a
cure, and current treatment options are limited to symptomatic relief.
Prediction of OA progression is a very challenging and timely issue, and it
could, if resolved, accelerate the disease modifying drug development and
ultimately help to prevent millions of total joint replacement surgeries
performed annually. Here, we present a multi-modal machine learning-based OA
progression prediction model that utilizes raw radiographic data, clinical
examination results and previous medical history of the patient. We validated
this approach on an independent test set of 3,918 knee images from 2,129
subjects. Our method yielded area under the ROC curve (AUC) of 0.79 (0.78-0.81)
and Average Precision (AP) of 0.68 (0.66-0.70). In contrast, a reference
approach, based on logistic regression, yielded AUC of 0.75 (0.74-0.77) and AP
of 0.62 (0.60-0.64). The proposed method could significantly improve the
subject selection process for OA drug-development trials and help the
development of personalized therapeutic plans
Understanding metric-related pitfalls in image analysis validation
Validation metrics are key for the reliable tracking of scientific progress
and for bridging the current chasm between artificial intelligence (AI)
research and its translation into practice. However, increasing evidence shows
that particularly in image analysis, metrics are often chosen inadequately in
relation to the underlying research problem. This could be attributed to a lack
of accessibility of metric-related knowledge: While taking into account the
individual strengths, weaknesses, and limitations of validation metrics is a
critical prerequisite to making educated choices, the relevant knowledge is
currently scattered and poorly accessible to individual researchers. Based on a
multi-stage Delphi process conducted by a multidisciplinary expert consortium
as well as extensive community feedback, the present work provides the first
reliable and comprehensive common point of access to information on pitfalls
related to validation metrics in image analysis. Focusing on biomedical image
analysis but with the potential of transfer to other fields, the addressed
pitfalls generalize across application domains and are categorized according to
a newly created, domain-agnostic taxonomy. To facilitate comprehension,
illustrations and specific examples accompany each pitfall. As a structured
body of information accessible to researchers of all levels of expertise, this
work enhances global comprehension of a key topic in image analysis validation.Comment: Shared first authors: Annika Reinke, Minu D. Tizabi; shared senior
authors: Paul F. J\"ager, Lena Maier-Hei