The work discusses the use of machine learning algorithms for anomaly
detection in medical image analysis and how the performance of these algorithms
depends on the number of annotators and the quality of labels. To address the
issue of subjectivity in labeling with a single annotator, we introduce a
simple and effective approach that aggregates annotations from multiple
annotators with varying levels of expertise. We then aim to improve the
efficiency of predictive models in abnormal detection tasks by estimating
hidden labels from multiple annotations and using a re-weighted loss function
to improve detection performance. Our method is evaluated on a real-world
medical imaging dataset and outperforms relevant baselines that do not consider
disagreements among annotators.Comment: This is a short version submitted to the Midwest Machine Learning
Symposium (MMLS 2023), Chicago, IL, US