O impacto da qualidade das anotações na aprendizagem profunda para a segmentação de lesões de pele

Orientador: Eduardo Alves do Valle JuniorDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: Todos os anos, o Instituto Nacional do Câncer, no Brasil, registra mais de 150.000 novos casos de câncer de pele, configurando um problema real no sistema de saúde pública do país. O câncer de pele se desenvolve de maneiras diferentes, o mais comum é o carcinoma das células basais, mas o melanoma é o mais perigoso, com a maior taxa de mortalidade. As chances de cura diminuem com a maturidade da doença. Nesse cenário, métodos automáticos de triagem de lesões de pele são uma esperança para aumentar a detecção precoce e melhorar a expectativa de vida dos pacientes de câncer. Nesse estudo, nós endereçamos uma das principais tarefas do pipeline de deteção de câncer de pele: a segmentação das lesões de pele. Essa tarefa por si só é bastante desafiadora na perspectiva de visão computacional. Conjuntos de dados públicos não são tão extensos como para outros domínios de imagem e as anotações das imagens não são ótimas. Esses problemas têm um impacto real na performance do modelo e na sua capacidade de generalização. Ao longo desse trabalho, nós desejamos atacar a segunda questão, a qualidade das anotações das imagens. Nós analisamos as estatísticas de concordância entre anotadores no conjunto de dados de lesões de pele público mais famoso disponível e desenvolvemos algumas conclusões sobre as anotações disponíveis. Então, nós propusemos uma série de condicionamentos a serem aplicados nos dados de treino para avaliar como eles melhoram a concordância entre diferentes especialistas. Finalmente, nós analisamos como os condicionamentos afetam o treino e a avaliação de redes neurais profundas para a tarefa de segmentação de lesões de pele. Nossas conclusões sugerem que a baixa concordância entre anotadores presente no conjunto de dados ISIC Archive tem um impacto expressivo na performance dos modelos treinados, e considerar essa discordância pode, de fato, melhorar as capacidades de generalização das redesAbstract: Every year, the National Institute of Cancer, in Brazil, registers more than 150,000 new cases of skin cancer, making it a real issue in the country's public health system. Skin cancer evolves in different manners, the most common is the basal cell carcinoma, but melanoma is the most dangerous, with the highest mortality rate. The probability of cure decreases with the matureness of the disease. In this scenario, automatic methods for skin lesion triage is hope for boosting early detection and increasing the life expectancy of cancer patients. In this study, we address one of the main subjects of the skin cancer detection pipeline: skin lesion segmentation. The task itself is challenging from the computer vision perspective. Public data sets are not as large as for other image domains, and the annotations are not optimal. These problems have a real impact on the model's performance and capability to generalize. Along with our work, we aim to tackle the second issue, the quality of image ground truths. We analyze the inter-annotator agreement statistics inside the most popular skin lesion dataset public available and draw some conclusions about the available annotations. Then, we propose a series of conditioning to apply in the training data to evaluate how they improve the agreement between different specialists. Finally, we analyze how the conditionings affect the training and evaluation of deep neural networks for the skin lesion segmentation task. Our conclusions show that the low inter-annotator agreement available in the ISIC Archive dataset has a meaningful impact in the performance of trained models and taking the disagreement into account can indeed improve the generalization capability of the networksMestradoEngenharia de ComputaçãoMestre em Engenharia Elétric

Robust T-Loss for Medical Image Segmentation

This paper presents a new robust loss function, the T-Loss, for medical image segmentation. The proposed loss is based on the negative log-likelihood of the Student-t distribution and can effectively handle outliers in the data by controlling its sensitivity with a single parameter. This parameter is updated during the backpropagation process, eliminating the need for additional computation or prior information about the level and spread of noisy labels. Our experiments show that the T-Loss outperforms traditional loss functions in terms of dice scores on two public medical datasets for skin lesion and lung segmentation. We also demonstrate the ability of T-Loss to handle different types of simulated label noise, resembling human error. Our results provide strong evidence that the T-Loss is a promising alternative for medical image segmentation where high levels of noise or outliers in the dataset are a typical phenomenon in practice. The project website can be found at https://robust-tloss.github.ioComment: Early accepted to MICCAI 202

MSE-Nets: Multi-annotated Semi-supervised Ensemble Networks for Improving Segmentation of Medical Image with Ambiguous Boundaries

Medical image segmentation annotations exhibit variations among experts due to the ambiguous boundaries of segmented objects and backgrounds in medical images. Although using multiple annotations for each image in the fully-supervised has been extensively studied for training deep models, obtaining a large amount of multi-annotated data is challenging due to the substantial time and manpower costs required for segmentation annotations, resulting in most images lacking any annotations. To address this, we propose Multi-annotated Semi-supervised Ensemble Networks (MSE-Nets) for learning segmentation from limited multi-annotated and abundant unannotated data. Specifically, we introduce the Network Pairwise Consistency Enhancement (NPCE) module and Multi-Network Pseudo Supervised (MNPS) module to enhance MSE-Nets for the segmentation task by considering two major factors: (1) to optimize the utilization of all accessible multi-annotated data, the NPCE separates (dis)agreement annotations of multi-annotated data at the pixel level and handles agreement and disagreement annotations in different ways, (2) to mitigate the introduction of imprecise pseudo-labels, the MNPS extends the training data by leveraging consistent pseudo-labels from unannotated data. Finally, we improve confidence calibration by averaging the predictions of base networks. Experiments on the ISIC dataset show that we reduced the demand for multi-annotated data by 97.75\% and narrowed the gap with the best fully-supervised baseline to just a Jaccard index of 4\%. Furthermore, compared to other semi-supervised methods that rely only on a single annotation or a combined fusion approach, the comprehensive experimental results on ISIC and RIGA datasets demonstrate the superior performance of our proposed method in medical image segmentation with ambiguous boundaries

Leveraging inter-annotator disagreement for semi-supervised segmentation

The low signal-to-noise ratio typically found in biomedical images often leads experts to disagree about the underlying ground-truth segmentation. While existing approaches for multiple annotations try to resolve conflicting annotations, we instead focus on efficiently using pixels of disagreement to estimate areas of high uncertainty in the data and exploit this information for semi-supervised segmentation.Pseudo-labelling approaches, which utilise unlabelled data by trying to match their own predictions, need to distinguish reliable from unreliable predictions. We propose to identify unreliable pseudo-labels from the output of a separate network that is trained to predict the uncertainty in the data based on conflicting annotations from different annotators.Compared to other uncertainty estimation techniques like MC-Dropout or ensembling approaches, our approach has the two key advantages that its estimates stem directly from the data and that it is computationally more efficient. Using two public datasets, we show the effectiveness of our approach

A survey on bias in machine learning research

Current research on bias in machine learning often focuses on fairness, while overlooking the roots or causes of bias. However, bias was originally defined as a "systematic error," often caused by humans at different stages of the research process. This article aims to bridge the gap between past literature on bias in research by providing taxonomy for potential sources of bias and errors in data and models. The paper focus on bias in machine learning pipelines. Survey analyses over forty potential sources of bias in the machine learning (ML) pipeline, providing clear examples for each. By understanding the sources and consequences of bias in machine learning, better methods can be developed for its detecting and mitigating, leading to fairer, more transparent, and more accurate ML models.Comment: Submitted to journal. arXiv admin note: substantial text overlap with arXiv:2308.0946

Cloud-Based Benchmarking of Medical Image Analysis

Medical imagin

An Annotation Tool for a Digital Library System of Epidermal Data

Melanoma is one of the deadliest form of skin cancers so it becomes crucial the developing of automated systems that analyze and investigate epidermal images to early identify them also reducing unnecessary medical exams. A key element is the availability of user-friendly annotation tools that can be used by non-IT experts to produce well-annotated and high-quality medical data. In this work, we present an annotation tool to manually crate and annotate digital epidermal images, with the aim to extract meta-data (annotations, contour patterns and intersections, color information) stored and organized in an integrated digital library. This tool is obtained following rigid usability principles also based on doctors interviews and opinions. A preliminary but functional evaluation phase has been conducted with non-medical subjects by using questionnaires, in order to check the general usability and the efficacy of the proposed tool

Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries

This two-volume set LNCS 12962 and 12963 constitutes the thoroughly refereed proceedings of the 7th International MICCAI Brainlesion Workshop, BrainLes 2021, as well as the RSNA-ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge, the Federated Tumor Segmentation (FeTS) Challenge, the Cross-Modality Domain Adaptation (CrossMoDA) Challenge, and the challenge on Quantification of Uncertainties in Biomedical Image Quantification (QUBIQ). These were held jointly at the 23rd Medical Image Computing for Computer Assisted Intervention Conference, MICCAI 2020, in September 2021. The 91 revised papers presented in these volumes were selected form 151 submissions. Due to COVID-19 pandemic the conference was held virtually. This is an open access book