Diagnosis, Localization, and Prognosis of Melanoma in WSIs with a Complete Pipeline by Digital Pathology

The most dangerous and aggressive form of skin cancer is melanoma, responsible for 90% of skin cancer mortality. Early detection of melanoma plays a crucial role in the prognostic outcome. The diagnostic has to be performed by a pathologist, which is time- consuming. The recent increase in melanoma incidents indicates the growing demand for a more efficient diagnostic process. This thesis’s main objective is to develop a pipeline utilizing two independent pre- trained models built on the VGG16 architecture. This pipeline consists of a diagnostic and a prognostic model. The diagnosis model is responsible for localizing malignant patches in WSIs and giving a patient-level diagnosis. The prognosis model uses the output from the diagnosis model to provide a patient-level prognosis. The complete pipeline provides both a prognostic and a diagnostic tool, which can be used by a pathologist when evaluating Whole Slide Images (WSIs). A total of 243 WSIs were provided by Stavanger University Hospital for this thesis. All have been provided a patient-level label. 203 of the WSIs were used for parameter tuning and 40 were used for testing. The diagnosis model performed with a 100% accuracy when evaluated on the original test, which was provided together with the training set. The prognosis model also per- formed well on the original dataset, with an accuracy of 0.7885. The model’s capability to predict diagnosis and prognosis decreases significantly when being introduced to the new dataset. In addition to developing the pipeline, some parameters for the diagnosis model was found using a ROC cuve. By using the new parameters for the diagnosis model on the validation set, the performance of the diagnosis model increased when using the test set. The prognosis model performed relatively equally in all experiments. A correlation between the number of patches in a WSI and the number of patches predicted malignant was discovered and counteracted by altering the patient-level threshold calculation method.The most dangerous and aggressive form of skin cancer is melanoma, responsible for 90% of skin cancer mortality. Early detection of melanoma plays a crucial role in the prognostic outcome. The diagnostic has to be performed by a pathologist, which is time- consuming. The recent increase in melanoma incidents indicates the growing demand for a more efficient diagnostic process. This thesis’s main objective is to develop a pipeline utilizing two independent pre- trained models built on the VGG16 architecture. This pipeline consists of a diagnostic and a prognostic model. The diagnosis model is responsible for localizing malignant patches in WSIs and giving a patient-level diagnosis. The prognosis model uses the output from the diagnosis model to provide a patient-level prognosis. The complete pipeline provides both a prognostic and a diagnostic tool, which can be used by a pathologist when evaluating Whole Slide Images (WSIs). A total of 243 WSIs were provided by Stavanger University Hospital for this thesis. All have been provided a patient-level label. 203 of the WSIs were used for parameter tuning and 40 were used for testing. The diagnosis model performed with a 100% accuracy when evaluated on the original test, which was provided together with the training set. The prognosis model also per- formed well on the original dataset, with an accuracy of 0.7885. The model’s capability to predict diagnosis and prognosis decreases significantly when being introduced to the new dataset. In addition to developing the pipeline, some parameters for the diagnosis model was found using a ROC cuve. By using the new parameters for the diagnosis model on the validation set, the performance of the diagnosis model increased when using the test set. The prognosis model performed relatively equally in all experiments. A correlation between the number of patches in a WSI and the number of patches predicted malignant was discovered and counteracted by altering the patient-level threshold calculation method

Advantages of manual and automatic computer-aided compared to traditional histopathological diagnosis of melanoma: A pilot study

Background: Cutaneous malignant melanoma (CMM) accounts for the highest mortality rate among all skin cancers. Traditional histopathologic diagnosis may be limited by the pathologists’ subjectivity. Second-opinion strategies and multidisciplinary consultations are usually performed to overcome this issue. An available solution in the future could be the use of automated solutions based on a computational algorithm that could help the pathologist in everyday practice. The aim of this pilot study was to investigate the potential diagnostic aid of a machine-based algorithm in the histopathologic diagnosis of CMM. Methods: We retrospectively examined excisional biopsies of 50 CMM and 20 benign congenital compound nevi. Hematoxylin and eosin (H&E) stained WSI were reviewed independently by two expert dermatopathologists. A fully automated pipeline for WSI processing to support the estimation and prioritization of the melanoma areas was developed. Results: The spatial distribution of the nuclei in the sample provided a multi-scale overview of the tumor. A global overview of the lesion's silhouette was achieved and, by increasing the magnification, the topological distribution of the nuclei and the most informative areas of interest for the CMM diagnosis were identified and highlighted. These silhouettes allow the histopathologist to discriminate between nevus and CMM with an accuracy of 96% without any extra information. Conclusion: In this study we proposed an easy-to-use model that produces segmentations of CMM silhouettes at fine detail level

Domain shifts in dermoscopic skin cancer datasets: Evaluation of essential limitations for clinical translation

The limited ability of Convolutional Neural Networks to generalize to images from previously unseen domains is a major limitation, in particular, for safety-critical clinical tasks such as dermoscopic skin cancer classification. In order to translate CNN-based applications into the clinic, it is essential that they are able to adapt to domain shifts. Such new conditions can arise through the use of different image acquisition systems or varying lighting conditions. In dermoscopy, shifts can also occur as a change in patient age or occurence of rare lesion localizations (e.g. palms). These are not prominently represented in most training datasets and can therefore lead to a decrease in performance. In order to verify the generalizability of classification models in real world clinical settings it is crucial to have access to data which mimics such domain shifts. To our knowledge no dermoscopic image dataset exists where such domain shifts are properly described and quantified. We therefore grouped publicly available images from ISIC archive based on their metadata (e.g. acquisition location, lesion localization, patient age) to generate meaningful domains. To verify that these domains are in fact distinct, we used multiple quantification measures to estimate the presence and intensity of domain shifts. Additionally, we analyzed the performance on these domains with and without an unsupervised domain adaptation technique. We observed that in most of our grouped domains, domain shifts in fact exist. Based on our results, we believe these datasets to be helpful for testing the generalization capabilities of dermoscopic skin cancer classifiers

Melanoma prognosis prediction using image processing and machine learning

Death of melanoma cancer is most common in Europe, and northern Europe has the second highest mortality rate of melanoma in the world, with 1.9 per 100 000 dying from melanoma in northern Europe in 2020. Prognosis of melanoma is nowadays based on educated guesses by a pathologist, from analyzing patient tumors. Analyzing tumors takes much time, which limits the pathologist’s capability for the number of tumors they are able to analyze in a given amount of time. The primary objective of this thesis is to suggest a machine learning based method for predicting prognosis of melanoma, to aid the pathologist. The proposed method is based on the VGG16 architecture with pre-trained weights as the backbone, adding some fully connected layers. The network is trained and validated on whole slide images (WSI) from 51 patients with known melanoma prognosis, produced at Stavanger University Hospital. Regions of interest (ROI) areas in these images are marked by a pathologist. A foreground segmentation algorithm for skin histological WSI is presented. Tiles are extracted from ROI areas in the WSIs, and resized to contain three different magnification levels, which are used in model training and validation. Multiple magnification levels are used to mimic the way a pathologist analyzes tissues at different magnifications. Experiments are done by combining different magnification scales, utilizing tiles from respectively one, two and three magnification level(s) to train the models. The best performing model used only one magnification scale at 20x. Cross validation results gave a F1 score of 0.7667, and an area under the curve in a receiver operating characteristic curve of 0.81. This result is promising, considering the small number of patients in the dataset. For future work, the method has to be tested on a larger dataset. It is also recommended to test a larger set of possible hyperparameters and/or model architectures.Death of melanoma cancer is most common in Europe, and northern Europe has the second highest mortality rate of melanoma in the world, with 1.9 per 100 000 dying from melanoma in northern Europe in 2020. Prognosis of melanoma is nowadays based on educated guesses by a pathologist, from analyzing patient tumors. Analyzing tumors takes much time, which limits the pathologist’s capability for the number of tumors they are able to analyze in a given amount of time. The primary objective of this thesis is to suggest a machine learning based method for predicting prognosis of melanoma, to aid the pathologist. The proposed method is based on the VGG16 architecture with pre-trained weights as the backbone, adding some fully connected layers. The network is trained and validated on whole slide images (WSI) from 51 patients with known melanoma prognosis, produced at Stavanger University Hospital. Regions of interest (ROI) areas in these images are marked by a pathologist. A foreground segmentation algorithm for skin histological WSI is presented. Tiles are extracted from ROI areas in the WSIs, and resized to contain three different magnification levels, which are used in model training and validation. Multiple magnification levels are used to mimic the way a pathologist analyzes tissues at different magnifications. Experiments are done by combining different magnification scales, utilizing tiles from respectively one, two and three magnification level(s) to train the models. The best performing model used only one magnification scale at 20x. Cross validation results gave a F1 score of 0.7667, and an area under the curve in a receiver operating characteristic curve of 0.81. This result is promising, considering the small number of patients in the dataset. For future work, the method has to be tested on a larger dataset. It is also recommended to test a larger set of possible hyperparameters and/or model architectures

Artificial Intelligence in Skin Cancer: A Literature Review from Diagnosis to Prevention and Beyond

Artificial Intelligence (AI) in medicine is quickly expanding, offering significant potential benefits in diagnosis and prognostication. While concerns may exist regarding its implementation, it is important for dermatologists and dermatopathologists to collaborate with technical specialists to embrace AI as a tool for enhancing medical decision-making and improving healthcare accessibility. This is particularly relevant in melanocytic neoplasms, which continue to present challenges despite years of experience. Dermatology, with its extensive medical data and images, provides an ideal field for training AI algorithms to enhance patient care. Collaborative efforts between medical professionals and technical specialists are crucial in harnessing the power of AI while ensuring it complements and enhances the existing healthcare framework. By staying informed about AI concepts and ongoing research, dermatologists can remain at the forefront of this emerging field and leverage its potential to improve patient outcomes. In conclusion, AI holds great promise in dermatology, especially in the management and analysis of Skin cancer (SC). In this review we strive to introduce the concepts of AI and its association with dermatology, providing an overview of recent studies in the field, such as existing applications and future potential in dermatology

Computer Aided Diagnostic Support System for Skin cancer: Review of techniques and algorithms

Image-based computer aided diagnosis systems have significant potential for screening and early detection of malignant melanoma. We review the state of the art in these systems and examine current practices, problems, and prospects of image acquisition, pre-processing, segmentation, feature extraction and selection, and classification of dermoscopic images. This paper reports statistics and results from the most important implementations reported to date. We compared the performance of several classifiers specifically developed for skin lesion diagnosis and discussed the corresponding findings. Whenever available, indication of various conditions that affect the technique’s performance is reported. We suggest a framework for comparative assessment of skin cancer diagnostic models and review the results based on these models. The deficiencies in some of the existing studies are highlighted and suggestions for future research are provided

Computer aided diagnostic support system for skin cancer: A review of techniques and algorithms

Image-based computer aided diagnosis systems have significant potential for screening and early detection of malignant melanoma. We review the state of the art in these systems and examine current practices, problems, and prospects of image acquisition, pre-processing, segmentation, feature extraction and selection, and classification of dermoscopic images. This paper reports statistics and results from the most important implementations reported to date. We compared the performance of several classifiers specifically developed for skin lesion diagnosis and discussed the corresponding findings. Whenever available, indication of various conditions that affect the technique's performance is reported. We suggest a framework for comparative assessment of skin cancer diagnostic models and review the results based on these models. The deficiencies in some of the existing studies are highlighted and suggestions for future research are provided. © 2013 Ammara Masood and Adel Ali Al-Jumaily