The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions

Training of neural networks for automated diagnosis of pigmented skin lesions is hampered by the small size and lack of diversity of available datasets of dermatoscopic images. We tackle this problem by releasing the HAM10000 ("Human Against Machine with 10000 training images") dataset. We collected dermatoscopic images from different populations acquired and stored by different modalities. Given this diversity we had to apply different acquisition and cleaning methods and developed semi-automatic workflows utilizing specifically trained neural networks. The final dataset consists of 10015 dermatoscopic images which are released as a training set for academic machine learning purposes and are publicly available through the ISIC archive. This benchmark dataset can be used for machine learning and for comparisons with human experts. Cases include a representative collection of all important diagnostic categories in the realm of pigmented lesions. More than 50% of lesions have been confirmed by pathology, while the ground truth for the rest of the cases was either follow-up, expert consensus, or confirmation by in-vivo confocal microscopy

Inequalities in the patterns of dermoscopy use and training across Europe : conclusions of the Eurodermoscopy pan-European survey

Publisher Copyright: © 2020, JLE/Springer.Background: Dermoscopy is a widely used technique, recommended in clinical practice guidelines worldwide for the early diagnosis of skin cancers. Intra-European disparities are reported for early detection and prognosis of skin cancers, however, no information exists about regional variation in patterns of dermoscopy use across Europe. Objective: To evaluate the regional differences in patterns of dermoscopy use and training among European dermatologists. Materials & Methods: An online survey of European-registered dermatologists regarding dermoscopy training, practice and attitudes was established. Answers from Eastern (EE) versus Western European (WE) countries were compared and their correlation with their respective countries’ gross domestic product/capita (GDPc) and total and government health expenditure/capita (THEc and GHEc) was analysed. Results: We received 4,049 responses from 14 WE countries and 3,431 from 18 EE countries. A higher proportion of WE respondents reported dermoscopy use (98% vs. 77%, p<0.001) and training during residency (43% vs. 32%) or anytime (96.5% vs. 87.6%) (p<0.001) compared to EE respondents. The main obstacles in dermoscopy use were poor access to dermoscopy equipment in EE and a lack of confidence in one’s skills in WE. GDPc, THEc and GHEc correlated with rate of dermoscopy use and dermoscopy training during residency (Spearman rho: 0.5–0.7, p<0.05), and inversely with availability of dermoscopy equipment. Conclusion: The rates and patterns of dermoscopy use vary significantly between Western and Eastern Europe, on a background of economic inequality. Regionally adapted interventions to increase access to dermoscopy equipment and training might enhance the use of this technique towards improving the early detection of skin cancers.Peer reviewe

Indications for Digital Monitoring of Patients With Multiple Nevi: Recommendations from the International Dermoscopy Society

Introduction: In patients with multiple nevi, sequential imaging using total body skin photography (TBSP) coupled with digital dermoscopy (DD) documentation reduces unnecessary excisions and improves the early detection of melanoma. Correct patient selection is essential for optimizing the efficacy of this diagnostic approach. Objectives: The purpose of the study was to identify, via expert consensus, the best indications for TBSP and DD follow-up. Methods: This study was performed on behalf of the International Dermoscopy Society (IDS). We attained consensus by using an e-Delphi methodology. The panel of participants included international experts in dermoscopy. In each Delphi round, experts were asked to select from a list of indications for TBSP and DD. Results: Expert consensus was attained after 3 rounds of Delphi. Participants considered a total nevus count of 60 or more nevi or the presence of a CDKN2A mutation sufficient to refer the patient for digital monitoring.  Patients with more than 40 nevi were only considered an indication in case of personal history of melanoma or red hair and/or a MC1R mutation or history of organ transplantation. Conclusions: Our recommendations support clinicians in choosing appropriate follow-up regimens for patients with multiple nevi and in applying the time-consuming procedure of sequential imaging more efficiently. Further studies and real-life data are needed to confirm the usefulness of this list of indications in clinical practice

Seven non-melanoma features to rule out facial melanoma

Facial melanoma is difficult to diagnose and dermatoscopic features are often subtle. Dermatoscopic non-melanoma patterns may have a comparable diagnostic value. In this pilot study, facial lesions were collected retrospectively, resulting in a case set of 339 melanomas and 308 non-melanomas. Lesions were evaluated for the prevalence (> 50% of lesional surface) of 7 dermatoscopic non-melanoma features: scales, white follicles, erythema/reticular vessels, reticular and/or curved lines/fingerprints, structureless brown colour, sharp demarcation, and classic criteria of seborrhoeic keratosis. Melanomas had a lower number of non-melanoma patterns (p < 0.001). Scoring a lesion suspicious when no prevalent non-melanoma pattern is found resulted in a sensitivity of 88.5% and a specificity of 66.9% for the diagnosis of melanoma. Specificity was higher for solar lentigo (78.8%) and seborrhoeic keratosis (74.3%) and lower for actinic keratosis (61.4%) and lichenoid keratosis (25.6%). Evaluation of prevalent non-melanoma patterns can provide slightly lower sensitivity and higher specificity in detecting facial melanoma compared with already known malignant features

Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge

Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy.We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25?331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use.64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed.We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice

A reinforcement learning model for AI-based decision support in skin cancer

: We investigated whether human preferences hold the potential to improve diagnostic artificial intelligence (AI)-based decision support using skin cancer diagnosis as a use case. We utilized nonuniform rewards and penalties based on expert-generated tables, balancing the benefits and harms of various diagnostic errors, which were applied using reinforcement learning. Compared with supervised learning, the reinforcement learning model improved the sensitivity for melanoma from 61.4% to 79.5% (95% confidence interval (CI): 73.5-85.6%) and for basal cell carcinoma from 79.4% to 87.1% (95% CI: 80.3-93.9%). AI overconfidence was also reduced while simultaneously maintaining accuracy. Reinforcement learning increased the rate of correct diagnoses made by dermatologists by 12.0% (95% CI: 8.8-15.1%) and improved the rate of optimal management decisions from 57.4% to 65.3% (95% CI: 61.7-68.9%). We further demonstrated that the reward-adjusted reinforcement learning model and a threshold-based model outperformed naïve supervised learning in various clinical scenarios. Our findings suggest the potential for incorporating human preferences into image-based diagnostic algorithms