Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge

Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy.We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25?331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use.64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed.We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice.Melanoma Research Alliance and La Marató de TV3.Copyright © 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY-NC-ND 4.0 license. Published by Elsevier Ltd.. All rights reserved

Human surface anatomy terminology for dermatology: a Delphi consensus from the International Skin Imaging Collaboration

BackgroundThere is no internationally vetted set of anatomic terms to describe human surface anatomy.ObjectiveTo establish expert consensus on a standardized set of terms that describe clinically relevant human surface anatomy.MethodsWe conducted a Delphi consensus on surface anatomy terminology between July 2017 and July 2019. The initial survey included 385 anatomic terms, organized in seven levels of hierarchy. If agreement exceeded the 75% established threshold, the term was considered - accepted- and included in the final list. Terms added by the participants were passed on to the next round of consensus. Terms with <75% agreement were included in subsequent surveys along with alternative terms proposed by participants until agreement was reached on all terms.ResultsThe Delphi included 21 participants. We found consensus (- ¥75% agreement) on 361/385 (93.8%) terms and eliminated one term in the first round. Of 49 new terms suggested by participants, 45 were added via consensus. To adjust for a recently published International Classification of Diseases- Surface Topography list of terms, a third survey including 111 discrepant terms was sent to participants. Finally, a total of 513 terms reached agreement via the Delphi method.ConclusionsWe have established a set of 513 clinically relevant terms for denoting human surface anatomy, towards the use of standardized terminology in dermatologic documentation.Linked Commentary: R.J.G. Chalmers. J Eur Acad Dermatol Venereol 2020; 34: 2456- 2457. https://doi.org/10.1111/jdv.16978.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/163915/1/jdv16855_am.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/163915/2/jdv16855-sup-0001-FigS1-S3.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/163915/3/jdv16855.pd

Seven Plus One Steps to Assess Pigmented Nail Bands (Melanonychia Striata Longitudinalis)

Melanonychia striata longitudinalis might involve one or more fingers and/or toes and might result from several different causes, including benign and malignant tumors, trauma, infections, and activation of melanocytes that might be reactive or related to the pigmentary trait, drugs and some rare syndromes. This broad differential diagnosis renders the clinical assessment of melanonychia striata particularly challenging. Nail matrix melanoma is relatively rare, occurs almost always in adults involves more frequently the first toe or thumb. The most common nail unit cancer, squamous cell carcinoma / Bowen disease (SCC) of the nail matrix is seldom pigmented. Histopathologic examination remains the gold standard for melanoma and SCC diagnosis, but excisional or partial biopsies from the nail matrix require training and is not routinely performed by the majority of clinicians. Furthermore, the histopathologic evaluation of melanocytic lesions of the nail matrix is particularly challenging, since early melanoma has only bland histopathologic alterations. Dermatoscopy of the nail plate and its free edge significantly improves the clinical diagnosis, since specific patterns have been associated to each one of the causes of melanonychia. Based on knowledge generated and published in the last decades, we propose herein a stepwise diagnostic approach for melanonychia striata longitudinalis: 1) Hemorrhage first 2) Age matters 3) Number of nails matters 4) Free edge matters 5) Brown or gray? 6) Size matters 7) Regular or irregular and, finally, “follow back”

Incidence of New Primary Cutaneous Melanoma in Patients with Metastatic Melanoma Treated with Immune Checkpoint Inhibitors: A Single-Center Cohort Study

Importance: The development of new primary cutaneous melanoma (CM) after starting immune checkpoint inhibitor (ICI) therapy is poorly characterized. Objective: To determine the incidence of new CM in patients treated with ipilimumab, nivolumab, and/or pembrolizumab for metastatic melanoma. Design, Setting, and Participants: Single-center, retrospective, observational cohort study using an institutional database to identify patients diagnosed with melanoma at a tertiary care cancer hospital in New York, New York. Exposures: Ipilimumab, nivolumab, and/or pembrolizumab treatment for metastatic melanoma. Main Outcomes and Measures: Primary outcomes were the incidence proportion, the incidence rate, and the 5-year cause-specific cumulative risk. Results: A total of 2251 patients were included in the study; mean (SD) age at the time of ICI start was 62.8 (14.4) years. The majority were male (63.8%, n = 1437), White (92.7%, n = 2086), and non-Hispanic (92.1%, n = 2073). Forty-two of 2251 patients who received ipilimumab, nivolumab, and/or pembrolizumab were diagnosed with 48 new CMs at a median (range) of 397.5 (39-2409) days after ICI initiation. The median age of affected patients at the time of ICI first dose was 66.5 years. The majority were male (66.7%, n = 28), White (92.9%, n = 39), and non-Hispanic (100.0%, n = 42). There were no differences in age, sex, race, and ethnicity among patients who did and did not develop a new CM. Patients who developed a new CM were more likely to have a family history of melanoma (23.8% vs 16.3%, P =.02). Most new CMs (n = 30, 62.5%) were diagnosed after the last date of ICI administration. Twenty-seven (56.3%) new CMs were in situ and 21 (43.8%) were invasive. Of the invasive CMs with a reported Breslow thickness (n = 20), the median (range) thickness was 0.4 (0.1-8.4) mm. The overall incidence proportion of new CM was 1.9% (95% CI, 1.4%-2.5%) and the incidence rate was 1103 cases per 100000 person-years (95% CI, 815-1492). The 5-year cumulative cause-specific risk of new CM was 4.9% (95% CI, 3.3%-7.4%). Conclusions and Relevance: Patients treated with ICI therapy for metastatic melanoma remain at risk for the development of new CM.. © 2021 American Medical Association. All rights reserved

Clinical and dermoscopic features associated with lichen planus-like keratoses that undergo skin biopsy: A single-center, observational study

Background/Objectives: Lichen planus-like keratoses (LPLK) are benign skin lesions that can mimic malignancy; the clinical and dermoscopic features distinguishing lichen planus-like keratoses from skin tumors have not been extensively studied. The objective of this study was to identify dermoscopic features that may prevent unnecessary biopsies of lichen planus-like keratoses. Methods: Retrospective, single-center, observational study of biopsied skin lesions at a tertiary center. We compared 355 lichen planus-like keratoses to 118 non-lichen planus-like keratoses lesions with lichen planus-like keratosis in the differential diagnosis biopsied from August 1, 2015, to December 31, 2016. The investigators were blinded to the diagnosis of the lesions. Results: Lichen planus-like keratoses were most frequently non-pigmented (61.7%), truncal (52.1%), and on sun-damaged skin (69.6%); the majority occurred in Whites (95.5%) and females (62.8%). Dermoscopically, lichen planus-like keratoses were more likely than non-lichen planus-like keratoses to have scale (42.5% vs 31.4%, P = 0.03) and orange colour (8.2% vs 0.9%, P = 0.01). Among lesions with peppering (n = 76; 63 lichen planus-like keratoses and 13 non-lichen planus-like keratoses), coarse ± fine peppering (73% vs 38.5%, P = 0.02) and peppering as the only feature (34.9% vs 0%, P = 0.01) were associated with lichen planus-like keratoses. Conclusions: Lichen planus-like keratoses can be challenging to distinguish from benign and malignant skin tumors. The presence of dermoscopic scale and orange colour may aid in the recognition of lichen planus-like keratosis. Coarse peppering and the presence of peppering as the only dermoscopic feature may further aid the identification of pigmented lichen planus-like keratoses. © 2018 The Australasian College of Dermatologist

Evaluation of dermatoscopic criteria for early detection of squamous cell carcinoma arising on an actinic keratosis

Background: Advanced squamous cell carcinoma (SCC) can be discriminated easily from actinic keratosis (AK) based on clinical and dermatoscopic features. However, at the initial stage of dermal invasion, SCC might still be clinically flat and discrimination from AK remains challenging, even with the addition of dermatoscopy. Objective: The aim of this study was to investigate the clinical and dermatoscopic criteria that could suggest early invasion and serve as potent predictors to discriminate early SCC from AK. Methods: Clinical and dermatoscopic images of histopathologically diagnosed AKs and early SCCs were evaluated for the presence of predefined criteria by 3 independent investigators. Results: A total of 50 early SCCs and 45 AKs were included. The main positive dermatoscopic predictors of early SCC were dotted/glomerular vessels (odds ratio [OR] 3.83), hairpin vessels (OR 12.12), and white structureless areas (OR 3.58), whereas background erythema represented a negative SCC predictor (OR 0.22). Limitations: The retrospective evaluation of images. Moreover, the differential diagnosis included in the study is restricted between AK and early SCC. Conclusions: We identified potent predictors for the discrimination of AK and early SCC that may better guide management decisions in everyday clinical practice. © 2021 American Academy of Dermatology, Inc

Management of complex head-and-neck basal cell carcinomas using a combined reflectance confocal microscopy/optical coherence tomography: a descriptive study

Introduction: Recently, a combined reflectance confocal microscopy (RCM)–optical coherence tomography (OCT) has been tested for the diagnosis of basal cell carcinoma (BCC). Evaluating the role of RCM–OCT in management of complex BCCs has not been studied. The objective of the study was to investigate the utility of a new combined RCM–OCT device in the evaluation and management of complex BCCs in a descriptive study. Methods: Prospective study of consecutive cases (July 2018–June 2019) of biopsy-proven ‘complex’ BCC defined as BCC in the head-and-neck area with multiple high-risk criteria such as large size in the mask area, multiple recurrences, and high-risk subtype. All cases were evaluated with a combined RCM–OCT device that provided simultaneous image viewing on a screen. Lesions were evaluated bedside with RCM–OCT according to previously described criteria. Results: Ten patients with complex head-and-neck BCCs had mean age of 73.1 ± 13.0 years. Six (60%) patients were males. Mean BCC clinical size was 1.9 ± 1.2 cm (range 0.6–4.0 cm). RCM detected residual BCC in 8 out of 10 cases (80%) and OCT detected residual BCC in all 10 cases (100%). Six BCCs (60%) had a depth estimate of &gt; 1000 µm under OCT. In five cases, (50%) RCM–OCT imaging results led to a change/modification in BCC management. Conclusion: The use of a combined RCM–OCT device may help in the evaluation of complex head-and-neck BCCs by guiding treatment selection and defining the extent of surgery. © 2020, Springer-Verlag GmbH Germany, part of Springer Nature