Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge

Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy.We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25?331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use.64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed.We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice.Melanoma Research Alliance and La Marató de TV3.Copyright © 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY-NC-ND 4.0 license. Published by Elsevier Ltd.. All rights reserved

Muscular cystic hydatidosis: case report

BACKGROUND: Hydatidosis is a zoonosis caused by Echinococcus granulosus, and ingesting eggs released through the faeces from infected dogs infects humans. The location of the hydatid cysts is mostly hepatic and/or pulmonary, whereas musculoskeletal hydatidosis is very rare. CASE PRESENTATION: We report an unusual case of primary muscular hydatidosis in proximity of the big adductor in a young Sicilian man. The patient, 34 years old, was admitted to the Department of Infectious and Tropical Diseases for ultrasonographic detection, with successive confirmation by magnetic resonance imaging, of an ovular mass (13 × 8 cm) in the big adductor of the left thigh, cyst-like, and containing several small cystic formations. Serological tests for hydatidosis gave negative results. A second drawing of blood was done 10 days after the first one and showed an increase in the antibody titer for hydatidosis. The patient was submitted to surgical excision of the lesion with perioperatory prophylaxis with albendazole. The histopathological examination of the bioptic material was not diriment in the diagnosis, therefore further tests were performed: additional serological tests for hydatidosis for the evaluation of IgE and IgG serotype (Western Blot and REAST), and molecular analysis of the excised material. These more specific serological tests gave positive results for hydatidosis, and the sequencing of the polymerase chain reaction products from the cyst evidenced E. granulosus DNA, genotype G1. Any post-surgery complications was observed during 6 following months. CONCLUSION: Cystic hydatidosis should always be considered in the differential diagnosis of any cystic mass, regardless of its location, also in epidemiological contests less suggestive of the disease. The diagnosis should be achieved by taking into consideration the clinical aspects, the epidemiology of the disease, the imaging and immunological tests but, as demonstrated in this case, without neglecting the numerous possibilities offered by new serological devices and modern day molecular biology techniques

Machine Learning in Melanoma Diagnosis. Limitations About to be Overcome

[spa] Antecedentes: La clasificación automática de imágenes es una rama prometedora del aprendi-zaje automático (de sus siglas en inglés Machine Learning [ML]), y es una herramienta útil enel diagnóstico de cáncer de piel. Sin embargo, poco se ha estudiado acerca de las limitacionesde su uso en la práctica clínica diaria.Objetivo: Determinar las limitaciones que existen en cuanto a la selección de imágenes usadaspara el análisis por ML de las neoplasias cutáneas, en particular del melanoma.Métodos: Se dise ̃nó un estudio de cohorte retrospectivo, donde se incluyeron de forma conse-cutiva 2.849 imágenes dermatoscópicas de alta calidad de tumores cutáneos para su valoraciónpor un sistema de ML, recogidas entre los a ̃nos 2010 y 2014. Cada imagen dermatoscópica fueclasificada según las características de elegibilidad para el análisis por ML.Resultados: De las 2.849 imágenes elegidas a partir de nuestra base de datos, 968 (34%) cum-plieron los criterios de inclusión. De los 528 melanomas, 335 (63,4%) fueron excluidos. Laausencia de piel normal circundante (40,5% de todos los melanomas de nuestra base de datos)y la ausencia de pigmentación (14,2%) fueron las causas más frecuentes de exclusión para elanálisis por ML.Discusión: Solo el 36,6% de nuestros melanomas se consideraron aceptables para el análisispor sistemas de ML de última generación. Concluimos que los futuros sistemas de ML deberánser entrenados a partir de bases de datos más grandes que incluyan imágenes representativasde la práctica clínica habitual. Afortunadamente, muchas de estas limitaciones están siendosuperadas gracias a los avances realizados recientemente por la comunidad científica, como seha demostrado en trabajos recientes. [eng] Background: Automated image classification is a promising branch of machine learning (ML)useful for skin cancer diagnosis, but little has been determined about its limitations for generalusability in current clinical practice.Objective: To determine limitations in the selection of skin cancer images for ML analysis,particularly in melanoma.Methods: Retrospective cohort study design, including 2,849 consecutive high-quality dermos-copy images of skin tumors from 2010 to 2014, for evaluation by a ML system. Each dermoscopyimage was assorted according to its eligibility for ML analysis.Results: Of the 2,849 images chosen from our database, 968 (34%) met the inclusion criteriafor analysis by the ML system. Only 64.7% of nevi and 36.6% of melanoma met the inclusioncriteria. Of the 528 melanomas, 335 (63.4%) were excluded. An absence of normal surroundingskin (40.5% of all melanomas from our database) and absence of pigmentation (14.2%) were themost common reasons for exclusion from ML analysis.Discussion: Only 36.6% of our melanomas were admissible for analysis by state-of-the-art MLsystems. We conclude that future ML systems should be trained on larger datasets which includerelevant non-ideal images from lesions evaluated in real clinical practice. Fortunately, many ofthese limitations are being overcome by the scientific community as recent works show

Position statement of the EADV Artificial Intelligence (AI) Task Force on AI‐assisted smartphone apps and web‐based services for skin disease

Background: As the use of smartphones continues to surge globally, mobile applications (apps) have become a powerful tool for healthcare engagement. Prominent among these are dermatology apps powered by Artificial Intelligence (AI), which provide immediate diagnostic guidance and educational resources for skin diseases, including skin cancer. Objective: This article, authored by the EADV AI Task Force, seeks to offer insights and recommendations for the present and future deployment of AI‐assisted smartphone applications (apps) and web‐based services for skin diseases with emphasis on skin cancer detection.MethodsAn initial position statement was drafted on a comprehensive literature review, which was subsequently refined through two rounds of digital discussions and meticulous feedback by the EADV AI Task Force, ensuring its accuracy, clarity and relevance. Results: Eight key considerations were identified, including risks associated with inaccuracy and improper user education, a decline in professional skills, the influence of non‐medical commercial interests, data security, direct and indirect costs, regulatory approval and the necessity of multidisciplinary implementation. Following these considerations, three main recommendations were formulated: (1) to ensure user trust, app developers should prioritize transparency in data quality, accuracy, intended use, privacy and costs; (2) Apps and web‐based services should ensure a uniform user experience for diverse groups of patients; (3) European authorities should adopt a rigorous and consistent regulatory framework for dermatology apps to ensure their safety and accuracy for users. Conclusions: The utilisation of AI‐assisted smartphone apps and web‐based services in diagnosing and treating skin diseases has the potential to greatly benefit patients in their dermatology journeys. By prioritising innovation, fostering collaboration and implementing effective regulations, we can ensure the successful integration of these apps into clinical practice

Global patterns of care in advanced stage mycosis fungoides/Sezary syndrome: a multicenter retrospective follow-up study from the Cutaneous Lymphoma International Consortium

ABSTRACT Background Advanced-stage mycosis fungoides (MF)/Sezary syndrome (SS) patients are weighted by an unfavorable prognosis and share an unmet clinical need of effective treatments. International guidelines are available detailing treatment options for the different stages but without recommending treatments in any particular order due to lack of comparative trials. The aims of this second CLIC study were to retrospectively analyze the pattern of care worldwide for advanced-stage MF/SS patients, the distribution of treatments according to geographical areas (USA versus non-USA), and whether the heterogeneity of approaches has potential impact on survival. Patients and methods This study included 853 patients from 21 specialist centers (14 European, 4 USA, 1 each Australian, Brazilian, and Japanese). Results Heterogeneity of treatment approaches was found, with up to 24 different modalities or combinations used as first-line and 36% of patients receiving four or more treatments. Stage IIB disease was most frequently treated by total-skin-electron-beam radiotherapy, bexarotene and gemcitabine; erythrodermic and SS patients by extracorporeal photochemotherapy, and stage IVA2 by polychemotherapy. Significant differences were found between USA and non-USA centers, with bexarotene, photopheresis and histone deacetylase inhibitors most frequently prescribed for first-line treatment in USA while phototherapy, interferon, chlorambucil and gemcitabine in non-USA centers. These differences did not significantly impact on survival. However, when considering death and therapy change as competing risk events and the impact of first treatment line on both events, both monochemotherapy (SHR = 2.07) and polychemotherapy (SHR = 1.69) showed elevated relative risks. Conclusion This large multicenter retrospective study shows that there exist a large treatment heterogeneity in advanced MF/SS and differences between USA and non-USA centers but these were not related to survival, while our data reveal that chemotherapy as first treatment is associated with a higher risk of death and/or change of therapy and thus other therapeutic options should be preferable as first treatment approach