Man against machine reloaded : performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions

Copyright © 2019 European Society for Medical Oncology. Published by Elsevier Ltd. All rights reserved.Background: Convolutional neural networks (CNNs) efficiently differentiate skin lesions by image analysis. Studies comparing a market-approved CNN in a broad range of diagnoses to dermatologists working under less artificial conditions are lacking. Materials and methods: One hundred cases of pigmented/non-pigmented skin cancers and benign lesions were used for a two-level reader study in 96 dermatologists (level I: dermoscopy only; level II: clinical close-up images, dermoscopy, and textual information). Additionally, dermoscopic images were classified by a CNN approved for the European market as a medical device (Moleanalyzer Pro, FotoFinder Systems, Bad Birnbach, Germany). Primary endpoints were the sensitivity and specificity of the CNN's dichotomous classification in comparison with the dermatologists’ management decisions. Secondary endpoints included the dermatologists’ diagnostic decisions, their performance according to their level of experience, and the CNN's area under the curve (AUC) of receiver operating characteristics (ROC). Results: The CNN revealed a sensitivity, specificity, and ROC AUC with corresponding 95% confidence intervals (CI) of 95.0% (95% CI 83.5% to 98.6%), 76.7% (95% CI 64.6% to 85.6%), and 0.918 (95% CI 0.866–0.970), respectively. In level I, the dermatologists’ management decisions showed a mean sensitivity and specificity of 89.0% (95% CI 87.4% to 90.6%) and 80.7% (95% CI 78.8% to 82.6%). With level II information, the sensitivity significantly improved to 94.1% (95% CI 93.1% to 95.1%; P < 0.001), while the specificity remained unchanged at 80.4% (95% CI 78.4% to 82.4%; P = 0.97). When fixing the CNN's specificity at the mean specificity of the dermatologists’ management decision in level II (80.4%), the CNN's sensitivity was almost equal to that of human raters, at 95% (95% CI 83.5% to 98.6%) versus 94.1% (95% CI 93.1% to 95.1%); P = 0.1. In contrast, dermatologists were outperformed by the CNN in their level I management decisions and level I and II diagnostic decisions. More experienced dermatologists frequently surpassed the CNN's performance. Conclusions: Under less artificial conditions and in a broader spectrum of diagnoses, the CNN and most dermatologists performed on the same level. Dermatologists are trained to integrate information from a range of sources rendering comparative studies that are solely based on one single case image inadequate.publishersversionPeer reviewe

Differences in efficacy between intention-to-treat and per-protocol analyses for patients with psoriasis vulgaris and atopic dermatitis: clinical and pharmacoeconomic implications

BACKGROUND: Pharmacoeconomic outcome research is based on three criteria: (i) evaluation of objective therapeutic effects; (ii) quality of life; and (iii) treatment costs. Evaluation of therapeutic effect is mainly based on the results of clinical trials using objective clinical measures, e.g: Psoriasis Area and Severity Index (PASI) (score for psoriasis vulgaris) and the Severity Scoring of Atopic Dermatitis (SCORAD) (score for atopic dermatitis). In most studies, only results for a treatment-optimized subpopulation (patients treated according to the protocol) are presented in publications. The relevance of such data for daily routine therapy is doubtful. OBJECTIVES: Our purpose was to investigate the expected loss of effectiveness of switching from a clinical trial to daily routine therapy for the synchronous application of narrow-band ultraviolet (UV) B phototherapy (311 nm) and bathing in 10% Dead Sea salt solution (synchronous balneophototherapy) for patients with psoriasis vulgaris and atopic dermatitis. METHODS: We conducted a multicentre, uncontrolled observational study of outpatients. To achieve data for 'clinical trial' and 'daily routine' situations, two populations were compared: (i) all patients strictly treated according to the protocol (ATP) with no protocol deviations (data published in clinical trials), and (ii) all patients participating in the study who received active treatment at least once, despite treatment irregularities, non-compliance, early withdrawal or other protocol violations [intention-to-treat-population (ITT), model for 'daily routine']. RESULTS: A total of 2526 patients were included in the ITT analysis for psoriasis vulgaris (n = 487 for atopic dermatitis), of which 818 patients could be analysed according to protocol (n = 104 for atopic dermatitis). Striking differences in the therapeutic effect between both groups (ITT and ATP) were found using relative PASI and SCORAD score improvement: 11% (57% 'daily routine' vs. 68% in 'clinical trial') for psoriasis vulgaris and 16% (39% 'daily routine' vs. 55% 'clinical trial') for atopic dermatitis. The main reasons for excluding patients from the 'clinical trial' group were early study withdrawal in 29% (atopic dermatitis, 47%) of patients and fewer treatments per week than planned in the protocol in 24% (atopic dermatitis, 52%). CONCLUSIONS: Our data clearly indicate that for the prediction of the therapeutic effect for daily routine therapy the ITT data appear to be more relevant than the ATP results (i.e. those presented in clinical trials). Although these data are only a first step for evaluating the 'real' therapeutic effect of a treatment modality in daily routine, they seem to support the requirements for ITT analyses in efficacy studies and demonstrate the necessity of ITT data for pharmacoeconomic evaluation

Evaluation of a multicentre study of synchronous application of narrowband ultraviolet B phototherapy (TL-01) and bathing in Dead Sea salt solution for psoriasis vulgaris

The synchronous application of narrowband UVB phototherapy with 311 nm lamps (Philips TL-01) and bathing in Dead Sea salt solution was evaluated in a multicentre trial (n = 60) in outpatients suffering from psoriasis vulgaris. The study design consisted of an initial therapy phase of up to 35 treatments (three to five times a week) followed by maintenance therapy with up to 35 further applications (once or twice a week). Evaluation was performed separately for patients in according-to-protocol (ATP) (n = 280) and intention-to-treat (ITT) (n = 692) groups. An overall significant improvement of the Psoriasis Area and Severity Index (PASI) score (P < 0.05) could be shown for both groups during initial therapy with 71.4% improvement for ATP and 61% for ITT patients. The mean PASI for ATP (values for ITT in parentheses) was 17.7 (18.6) at baseline, 9.5 (10.7) after 20 applications and 5.2 (7.4) at the end of initial therapy. On average, ATP patients received 3.9 (3.5) applications per week with a cumulative irradiation dose of 19.5 J cm-2 (16.2 J cm-2). The most frequent side-effect was erythema, observed in 8.7% of the patients. Subjective evaluation of the therapy by the patients (n = 168) was excellent. Seventy-nine per cent of patients preferred the new treatment strategy in comparison with other previous therapies and 88% regarded this therapy as pleasant and comfortable. In conclusion, we could demonstrate a significant effect of therapy in both the ATP and the ITT groups for this new treatment system which imitates, as far as possible, the Dead Sea climatic conditions, with no severe side-effects and a high acceptance by the patients

Modular Synthesis of Dipyrroloquinolines: A Combined Synthetic and Mechanistic Study

A straightforward synthesis of [1,2-<i>a</i>]­[3′,2′-<i>c</i>]­dipyrroloquinolines has been developed generating up to eight new σ-bonds and five new stereogenic centers in a simple and modular one-pot operation. Generally good to excellent yields and moderate to good stereoselectivities in favor of the all-<i>cis</i> stereoisomer were observed. A detailed investigation combining synthetic studies, analytical measurements, and theoretical calculations has been conducted to elucidate the reaction mechanism using ESI- and liquid-beam IR-laser desorption mass spectrometry as well as DFT calculations. Key steps of this sequential transformation include a Lewis acid-catalyzed vinylogous Mukaiyama–Mannich reaction of <i>bis</i>(silyl) dienediolate <b>1</b> and a Brønsted acid-promoted Mannich–Pictet–Spengler reaction cascade reaction to complete the synthesis of the dipyrroloquinoline core of the target compounds

Liquid Beam Desorption Mass Spectrometry for the Investigation of Continuous Flow Reactions in Microfluidic Chips

In this work, we present the combination of microfluidic chips and mass spectrometry employing laser-induced liquid beam ionization/desorption. The developed system was evaluated with respect to stable beam generation and laser parameters as well as solvent compatibility. The device was exemplarily applied to study a vinylogous Mannich reaction performed in continuous flow on chip. Fast processes can be observed with this technique which in the future could be beneficial for studying intermediates or contribute to the elucidation of reaction mechanisms

Skin lesions of face and scalp - Classification by a market-approved convolutional neural network in comparison with 64 dermatologists.

BACKGROUND: The clinical differentiation of face and scalp lesions (FSLs) is challenging even for trained dermatologists. Studies comparing the diagnostic performance of a convolutional neural network (CNN) with dermatologists in FSL are lacking. METHODS: A market-approved CNN (Moleanalyzer-Pro, FotoFinder Systems) was used for binary classifications of 100 dermoscopic images of FSL. The same lesions were used in a two-level reader study including 64 dermatologists (level I: dermoscopy only; level II: dermoscopy, clinical close-up images, textual information). Primary endpoints were the CNN's sensitivity and specificity in comparison with the dermatologists' management decisions in level II. Generalizability of the CNN results was tested by using four additional external data sets. RESULTS: The CNN's sensitivity, specificity and ROC AUC were 96.2% [87.0%-98.9%], 68.8% [54.7%-80.1%] and 0.929 [0.880-0.978], respectively. In level II, the dermatologists' management decisions showed a mean sensitivity of 84.2% [82.2%-86.2%] and specificity of 69.4% [66.0%-72.8%]. When fixing the CNN's specificity at the dermatologists' mean specificity (69.4%), the CNN's sensitivity (96.2% [87.0%-98.9%]) was significantly higher than that of dermatologists (84.2% [82.2%-86.2%]; p < 0.001). Dermatologists of all training levels were outperformed by the CNN (all p < 0.001). In confirmation, the CNN's accuracy (83.0%) was significantly higher than dermatologists' accuracies in level II management decisions (all p < 0.001). The CNN's performance was largely confirmed in three additional external data sets but particularly showed a reduced specificity in one Australian data set including FSL on severely sun-damaged skin. CONCLUSIONS: When applied as an assistant system, the CNN's higher sensitivity at an equivalent specificity may result in an improved early detection of face and scalp skin cancers