56 research outputs found

    Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge

    Full text link
    Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy.We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25?331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use.64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed.We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice.Melanoma Research Alliance and La Marató de TV3.Copyright © 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY-NC-ND 4.0 license. Published by Elsevier Ltd.. All rights reserved

    Rapid, complex adaption of transmitted HIV-1 full-length genomes in subtype C-infected individuals with differing disease progression.

    Get PDF
    CAPRISA 2013.Objective(s): There is limited information on full-length genome sequences and the early evolution of transmitted HIV-1 subtype C viruses, which constitute the majority of viruses spread in Africa. The purpose of this study was to characterize the earliest changes across the genome of subtype C viruses following transmission, to better understand early control of viremia. Design: We derived the near full-length genome sequence responsible for clinical infection from five HIV subtype C-infected individuals with different disease progression profiles and tracked adaptation to immune responses in the first 6 months of infection. Methods: Near full-length genomes were generated by single genome amplification and direct sequencing. Sequences were analyzed for amino acid mutations associated with cytotoxic T lymphocyte (CTL) or antibody-mediated immune pressure, and for reversion. Results: Fifty-five sequence changes associated with adaptation to the new host were identified, with 38% attributed to CTL pressure, 35% to antibody pressure, 16% to reversions and the remainder were unclassified. Mutations in CTL epitopes were most frequent in the first 5 weeks of infection, with the frequency declining over time with the decline in viral load. CTL escape predominantly occurred in nef, followed by pol and env. Shuffling/toggling of mutations was identified in 81% of CTL epitopes, with only 7% reaching fixation within the 6-month period. Conclusion: There was rapid virus adaptation following transmission, predominantly driven by CTL pressure, with most changes occurring during high viremia. Rapid escape and complex escape pathways provide further challenges for vaccine protection

    Intra- and Inter-clade Cross-reactivity by HIV-1 Gag Specific T-Cells Reveals Exclusive and Commonly Targeted Regions: Implications for Current Vaccine Trials

    Get PDF
    The genetic diversity of HIV-1 across the globe is a major challenge for developing an HIV vaccine. To facilitate immunogen design, it is important to characterize clusters of commonly targeted T-cell epitopes across different HIV clades. To address this, we examined 39 HIV-1 clade C infected individuals for IFN-γ Gag-specific T-cell responses using five sets of overlapping peptides, two sets matching clade C vaccine candidates derived from strains from South Africa and China, and three peptide sets corresponding to consensus clades A, B, and D sequences. The magnitude and breadth of T-cell responses against the two clade C peptide sets did not differ, however clade C peptides were preferentially recognized compared to the other peptide sets. A total of 84 peptides were recognized, of which 19 were exclusively from clade C, 8 exclusively from clade B, one peptide each from A and D and 17 were commonly recognized by clade A, B, C and D. The entropy of the exclusively recognized peptides was significantly higher than that of commonly recognized peptides (p = 0.0128) and the median peptide processing scores were significantly higher for the peptide variants recognized versus those not recognized (p = 0.0001). Consistent with these results, the predicted Major Histocompatibility Complex Class I IC50 values were significantly lower for the recognized peptide variants compared to those not recognized in the ELISPOT assay (p<0.0001), suggesting that peptide variation between clades, resulting in lack of cross-clade recognition, has been shaped by host immune selection pressure. Overall, our study shows that clade C infected individuals recognize clade C peptides with greater frequency and higher magnitude than other clades, and that a selection of highly conserved epitope regions within Gag are commonly recognized and give rise to cross-clade reactivities

    Cognitive testing of physical activity and acculturation questions in recent and long-term Latino immigrants

    Get PDF
    <p>Abstract</p> <p>Background</p> <p>We ascertained the degree to which language (English versus Spanish), and residence time in the US influence responses to survey questions concerning two topics: self-reported acculturation status, and recent physical activity (PA). This topic is likely to be of general interest because of growing numbers of immigrants in countries worldwide.</p> <p>Methods</p> <p>We carried out qualitative (cognitive) interviews of survey items on acculturation and physical activity on 27 Latino subjects from three groups: (a) In Spanish, of those of low residence time (less than five years living in the U.S.) (n = 9); (b) In Spanish, of those of high residence time (15 or more years in the U.S) (n = 9); and (c) in English, of those of high residence time (n = 9).</p> <p>Results</p> <p>There were very few language translation problems; general question design defects and socio-cultural challenges to survey responses were more common. Problems were found for both acculturation and PA questions, with distinct problem types for the two question areas. Residence time/language group was weakly associated with overall frequency of problems observed: low residence time/Spanish (86%), high residence time/Spanish (67%), and English speaking groups (62%).</p> <p>Conclusions</p> <p>Standardized survey questions related to acculturation and physical activity present somewhat different cognitive challenges. For PA related questions, problems with such questions were similar regardless of subject residence time or language preference. For acculturation related questions, residence time/language or education level influenced responses to such questions. These observations should help in the interpretation of survey results for culturally diverse populations.</p
    • …
    corecore