Evaluation of Combined Artificial Intelligence and Radiologist Assessment to Interpret Screening Mammograms

Importance: Mammography screening currently relies on subjective human interpretation. Artificial intelligence (AI) advances could be used to increase mammography screening accuracy by reducing missed cancers and false positives. Objective: To evaluate whether AI can overcome human mammography interpretation limitations with a rigorous, unbiased evaluation of machine learning algorithms. Design, Setting, and Participants: In this diagnostic accuracy study conducted between September 2016 and November 2017, an international, crowdsourced challenge was hosted to foster AI algorithm development focused on interpreting screening mammography. More than 1100 participants comprising 126 teams from 44 countries participated. Analysis began November 18, 2016. Main Outcomes and Measurements: Algorithms used images alone (challenge 1) or combined images, previous examinations (if available), and clinical and demographic risk factor data (challenge 2) and output a score that translated to cancer yes/no within 12 months. Algorithm accuracy for breast cancer detection was evaluated using area under the curve and algorithm specificity compared with radiologists' specificity with radiologists' sensitivity set at 85.9% (United States) and 83.9% (Sweden). An ensemble method aggregating top-performing AI algorithms and radiologists' recall assessment was developed and evaluated. Results: Overall, 144 231 screening mammograms from 85 580 US women (952 cancer positive ≤12 months from screening) were used for algorithm training and validation. A second independent validation cohort included 166 578 examinations from 68 008 Swedish women (780 cancer positive). The top-performing algorithm achieved an area under the curve of 0.858 (United States) and 0.903 (Sweden) and 66.2% (United States) and 81.2% (Sweden) specificity at the radiologists' sensitivity, lower than community-practice radiologists' specificity of 90.5% (United States) and 98.5% (Sweden). Combining top-performing algorithms and US radiologist assessments resulted in a higher area under the curve of 0.942 and achieved a significantly improved specificity (92.0%) at the same sensitivity. Conclusions and Relevance: While no single AI algorithm outperformed radiologists, an ensemble of AI algorithms combined with radiologist assessment in a single-reader screening environment improved overall accuracy. This study underscores the potential of using machine learning methods for enhancing mammography screening interpretation

Development of precise tremor assessment software to aid deep brain stimulation parameter optimization

The Steered Response Power with PHAT transform (SRP-PHAT) or Global Coherence Field (GCF), has become a standard method for acoustic source localization, thanks to their simplicity, computational inexpensiveness and robustness against mid-high reverberation. However, originally formulated for the single source localization case, it does not apply satisfactorily to the multiple source case. In this paper, we analyze the structure of the spatial function and reshape it according to a generic multidimensional metric. We show that traditional functions are based on the L1 norm which is prone to generate ambiguous locations with high likelihood (i.e. ghosts). A more generic multidimensional kernel based on higher norms and on a partitioned representation of the cross-power spectrum is introduced, which better exploits the source sparseness in the discrete time-frequency domain. Evaluation results over simulated data show that the new spatial functions considerably improve the detection of multiple competing sources in both spatial and multidimensional TDOA domains

Validation of a precision tremor measurement system for multiple sclerosis

Background: Tremor is a debilitating symptom of Multiple Sclerosis (MS). Little is known about its pathophysiology and treatments are limited. Clinical trials investigating new interventions often rely on subjective clinical rating scales to provide supporting evidence of efficacy. New Method: We present a novel instrument (TREMBAL) which uses electromagnetic motion capture technology to quantify MS tremor. We aim to validate TREMBAL by comparison to clinical ratings using regression modelling with 310 samples of tremor captured from 13 MS participants who performed five different hand exercises during several follow-up visits. Minimum detectable change (MDC) and test-retest reliability were calculated and comparisons were made between MS tremor and data from 12 healthy volunteers. Results: Velocity of the index finger was most congruent with clinical observation. Regression modelling combining different features, sensor configurations, and labelling exercises did not improve results. TREMBAL MDC was 84% of its initial measurement compared to 91% for the clinical rating. Intra-class correlations for test-retest reliability were 0.781 for TREMBAL and 0.703 for clinical ratings. Tremor was lower (p = 0.002) in healthy subjects. Comparison with Existing Methods: Subjective scales have low sensitivity, suffer from ceiling effects, and mitigation against inter-rater variability is challenging. Inertial sensors are ubiquitous, however, their output is nonlinearly related to tremor frequency, compensation is required for gravitational artefacts, and their raw data cannot be intuitively comprehended. Conclusions: TREMBAL, compared with clinical ratings, gave measures in agreement with clinical observation, had marginally lower MDC, and similar test-retest reliability

Evaluation of Combined Artificial Intelligence and Radiologist Assessment to Interpret Screening Mammograms

10.1001/jamanetworkopen.2020.0265JAMA network open33e20026