Artificial Intelligence in Clinical Decision Support : Challenges for Evaluating AI and Practical Implications

OBJECTIVES This paper draws attention to: i) key considerations for evaluating artificial intelligence (AI) enabled clinical decision support; and ii) challenges and practical implications of AI design, development, selection, use, and ongoing surveillance. METHOD A narrative review of existing research and evaluation approaches along with expert perspectives drawn from the International Medical Informatics Association (IMIA) Working Group on Technology Assessment and Quality Development in Health Informatics and the European Federation for Medical Informatics (EFMI) Working Group for Assessment of Health Information Systems. RESULTS There is a rich history and tradition of evaluating AI in healthcare. While evaluators can learn from past efforts, and build on best practice evaluation frameworks and methodologies, questions remain about how to evaluate the safety and effectiveness of AI that dynamically harness vast amounts of genomic, biomarker, phenotype, electronic record, and care delivery data from across health systems. This paper first provides a historical perspective about the evaluation of AI in healthcare. It then examines key challenges of evaluating AI-enabled clinical decision support during design, development, selection, use, and ongoing surveillance. Practical aspects of evaluating AI in healthcare, including approaches to evaluation and indicators to monitor AI are also discussed. CONCLUSION Commitment to rigorous initial and ongoing evaluation will be critical to ensuring the safe and effective integration of AI in complex sociotechnical settings. Specific enhancements that are required for the new generation of AI-enabled clinical decision support will emerge through practical application

Does a SLAP lesion affect shoulder muscle recruitment as measured by EMG activity during a rugby tackle?

Background: The study objective was to assess the influence of a SLAP lesion on onset of EMG activity in shoulder muscles during a front on rugby football tackle within professional rugby players. Methods: Mixed cross-sectional study evaluating between and within group differences in EMG onset times. Testing was carried out within the physiotherapy department of a university sports medicine clinic. The test group consisted of 7 players with clinically diagnosed SLAP lesions, later verified on arthroscopy. The reference group consisted of 15 uninjured and full time professional rugby players from within the same playing squad. Controlled tackles were performed against a tackle dummy. Onset of EMG activity was assessed from surface EMG of Pectorialis Major, Biceps Brachii, Latissimus Dorsi, Serratus Anterior and Infraspinatus muscles relative to time of impact. Analysis of differences in activation timing between muscles and limbs (injured versus non-injured side and non injured side versus matched reference group). Results: Serratus Anterior was activated prior to all other muscles in all (P = 0.001-0.03) subjects. In the SLAP injured shoulder Biceps was activated later than in the non-injured side. Onset times of all muscles of the noninjured shoulder in the injured player were consistently earlier compared with the reference group. Whereas, within the injured shoulder, all muscle activation timings were later than in the reference group. Conclusions: This study shows that in shoulders with a SLAP lesion there is a trend towards delay in activation time of Biceps and other muscles with the exception of an associated earlier onset of activation of Serratus anterior, possibly due to a coping strategy to protect glenohumeral stability and thoraco-scapular stability. This trend was not statistically significant in all cases

Diagnostic reliability of magnetic resonance imaging for central nervous system syndromes in systemic lupus erythematosus: a prospective cohort study

<p>Abstract</p> <p>Background</p> <p>Previous studies of magnetic resonance imaging (MRI) as a diagnostic tool for central nervous system (CNS) syndromes in systemic lupus erythematosus (SLE) contained several limitations such as study design, number of enrolled patients, and definition of CNS syndromes. We overcame these problems and statistically evaluated the diagnostic values of abnormal MRI signals and their chronological changes in CNS syndromes of SLE.</p> <p>Methods</p> <p>We prospectively studied 191 patients with SLE, comparing those with (n = 57) and without (n = 134) CNS syndrome. CNS syndromes were characterized using the American College of Rheumatology case definitions.</p> <p>Results</p> <p>Any abnormal MRI signals were more frequently observed in subjects in the CNS group (n = 25) than in the non-CNS group (n = 32) [relative risk (RR), 1.7; 95% confidence interval (CI), 1.1-2.7; <it>p </it>= 0.016] and the positive and negative predictive values for the diagnosis of CNS syndrome were 42% and 76%, respectively. Large abnormal MRI signals (ø ≥ 10 mm) were seen only in the CNS group (n = 7; RR, 3.7; CI, 2.9-4.7; <it>p </it>= 0.0002), whereas small abnormal MRI signals (ø < 10 mm) were seen in both groups with no statistical difference. Large signals always paralleled clinical outcome (<it>p </it>= 0.029), whereas small signals did not (<it>p </it>= 1.000).</p> <p>Conclusions</p> <p>Abnormal MRI signals, which showed statistical associations with CNS syndrome, had insufficient diagnostic values. A large MRI signal was, however, useful as a diagnostic and surrogate marker for CNS syndrome of SLE, although it was less common.</p

Utilisation of an operative difficulty grading scale for laparoscopic cholecystectomy

Background A reliable system for grading operative difficulty of laparoscopic cholecystectomy would standardise description of findings and reporting of outcomes. The aim of this study was to validate a difficulty grading system (Nassar scale), testing its applicability and consistency in two large prospective datasets. Methods Patient and disease-related variables and 30-day outcomes were identified in two prospective cholecystectomy databases: the multi-centre prospective cohort of 8820 patients from the recent CholeS Study and the single-surgeon series containing 4089 patients. Operative data and patient outcomes were correlated with Nassar operative difficultly scale, using Kendall’s tau for dichotomous variables, or Jonckheere–Terpstra tests for continuous variables. A ROC curve analysis was performed, to quantify the predictive accuracy of the scale for each outcome, with continuous outcomes dichotomised, prior to analysis. Results A higher operative difficulty grade was consistently associated with worse outcomes for the patients in both the reference and CholeS cohorts. The median length of stay increased from 0 to 4 days, and the 30-day complication rate from 7.6 to 24.4% as the difficulty grade increased from 1 to 4/5 (both p < 0.001). In the CholeS cohort, a higher difficulty grade was found to be most strongly associated with conversion to open and 30-day mortality (AUROC = 0.903, 0.822, respectively). On multivariable analysis, the Nassar operative difficultly scale was found to be a significant independent predictor of operative duration, conversion to open surgery, 30-day complications and 30-day reintervention (all p < 0.001). Conclusion We have shown that an operative difficulty scale can standardise the description of operative findings by multiple grades of surgeons to facilitate audit, training assessment and research. It provides a tool for reporting operative findings, disease severity and technical difficulty and can be utilised in future research to reliably compare outcomes according to case mix and intra-operative difficulty

The relation between neuromechanical parameters and Ashworth score in stroke patients

Quantifying increased joint resistance into its contributing factors i.e. stiffness and viscosity ("hypertonia") and stretch reflexes ("hyperreflexia") is important in stroke rehabilitation. Existing clinical tests, such as the Ashworth Score, do not permit discrimination between underlying tissue and reflexive (neural) properties. We propose an instrumented identification paradigm for early and tailor made interventions.BioMechanical EngineeringMechanical, Maritime and Materials Engineerin

Population‐based cohort study of outcomes following cholecystectomy for benign gallbladder diseases

Background The aim was to describe the management of benign gallbladder disease and identify characteristics associated with all‐cause 30‐day readmissions and complications in a prospective population‐based cohort. Methods Data were collected on consecutive patients undergoing cholecystectomy in acute UK and Irish hospitals between 1 March and 1 May 2014. Potential explanatory variables influencing all‐cause 30‐day readmissions and complications were analysed by means of multilevel, multivariable logistic regression modelling using a two‐level hierarchical structure with patients (level 1) nested within hospitals (level 2). Results Data were collected on 8909 patients undergoing cholecystectomy from 167 hospitals. Some 1451 cholecystectomies (16·3 per cent) were performed as an emergency, 4165 (46·8 per cent) as elective operations, and 3293 patients (37·0 per cent) had had at least one previous emergency admission, but had surgery on a delayed basis. The readmission and complication rates at 30 days were 7·1 per cent (633 of 8909) and 10·8 per cent (962 of 8909) respectively. Both readmissions and complications were independently associated with increasing ASA fitness grade, duration of surgery, and increasing numbers of emergency admissions with gallbladder disease before cholecystectomy. No identifiable hospital characteristics were linked to readmissions and complications. Conclusion Readmissions and complications following cholecystectomy are common and associated with patient and disease characteristics

Contrasting resource allocation patterns in Sedum lanceolatum Torr.: Biomass versus energy estimates

Biomass determinations and microbomb calorimetry were used to assess resource allocation in Sedum lanceolatum Torr. between 2,257 and 3,726 m above sea level in the Front Range of the Rocky Mountains, Colorado, USA. In general, energy values did not differ within a tissue among sites, but did differ among tissue types. Flowers and leaves had the greatest energy content per gram ashfree dry weight. Total kilojoules per plant were homogeneous along the elevational gradient.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/47753/1/442_2004_Article_BF00379785.pd