A roadmap to fair and trustworthy prediction model validation in healthcare

A prediction model is most useful if it generalizes beyond the development data with external validations, but to what extent should it generalize remains unclear. In practice, prediction models are externally validated using data from very different settings, including populations from other health systems or countries, with predictably poor results. This may not be a fair reflection of the performance of the model which was designed for a specific target population or setting, and may be stretching the expected model generalizability. To address this, we suggest to externally validate a model using new data from the target population to ensure clear implications of validation performance on model reliability, whereas model generalizability to broader settings should be carefully investigated during model development instead of explored post-hoc. Based on this perspective, we propose a roadmap that facilitates the development and application of reliable, fair, and trustworthy artificial intelligence prediction models.Comment: 12 pages, 2 figure

Risk Stratification with Extreme Learning Machine: A Retrospective Study on Emergency Department Patients

This paper presents a novel risk stratification method using extreme learning machine (ELM). ELM was integrated into a scoring system to identify the risk of cardiac arrest in emergency department (ED) patients. The experiments were conducted on a cohort of 1025 critically ill patients presented to the ED of a tertiary hospital. ELM and voting based ELM (V-ELM) were evaluated. To enhance the prediction performance, we proposed a selective V-ELM (SV-ELM) algorithm. The results showed that ELM based scoring methods outperformed support vector machine (SVM) based scoring method in the receiver operation characteristic analysis

Emergency medical services key performance measurement in Asian cities

10.1186/s12245-015-0062-7International Journal of Emergency Medicine8

Predicting 30-Day Readmissions: Performance of the LACE Index Compared with a Regression Model among General Medicine Patients in Singapore

The LACE index (length of stay, acuity of admission, Charlson comorbidity index, CCI, and number of emergency department visits in preceding 6 months) derived in Canada is simple and may have clinical utility in Singapore to predict readmission risk. We compared the performance of the LACE index with a derived model in identifying 30-day readmissions from a population of general medicine patients in Singapore. Additional variables include patient demographics, comorbidities, clinical and laboratory variables during the index admission, and prior healthcare utilization in the preceding year. 5,862 patients were analysed and 572 patients (9.8%) were readmitted in the 30 days following discharge. Age, CCI, count of surgical procedures during index admission, white cell count, serum albumin, and number of emergency department visits in previous 6 months were significantly associated with 30-day readmission risk. The final logistic regression model had fair discriminative ability c-statistic of 0.650 while the LACE index achieved c-statistic of 0.628 in predicting 30-day readmissions. Our derived model has the advantage of being available early in the admission to identify patients at high risk of readmission for interventions. Additional factors predicting readmission risk and machine learning techniques should be considered to improve model performance

Time series analysis of demographic and temporal trends of tuberculosis in Singapore

Background: Singapore is an intermediate tuberculosis (TB) incidence country, with a recent rise in TB incidence from 2008, after a fall in incidence since 1998. This study identified population characteristics that were associated with the recent increase in TB cases, and built a predictive model of TB risk in Singapore. Methods: Retrospective time series analysis was used to study TB notification data collected from 1995 to 2011 from the Singapore Tuberculosis Elimination Program (STEP) registry. A predictive model was developed based on the data collected from 1995 to 2010 and validated using the data collected in 2011. Results: There was a significant difference in demographic characteristics between resident and non-resident TB cases. TB risk was higher in non-residents than in residents throughout the period. We found no significant association between demographic and macro-economic factors and annual incidence of TB with or without adjusting for the population-at-risk. Despite growing non-resident population, there was a significant decrease in the non-resident TB risk (p < 0.0001). However, there was no evidence of trend in the resident TB risk over this time period, though differences between different demographic groups were apparent with ethnic minorities experiencing higher incidence rates. Conclusion: The study found that despite an increasing size of non-resident population, TB risk among non-residents was decreasing at a rate of about 3% per year. There was an apparent seasonality in the TB reporting

Comparison of supraglottic airway versus endotracheal intubation for the pre-hospital treatment of out-of-hospital cardiac arrest

10.1186/cc10483Critical Care155R23

Cardiopulmonary resuscitation (CPR) training strategies in the times of COVID-19: A systematic literature review comparing different training methodologies

Background: Traditional, instructor led, in-person training of CPR skills has become more challenging due to COVID-19 pandemic. We compared the learning outcomes of standard in-person CPR training (ST) with alternative methods of training such as hybrid or online-only training (AT) on CPR performance, quality, and knowledge among laypersons with no previous CPR training.Methods: We searched PubMed and Google Scholar for relevant articles from January 1995 to May 2020. Covidence was used to review articles by two independent researchers. Effective Public Health Practice Project (EPHPP) Quality Assessment Tool was used to assess quality of the manuscripts.Results: Of the 978 articles screened, twenty met the final inclusion criteria. All included studies had an experimental design and moderate to strong global quality rating. The trainees in ST group performed better on calling 911, time to initiate chest compressions, hand placement and chest compression depth. Trainees in AT group performed better in assessing scene safety, calling for help, response time including initiating first rescue breathing, adequate ventilation volume, compression rates, shorter hands-off time, confidence, willingness to perform CPR, ability to follow CPR algorithm, and equivalent or better knowledge retention than standard teaching methodology.Conclusion: AT methods of CPR training provide an effective alternative to the standard in-person CPR for large scale public training