Adverse events detection through global trigger tool methodology: results from a 5-year study in an italian hospital and opportunities to improve interrater reliability

Objective: Global Trigger Tool (GTT) has been proposed as a low-cost method to detect adverse events (AEs). The validity of the methodology has been questioned because of moderate interrater agreement. Continuous training has been suggested as a means to improve consistency over time. We present the main findings of the implementation of the Italian version of the GTT and evaluate efforts to improve the interrater reliability over time. Methods: The Italian version of the GTT was developed and implemented at the San Bonifacio Hospital, a 270-bed secondary care acute hospital in Verona, Italy. Ten clinical records randomly selected every 2 weeks were reviewed from 2009 to 2014. Two-stage interrater reliability assessment between team members was conducted on 2 subsamples of 50 clinical records before and after the implementation of specific review rules and staff training. Results: Among 1320 medical records reviewed, a total of 366 AEs were found with at least 1 AE on 20.2% of all discharges, 27.7 AEs/100 admissions, and 30.6 AEs/1000 patient-days. Adverse events with harm score E and F were respectively 58.2% (n = 213) and 38.8% (n = 142). First round interrater reliability was comparable with other international studies. The interrater agreement improved significantly after intervention ([kappa] interrater I = 0.52, [kappa] interrater II = 0.80, P < 0.001). Conclusions: Despite the improvements in the interrater consistency, overall results did not show any significant trend in AEs over time. Future studies may be directed to apply and adapt the GTT methodology to more specific settings to explore how to improve its sensitivity

A machine learning approach to predict healthcare-associated infections at intensive care unit admission: findings from the SPIN-UTI project

BACKGROUND: Identifying patients at higher risk of healthcare-associated infections (HAIs) in intensive care unit (ICU) represents a major challenge for public health. Machine learning could improve patient risk stratification and lead to targeted infection prevention and control interventions.AIM: To evaluate the performance of the Simplified Acute Physiology Score (SAPS) II for HAIs risk prediction in ICUs, using both traditional statistical and machine learning approaches.METHODS: We used data of 7827 patients from the "Italian Nosocomial Infections Surveillance in Intensive Care Units" project. The Support Vector Machines (SVM) algorithm was applied to classify patients according to sex, patient origin, non-surgical treatment for acute coronary disease, surgical intervention, SAPS II at admission, presence of invasive devices, trauma, impaired immunity, antibiotic therapy in 48 hours before ICU admission.FINDINGS: The performance of SAPS II for predicting the risk of HAIs provides a ROC (Receiver Operating Characteristics) curve with an AUC (Area Under the Curve) of 0.612 (p<0.001) and an accuracy of 56%. Considering SAPS II along with other characteristics at ICU admission, we found an accuracy of the SVM classifier of 88% and an AUC of 0.90 (p<0.001) for the test set. In line, the predictive ability was lower when considering the same SVM model but removing the SAPS II variable (accuracy= 78% and AUC= 0.66).CONCLUSIONS: Our study suggested the SVM model as a tool to early predict patients at higher risk of HAI at ICU admission