An investigation of biases in Patient Safety Indicator score distribution among hospital cohorts

Denman Research Forum- 2nd Place, Health Professions-ClinicalThe Centers for Medicare and Medicaid Services (CMS) have implemented a hospital reimbursement system that incentivizes payment proportional to the quality of care delivered and performance on certain metrics. One such metric is the Agency for Healthcare Research and Quality’s Patient Safety Indicator 90 (PSI-90). It is composed of eight individual indicators designed to flag adverse patient events that are potentially preventable, such as post-operative wound dehiscence and accidental lacerations. CMS publicly reports four of these individual PSI scores (6, 12, 14 and 15) in addition to the composite PSI-90. Previous studies question the PSIs’ validity beyond screening purposes and furthermore question the underlying administrative data’s ability to accurately and reliably flag such events. This study looks to analyze biases in PSI score distribution for hospitals depending on teaching status, differences in patient demographics and lastly, interactions between teaching status and patient demographic factors and their ability to account for differences in PSI rates. Significant differences were found between teaching and non-teaching hospitals for PSIs 6, 12, 15 and 90 (p<0.01). Inpatient volume and patient severity (p<0.01) were found to be significantly different between teaching status cohorts. Lastly, significant differences in PSI scores were found between patient severity quartiles for PSI 6, 15 and 90 (p<0.05) and between socio-economic quartiles for PSI 6, 12, 15 and 90 (p<0.05); but interaction between patient severity and teaching status was only significant for PSI 90 (p<0.05) and between socioeconomic and teaching statuses for PSI 6 (p<0.05). These results indicate current PSI score distributions may be biased against teaching hospitals for 4 out of 5 PSI measures. Further studies will involve assessing the adequacy of risk-adjustment methodology for PSI metrics. Until then, use of PSI metrics to determine federal reimbursement can lead to bias against teaching hospitals.A three-year embargo was granted for this item.Academic Major: Health Information Management and System

What to do about poor clinical performance in clinical trials

The performance of individual clinicians is being monitored as never before. Su Mason and colleagues discuss the implications of this for clinical trials and recommend what should happen if during a trial the performance of one clinician or one centre is identified as being particularly poor. Tom Treasure, a surgeon, wants the monitoring to be done fairly and to take account of the complexities of clinical practice; and Heather Goodare, a patient, wants to be told when things go wrong. The Department of Health in England has issued guidelines for research governance stating that healthcare organisations remain responsible for the quality of all aspects of patients' care whether or not some aspects of the care are part of a research study.1 We discuss how this obligation can be met in multicentre trials, given that data on the performance of clinicians are held by the trial management team, not by the host organisation

Beyond Volume: The Impact of Complex Healthcare Data on the Machine Learning Pipeline

From medical charts to national census, healthcare has traditionally operated under a paper-based paradigm. However, the past decade has marked a long and arduous transformation bringing healthcare into the digital age. Ranging from electronic health records, to digitized imaging and laboratory reports, to public health datasets, today, healthcare now generates an incredible amount of digital information. Such a wealth of data presents an exciting opportunity for integrated machine learning solutions to address problems across multiple facets of healthcare practice and administration. Unfortunately, the ability to derive accurate and informative insights requires more than the ability to execute machine learning models. Rather, a deeper understanding of the data on which the models are run is imperative for their success. While a significant effort has been undertaken to develop models able to process the volume of data obtained during the analysis of millions of digitalized patient records, it is important to remember that volume represents only one aspect of the data. In fact, drawing on data from an increasingly diverse set of sources, healthcare data presents an incredibly complex set of attributes that must be accounted for throughout the machine learning pipeline. This chapter focuses on highlighting such challenges, and is broken down into three distinct components, each representing a phase of the pipeline. We begin with attributes of the data accounted for during preprocessing, then move to considerations during model building, and end with challenges to the interpretation of model output. For each component, we present a discussion around data as it relates to the healthcare domain and offer insight into the challenges each may impose on the efficiency of machine learning techniques.Comment: Healthcare Informatics, Machine Learning, Knowledge Discovery: 20 Pages, 1 Figur