Asking informed consent may lead to significant participation bias and suboptimal cardiovascular risk management in learning healthcare systems

BACKGROUND: The Utrecht Cardiovascular Cohort - CardioVascular Risk Management (UCC-CVRM) was set up as a learning healthcare system (LHS), aiming at guideline based cardiovascular risk factor measurement in all patients in routine clinical care. However, not all patients provided informed consent, which may lead to participation bias. We aimed to study participation bias in a LHS by assessing differences in and completeness of cardiovascular risk management (CVRM) indicators in electronic health records (EHRs) of consenting, non-consenting, and non-responding patients, using the UCC-CVRM as an example. METHODS: All patients visiting the University Medical Center Utrecht for first time evaluation of a(n) (a)symptomatic vascular disease or condition were invited to participate. Routine care data was collected in the EHR and an informed consent was asked. Differences in patient characteristics were compared between consent groups. We performed multivariable logistic regression to identify determinants of non-consent. We used multinomial regression for an exploratory analysis for the determinants of non-response. Presence of CVRM indicators were compared between consent groups. A waiver (19/641) was obtained from our ethics committee. RESULTS: Out of 5730 patients invited, 2378 were consenting, 1907 non-consenting, and 1445 non-responding. Non-consent was related to young and old age, lower education level, lower BMI, physical activity and haemoglobin levels, higher heartrate, cardiovascular disease history and absence of proteinuria. Non-response increased with young and old age, higher education level, physical activity, HbA1c and decreased with lower levels of haemoglobin, BMI, and systolic blood pressure. Presence of CVRM indicators was 5-30% lower in non-consenting patients and even lower in non-responding patients, compared to consenting patients. Non-consent and non-response varied across specialisms. CONCLUSIONS: A traditional informed consent procedure in a LHS may lead to participation bias and potentially to suboptimal CVRM, which is detrimental for feedback on findings in a LHS. This underlines the importance of reassessing the informed consent procedure in a LHS

Recommendations for IVDR compliant in-house software development in clinical practice, a how-to paper with three use cases

OBJECTIVES: The In Vitro Diagnostics Regulation (IVDR) will be effective in May 2022 by which in-house developed tests need to apply to the general safety and performance requirements defined in Annex I of the IVDR ruling. Yet, article 16 from Annex I about software can be hard to interpret and implement, particularly as laboratories are unfamiliar with quality standards for software development. METHODS: In this paper we provide recommendations on organizational structure, standards to use, and documentation, for IVDR compliant in-house software development. RESULTS: A practical insight is offered into novel standard operating procedures using three examples: an Excel file with a formula to calculate the pharmacokinetics of tacrolimus and to calculate the new dose, a rule for automated diagnosis of acute kidney injury and a bioinformatics pipeline for DNA variant calling. CONCLUSIONS: We recommend multidisciplinary development teams supported by higher management, use of ISO-15189 in synergy with IEC-62304, and concise documentation that includes intended purpose, classification, requirement management, risk management, verification and validation, configuration management and references to clinical or performance evidence

The Applied Data Analytics in Medicine Program: Lessons Learned From Four Years' Experience With Personalizing Health Care in an Academic Teaching Hospital

The University Medical Center (UMC) Utrecht piloted a hospital-wide innovation data analytics program over the past 4 years. The goal was, based on available data and innovative data analytics methodologies, to answer clinical questions to improve patient care. In this viewpoint, we aimed to support and inspire others pursuing similar efforts by sharing the three principles of the program: the data analytics value chain (data, insight, action, value), the innovation funnel (structured innovation approach with phases and gates), and the multidisciplinary team (patients, clinicians, and data scientists). We also discussed our most important lessons learned: the importance of a clinical question, collaboration challenges between health care professionals and different types of data scientists, the win-win result of our collaboration with external partners, the prerequisite of available meaningful data, the (legal) complexity of implementation, organizational power, and the embedding of collaborative efforts in the health care system as a whole

A Perspective on a Quality Management System for AI/ML-Based Clinical Decision Support in Hospital Care

Although many artificial intelligence (AI) and machine learning (ML) based algorithms are being developed by researchers, only a small fraction has been implemented in clinical-decision support (CDS) systems for clinical care. Healthcare organizations experience significant barriers implementing AI/ML models for diagnostic, prognostic, and monitoring purposes. In this perspective, we delve into the numerous and diverse quality control measures and responsibilities that emerge when moving from AI/ML-model development in a research environment to deployment in clinical care. The Sleep-Well Baby project, a ML-based monitoring system, currently being tested at the neonatal intensive care unit of the University Medical Center Utrecht, serves as a use-case illustrating our personal learning journey in this field. We argue that, in addition to quality assurance measures taken by the manufacturer, user responsibilities should be embedded in a quality management system (QMS) that is focused on life-cycle management of AI/ML-CDS models in a medical routine care environment. Furthermore, we highlight the strong similarities between AI/ML-CDS models and in vitro diagnostic devices and propose to use ISO15189, the quality guideline for medical laboratories, as inspiration when building a QMS for AI/ML-CDS usage in the clinic. We finally envision a future in which healthcare institutions run or have access to a medical AI-lab that provides the necessary expertise and quality assurance for AI/ML-CDS implementation and applies a QMS that mimics the ISO15189 used in medical laboratories

Whetting the SWORD: Detecting Workarounds by Using Active Learning and Logistic Regression

In many organizations, especially in healthcare, workers may work around prescribed procedures. Detecting these workarounds can give insights into difficulties concerning the procedures, which in turn can be used to improve them. Previous studies have shown that workarounds may be discovered from an event log using a set of predefined patterns such as the duration of a trace or the number of resources involved in one. However, domain experts may find it difficult to evaluate and monitor results if there are multiple patterns that indicate workarounds. Training a model that merges the features is often difficult because there are no available datasets covering workarounds. Labeling traces generally requires a lot of time from domain experts. In addition, this would have to be repeated for every new domain, company, or even department since the types of workarounds that occur may differ strongly between them. In this work, we propose to combine the features using a Logistic Regression model and train through Active Learning. In a case study at a hospital, we find that after training the model on only 10 to 15 traces, it stabilizes with an approximate F1 score of .75. This shows that we create and train a model that can detect workarounds well without requiring a large amount of labeled data or a lot of time from a domain expert

Routine Lupus Anticoagulant Sensitive aPTT Testing Can Prevent Unnecessary LA Testing

Even though routine screening of the general hospital population is discouraged, medical laboratories may use a “lupus sensitive” activated partial thromboplastin time test (aPTT) with phospholipid concentrations that are susceptible to inhibition by lupus anticoagulant (LA), to screen for the presence of LA. If deemed necessary, follow-up testing according to ISTH guidelines may be performed. However, LA testing is a laborious and time-consuming effort that is often not readily available due to a lack of automation and/or temporary unavailability of experienced staff. In contrast, the aPTT is a fully automated test that is available 24/7 in almost all medical laboratories and is easily interpreted with the use of reference ranges. In addition to clinical signs, the result of an LA sensitive aPTT may thus be used to lower the suspicion of the presence of LA and reduce costly follow-up testing. In this study, we show that a normal LA sensitive aPTT result may be safely used to refrain from LA testing in the absence of strong clinical suspicion

Whetting the SWORD: Detecting Workarounds by Using Active Learning and Logistic Regression

In many organizations, especially in healthcare, workers may work around prescribed procedures. Detecting these workarounds can give insights into difficulties concerning the procedures, which in turn can be used to improve them. Previous studies have shown that workarounds may be discovered from an event log using a set of predefined patterns such as the duration of a trace or the number of resources involved in one. However, domain experts may find it difficult to evaluate and monitor results if there are multiple patterns that indicate workarounds. Training a model that merges the features is often difficult because there are no available datasets covering workarounds. Labeling traces generally requires a lot of time from domain experts. In addition, this would have to be repeated for every new domain, company, or even department since the types of workarounds that occur may differ strongly between them. In this work, we propose to combine the features using a Logistic Regression model and train through Active Learning. In a case study at a hospital, we find that after training the model on only 10 to 15 traces, it stabilizes with an approximate F1 score of.75. This shows that we create and train a model that can detect workarounds well without requiring a large amount of labeled data or a lot of time from a domain expert

The incidence, mortality and renal outcomes of acute kidney injury in patients with suspected infection at the emergency department

Background Acute kidney injury (AKI) is a major health problem associated with considerable mortality and morbidity. Studies on clinical outcomes and mortality of AKI in the emergency department are scarce. The aim of this study is to assess incidence, mortality and renal outcomes after AKI in patients with suspected infection at the emergency department. Methods We used data from the SPACE-cohort (SePsis in the ACutely ill patients in the Emergency department), which included consecutive patients that presented to the emergency department of the internal medicine with suspected infection. Hazard ratios (HR) were assessed using Cox regression to investigate the association between AKI, 30-days mortality and renal function decline up to 1 year after AKI. Survival in patients with and without AKI was assessed using Kaplan-Meier analyses. Results Of the 3105 patients in the SPACE-cohort, we included 1716 patients who fulfilled the inclusion criteria. Of these patients, 10.8% had an AKI episode. Mortality was 12.4% for the AKI group and 4.2% for the non-AKI patients. The adjusted HR for all-cause mortality at 30-days in AKI patients was 2.8 (95% CI 1.7-4.8). Moreover, the cumulative incidence of renal function decline was 69.8% for AKI patients and 39.3% for non-AKI patients. Patients with an episode of AKI had higher risk of developing renal function decline (adjusted HR 3.3, 95% CI 2.4-4.5) at one year after initial AKI-episode at the emergency department. Conclusion Acute kidney injury is common in patients with suspected infection in the emergency department and is significantly associated with 30-days mortality and renal function decline one year after AKI