RiskFix: Supporting Expert Validation of Predictive Timeseries Models in High-Intensity Settings

Many real-world machine learning workflows exist in longitudinal, interactive machine learning (ML) settings. This longitudinal nature is often due to incremental increasing of data, e.g., in clinical settings, where observations about patients evolve over their care period. Additionally, experts may become a bottleneck in the workflow, as their limited availability, combined with their role as human oracles, often leads to a lack of ground truth data. In such cases where ground truth data is small, the validation of interactive machine learning workflows relies on domain experts. Only those humans can assess the validity of a model prediction, especially in new situations that have been covered only weakly by available training data. Based on our experiences working with domain experts of a pediatric hospital's intensive care unit, we derive requirements for the design of support interfaces for the validation of interactive ML workflows in fast-paced, high-intensity environments. We present RiskFix, a software package optimized for the validation workflow of domain experts of such contexts. RiskFix is adapted to the cognitive resources and needs of domain experts in validating and giving feedback to the model. Also, RiskFix supports data scientists in their model-building work, with appropriate data structuring for the re-calibration (and possible retraining) of ML models

Diagnostic errors in paediatric cardiac intensive care

AbstractIntroductionDiagnostic errors cause significant patient harm and increase costs. Data characterising such errors in the paediatric cardiac intensive care population are limited. We sought to understand the perceived frequency and types of diagnostic errors in the paediatric cardiac ICU.MethodsPaediatric cardiac ICU practitioners including attending and trainee physicians, nurse practitioners, physician assistants, and registered nurses at three North American tertiary cardiac centres were surveyed between October 2014 and January 2015.ResultsThe response rate was 46% (N=200). Most respondents (81%) perceived that diagnostic errors harm patients more than five times per year. More than half (65%) reported that errors permanently harm patients, and up to 18% perceived that diagnostic errors contributed to death or severe permanent harm more than five times per year. Medication side effects and psychiatric conditions were thought to be most commonly misdiagnosed. Physician groups also ranked pulmonary overcirculation and viral illness to be commonly misdiagnosed as bacterial illness. Inadequate care coordination, data assessment, and high clinician workload were cited as contributory factors. Delayed diagnostic studies and interventions related to the severity of the patient’s condition were thought to be the most commonly reported process breakdowns. All surveyed groups ranked improving teamwork and feedback pathways as strategies to explore for preventing future diagnostic errors.ConclusionsPaediatric cardiac intensive care practitioners perceive that diagnostic errors causing permanent harm are common and associated more with systematic and process breakdowns than with cognitive limitations.</jats:sec

iCVS-Inferring Cardio-Vascular hidden States from physiological signals available at the bedside.

Intensive care medicine is complex and resource-demanding. A critical and common challenge lies in inferring the underlying physiological state of a patient from partially observed data. Specifically for the cardiovascular system, clinicians use observables such as heart rate, arterial and venous blood pressures, as well as findings from the physical examination and ancillary tests to formulate a mental model and estimate hidden variables such as cardiac output, vascular resistance, filling pressures and volumes, and autonomic tone. Then, they use this mental model to derive the causes for instability and choose appropriate interventions. Not only this is a very hard problem due to the nature of the signals, but it also requires expertise and a clinician's ongoing presence at the bedside. Clinical decision support tools based on mechanistic dynamical models offer an appealing solution due to their inherent explainability, corollaries to the clinical mental process, and predictive power. With a translational motivation in mind, we developed iCVS: a simple, with high explanatory power, dynamical mechanistic model to infer hidden cardiovascular states. Full model estimation requires no prior assumptions on physiological parameters except age and weight, and the only inputs are arterial and venous pressure waveforms. iCVS also considers autonomic and non-autonomic modulations. To gain more information without increasing model complexity, both slow and fast timescales of the blood pressure traces are exploited, while the main inference and dynamic evolution are at the longer, clinically relevant, timescale of minutes. iCVS is designed to allow bedside deployment at pediatric and adult intensive care units and for retrospective investigation of cardiovascular mechanisms underlying instability. In this paper, we describe iCVS and inference system in detail, and using a dataset of critically-ill children, we provide initial indications to its ability to identify bleeding, distributive states, and cardiac dysfunction, in isolation and in combination