Transatlantic transferability of a new reinforcement learning model for optimizing haemodynamic treatment for critically ill patients with sepsis

© 2020Introduction: In recent years, reinforcement learning (RL) has gained traction in the healthcare domain. In particular, RL methods have been explored for haemodynamic optimization of septic patients in the Intensive Care Unit. Most hospitals however, lack the data and expertise for model development, necessitating transfer of models developed using external datasets. This approach assumes model generalizability across different patient populations, the validity of which has not previously been tested. In addition, there is limited knowledge on safety and reliability. These challenges need to be addressed to further facilitate implementation of RL models in clinical practice. Method: We developed and validated a new reinforcement learning model for hemodynamic optimization in sepsis on the MIMIC intensive care database from the USA using a dueling double deep Q network. We then transferred this model to the European AmsterdamUMCdb intensive care database. T-Distributed Stochastic Neighbor Embedding and Sequential Organ Failure Assessment scores were used to explore the differences between the patient populations. We apply off-policy policy evaluation methods to quantify model performance. In addition, we introduce and apply a novel deep policy inspection to analyse how the optimal policy relates to the different phases of sepsis and sepsis treatment to provide interpretable insight in order to assess model safety and reliability. Results: The off-policy evaluation revealed that the optimal policy outperformed the physician policy on both datasets despite marked differences between the two patient populations and physician's policies. Our novel deep policy inspection method showed insightful results and unveiled that the model could initiate therapy adequately and adjust therapy intensity to illness severity and disease progression which indicated safe and reliable model behaviour. Compared to current physician behavior, the developed policy prefers a more liberal use of vasopressors with a more restrained use of fluid therapy in line with previous work. Conclusion: We created a reinforcement learning model for optimal bedside hemodynamic management and demonstrated model transferability between populations from the USA and Europe for the first time. We proposed new methods for deep policy inspection integrating expert domain knowledge. This is expected to facilitate progression to bedside clinical decision support for the treatment of critically ill patients

Interactions macro- et microcirculatoires dans le choc

Microcirculatory alterations, frequently observed in shock states, are associated with the development of multiple organ failure and death. These microcirculatory alterations occur even when systemic hemodynamic variables are within resuscitation goals. In this review, we discuss the link between the microcirculation and systemic hemodynamics. Several studies suggested that microvascular perfusion may be dependent from systemic hemodynamics in severely hypotensive patients. However, the threshold of blood pressure at which this occurs is not well defined and it is difficult to define which mean arterial pressure target should be reached. The microcirculation is usually independent of systemic hemodynamics in the usual range of systemic hemodynamic values encountered when resuscitation targets are met. Fluids and vasoactive agents affect the microcirculation independently of their systemic effects. © 2013 Société de réanimation de langue française (SRLF) and Springer-Verlag France.SCOPUS: ar.jinfo:eu-repo/semantics/publishe