Generalisability of deep learning-based early warning in the intensive care unit: a retrospective empirical evaluation

Deep learning (DL) can aid doctors in detecting worsening patient states early, affording them time to react and prevent bad outcomes. While DL-based early warning models usually work well in the hospitals they were trained for, they tend to be less reliable when applied at new hospitals. This makes it difficult to deploy them at scale. Using carefully harmonised intensive care data from four data sources across Europe and the US (totalling 334,812 stays), we systematically assessed the reliability of DL models for three common adverse events: death, acute kidney injury (AKI), and sepsis. We tested whether using more than one data source and/or explicitly optimising for generalisability during training improves model performance at new hospitals. We found that models achieved high AUROC for mortality (0.838-0.869), AKI (0.823-0.866), and sepsis (0.749-0.824) at the training hospital. As expected, performance dropped at new hospitals, sometimes by as much as -0.200. Using more than one data source for training mitigated the performance drop, with multi-source models performing roughly on par with the best single-source model. This suggests that as data from more hospitals become available for training, model robustness is likely to increase, lower-bounding robustness with the performance of the most applicable data source in the training data. Dedicated methods promoting generalisability did not noticeably improve performance in our experiments

Predicting lethal courses in critically ill COVID-19 patients using a machine learning model trained on patients with non-COVID-19 viral pneumonia

In a pandemic with a novel disease, disease-specific prognosis models are available only with a delay. To bridge the critical early phase, models built for similar diseases might be applied. To test the accuracy of such a knowledge transfer, we investigated how precise lethal courses in critically ill COVID-19 patients can be predicted by a model trained on critically ill non-COVID-19 viral pneumonia patients. We trained gradient boosted decision tree models on 718 (245 deceased) non-COVID-19 viral pneumonia patients to predict individual ICU mortality and applied it to 1054 (369 deceased) COVID-19 patients. Our model showed a significantly better predictive performance (AUROC 0.86 [95% CI 0.86-0.87]) than the clinical scores APACHE2 (0.63 [95% CI 0.61-0.65]), SAPS2 (0.72 [95% CI 0.71-0.74]) and SOFA (0.76 [95% CI 0.75-0.77]), the COVID-19-specific mortality prediction models of Zhou (0.76 [95% CI 0.73-0.78]) and Wang (laboratory: 0.62 [95% CI 0.59-0.65]; clinical: 0.56 [95% CI 0.55-0.58]) and the 4C COVID-19 Mortality score (0.71 [95% CI 0.70-0.72]). We conclude that lethal courses in critically ill COVID-19 patients can be predicted by a machine learning model trained on non-COVID-19 patients. Our results suggest that in a pandemic with a novel disease, prognosis models built for similar diseases can be applied, even when the diseases differ in time courses and in rates of critical and lethal courses

The nociceptive flexion reflex as a measure of antinociception under general anesthesia

Neben der Bewusstlosigkeit ist die Unterdrückung von Nozizeption, also die Unterdrückung der Sinneswahrnehmung von Schmerz, einer der essentiellen Bestandteile der Allgemeinanästhesie. Da sowohl unzureichende, wie auch überschießende Gaben von Analgetika im Rahmen der Allgemeinanästhesie das perioperative Outcome negativ beeinflussen könnten, ist deren optimale Dosierung ein zentrales klinisches Ziel. Im Rahmen der hier zusammengefassten Studien stellen wir ein Verfahren vor, welches im Sinne eines „Analgesie- Index“ eine optimale Dosierung von antinozizeptiven Medikamenten im Rahmen der Allgemeinanästhesie unterstützen könnte. Das Verfahren basiert auf einer kontinuierlichen Bestimmung der Schwelle des nozizeptiven Flexorenreflexes, welcher mit Hilfe elektrischer Stimulation am Außenknöchel ausgelöst und mittels Elektromyografie über dem Oberschenkel abgeleitet werden kann. Wir konnten anhand von Probandenstudien zeigen, dass diese Schwelle des nozizeptiven Flexorenreflexes sowohl unter Mononarkosen mit Propofol und Sevofluran, sowie auch unter Narkose mit Propofol und Remifentanil in wechselnden Dosierungen, zuverlässig bestimmt werden kann und jeweils konzentrationsabhängig durch die einzelnen Medikamente beeinflusst wird. Des Weiteren konnten wir zeigen, dass anhand des Reflexes Vorhersagen von Bewegungen auf einen elektrischen Schmerzreiz getroffen werden können: dabei ergab sich unter Propofolmononarkose eine ähnliche Präzision wie mittels des Bispektralindex, und unter Sevofluran und Propofol-Remifentanil jeweils eine höhere Präzision. Im Rahmen einer Patientenstudie konnten wir darüber hinaus zeigen, dass der nozizeptive Flexorenreflex auch im klinischen Umfeld zuverlässig eingesetzt werden kann. Unter Kombinationsnarkose mit Propofol und Remifentanil erlaubte der nozizeptive Flexorenreflex hierbei eine Vorhersage sowohl von Bewegungsreaktionen, als auch von Herzfrequenzanstiegen, welche jeweils nach Einlage der Larynxmaske und nach Hautschnitt auftraten. Die Präzision der Vorhersagen übertraf dabei alle anderen untersuchten Monitoringparameter wie Bispektralindex, Composite Variability Index, Noxious Stimulation Response Index und die Medikamentenkonzentrationen. Der nozizeptive Flexorenreflex stellt somit einen zuverlässigen funktionellen Surrogatparameter für klinische Ausdrücke der Nozizeption dar. Die Korrelation des Reflexes mit den klinischen Schmerzreaktionen bleibt auch unter Anästhetika mit unterschiedlichen molekularen Wirkmechanismen und unter Medikamentenkombinationen erhalten.Next to unconsciousness, the suppression of nociception is one of the essential components of general anesthesia. Since both, too low dosing of anesthetics as well as too high dosing of anesthetics, can lead to negative perioperative outcome, optimal dosing of anesthetics is of vital importance. In the here presented studies we introduced a method which can be used as an “analgesia-index” during general anesthesia to assist optimal dosing of anesthetics. The method is based on a continual estimation of the threshold of the nociceptive flexion reflex, which can be recorded using electromyography of the biceps femoris muscle while stimulating the sural nerve. We were able to show that the here presented method allows a reliable estimation of the threshold of the nociceptive flexion reflex unter mono-anesthesia using propofol or sevofluran, as well as under anesthesia using propofol and remifentanil in varying concentrations. The reflex threshold was affected in a dose-dependant manner by all investigated anesthetics. We also able to demonstrate, that the reflex threshold allows a prediction of movement responses following a painful electrical stimulus. The accuracy of the predictions using the reflex threshold under propofol was similar as the accuracy of predictions using the eeg-parameter bispectral index. The accuracy of predictions using the reflex threshold under sevofluran and propofol- remifentanil combination were higher compared to the bispectral index. Furthermore, we were able to show in a clinical study on patients, that the nociceptive flexion reflex can be recorded reliably in the clinical setting. Under combined anesthesia using propofol and remifentanil, the nociceptive flexion reflex allowed a prediction of movement responses as well as heart rate responses after insertion of the laryngeal mask and after skin incision. The accuracy of the predictions based on the reflex threshold was higher than all other investigated monitoring parameters, namely the bispectral index, the composite variability index, the noxious stimulation response index and the drug concentrations. We conclude that the nociceptive flexion reflex is a reliable functional surrogate parameter for clinical expressions of nociception under general anesthesia. The correlation of the reflex threshold with clinical pain responses remains unaffected even when anesthetics with different molecular targets are used

Inhibitory mechanisms of the alpha-motoneuronal excitability of propofol, sevoflurane and nitrous oxide in the human spinal cord

Es wird angenommen, dass die Unterdrückung motorischer Reaktionen auf Schmerzreize durch Anästhetika zumindest teilweise über eine Abnahme der Erregbarkeit von alpha-Motoneuronen vermittelt wird. Diese Abnahme der Erregbarkeit kann eine Folge von prä- und/oder postsynaptischen Hemmungsmechanismen sein. Zur Bestimmung der relativen Anteile von prä- und postsynaptischen Mechanismen an der jeweiligen Gesamthemmung untersuchten wir am Menschen unter Einfluss von Propofol, Sevofluran und Lachgas im Vergleich zu Kontrollbedingungen: i) die Amplitude des maximalen H-Reflexes als einen direkter Parameter der alpha-motoneuronalen Erregbarkeit, welcher sowohl durch präsynaptische als auch durch postsynaptische Hemmungsmechanismen beeinflusst wird, und ii) die relative heteronyme Fazilitation des H-Reflexes als einen anti-proportionalen Parameter der alleinigen präsynaptischen Hemmung. Während Lachgas kaum präsynaptische Hemmung bei deutlicher Abnahme der alpha- motoneuronalen Erregbarkeit zeigt, ergeben sich für Sevofluran und Propofol starke prä- sowie postsynaptische Hemmungsanteile. Die unterschiedlichen Ausmaße der präsynaptischen Hemmung lassen sich durch die jeweiligen GABAergen Wirkungen der einzelnen Medikamente erklären: während für Lachgas kaum GABAerge Effekte bekannt sind, stellen diese ein Hauptziel der spinalen Wirkung von Sevofluran und Propofol dar. Die GABAergen Wirkungen von Sevofluran und Propofol scheinen daher für ihre Hemmung der alpha- motoneuronalen Erregbarkeit und somit für die Unterdrückung motorischer Reaktionen auf Schmerzreize durch diese Stoffe von Bedeutung zu sein.The suppression of movement reactions to noxious stimuli under influence of anesthetics is mediated at least in part by a reduction of the excitability of alpha-motoneurons. This reduction of excitability can be a result of pre- and/or postsynaptic inhibitory mechanisms. To investigate the relative amounts of contribution of pre- and postsynaptic inhibitory mechanisms to the overall reduction of motoneuronal excitability, we determined in humans under influence of propofol, sevoflurane and nitrous oxide in comparison to control states the following parameters: i) the amplitude of the maximum h-reflex as a direct parameter of the alpha-motoneuronal excitability, which is susceptible to both presynaptic and postsynaptic inhibitory mechanisms, and ii) the relative amount of heteronymous facilitation of the h-reflex as an anti- proportional parameter of the presynaptic inhibition only. While nitrous oxide shows only a marginal amount of presynaptic inhibition along with a high amount of reduction of motoneuronal excitability, sevoflurane and propofol show both strong pre- and postsynaptic inhibitory fractions. The different amounts of presynaptic inhibition could be explained by the GABAergic actions of the different substances: while for nitrous oxide only very marginal GABAergic effects are known, the GABAergic structures represent a major site of action for the spinal mechanisms of propofol and sevoflurane. So the GABAergic actions of sevoflurane and propofol seem to be of importance for their inhibitory mechanisms of motoneuronal excitability and therefore for their mechanisms of suppression of movement

Reliability of Subjective Pain Ratings and Nociceptive Flexion Reflex Responses as Measures of Conditioned Pain Modulation

BACKGROUND: The endogenous modulation of pain can be assessed through conditioned pain modulation (CPM), which can be quantified using subjective pain ratings or nociceptive flexion reflexes. However, to date, the test-retest reliability has only been investigated for subjective pain ratings

Technical considerations when using the EEG export of the SEDLine Root device

Electroencephalographic (EEG) patient monitoring during general anesthesia can help to assess the real-time neurophysiology of unconscious states. Some monitoring systems like the SEDLine Root allow export of the EEG to be used for retrospective analysis. We show that changes made to the SEDLine display during recording affected the recorded EEG. These changes can strongly impact retrospective analysis of EEG signals. Real-time changes of the feed speed in the SEDLine Root device display modifies the sampling rate of the exported EEG. We used a patient as well as a simulated EEG recording to highlight the effects of the display settings on the extracted EEG. Therefore, we changed EEG feed and amplitude resolution on the display in a systematic manner. To visualize the effects of these changes, we present raw EEG segments and the density spectral array of the recording. Changing the display's amplitude resolution affects the amplitudes. If the amplitude resolution is too fine, the exported EEG contains clipped amplitudes. If the resolution is too coarse, the EEG resolution becomes too low leading to a low-quality signal making frequency analysis impossible. The proportion of clipped or zero-line data caused by the amplitude setting was > 60% in our sedated patient. Changing the display settings results in undocumented changes in EEG amplitude, sampling rate, and signal quality. The occult nature of these changes could make the analysis of data sets difficult if not invalid. We strongly suggest researchers adequately define and keep the EEG display settings to export good quality EEG and to ensure comparability among patients

Presynaptic and Postsynaptic Effects of the Anesthetics Sevoflurane and Nitrous Oxide in the Human Spinal Cord

Background: Reduced spinal excitability contributes to the suppression of movement responses to noxious stimuli during the anesthetic state. This study examines and compares presynaptic and postsynaptic effects of two anesthetics in the human spinal cord. Methods: The authors tested two parameters during the administration of 0.8 vol% sevoflurane or 40 vol% nitrous oxide compared with control states before and after drug administration: (1) the size of the soleus H reflex (integrating presynaptic and postsynaptic effects) at increasing stimulus intensities (recruitment curve) and (2) the amount of presynaptic inhibition on Ia afferents of the quadriceps femoris, evaluated by the heteronymous facilitation of the soleus H reflex caused by a conditioning stimulation of the femoral nerve. The study was performed in 10 subjects for each drug. Results: At the chosen concentrations, the maximum H reflex was reduced by 26.3 ؎ 8.4% (mean ؎ SD) during sevoflurane and by 33.5 ؎ 15.6% during nitrous oxide administration. The averaged recruitment curves were similarly depressed under the influence of the two drugs. The reduction of H-reflex facilitation was significantly stronger for sevoflurane (28.8 ؎ 20.0%) than for nitrous oxide administration (6.2 ؎ 26.4%). Conclusions: These results demonstrate in humans presynaptic effects of the volatile anesthetic sevoflurane but not of nitrous oxide. A possible explanation for this difference may be the different potency of the respective drugs in enhancing ␥-aminobutyric acid type A receptor-mediated inhibition, because presynaptic inhibition in the spinal cord involves this receptor subtype

Das Gesundheitsdatennutzungsgesetz und was dies für die Forschung in der Intensiv- und Notfallmedizin bedeutet : Erläuterungen und erste Handlungsempfehlungen

The Health Data Utilization Act (GDNG) is the first nationwide legal basis for the use of healthcare data for research. Particularly relevant for intensive care and emergency medicine is the authorization of non-consensual data use (in-house research), including collaborative research by healthcare facilities. Another change is that there is a cultural shift from preventing unauthorised data use to transparency and control. These changes open up new opportunities for research, but are also associated with obligations. In this article, an interdisciplinary and interprofessional working group from three scientific societies explains the GDNG and formulates initial recommendations for action.Das Gesundheitsdatennutzungsgesetz (GDNG) schafft zum ersten Mal eine bundeseinheitliche Rechtsgrundlage für die Nutzung von Versorgungsdaten für die Forschung. Für die Intensiv- und Notfallmedizin ist besonders der Erlaubnistatbestand einer einwilligungsfreien Datennutzung (Eigenforschung) relevant, einschließlich der Verbundforschung von Gesundheitseinrichtungen. Darüber hinaus gibt es einen Kulturwechsel von der Verhinderung unerlaubter Datennutzung hin zu Transparenz und Kontrolle. Diese Veränderungen eröffnen neue Möglichkeiten für die Forschung, sind aber auch mit Pflichten verbunden. In einer interdisziplinären und interprofessionellen Arbeitsgruppe mit Vertretern von drei Fachgesellschaften wird in diesem Artikel das GDNG erläutert und erste Handlungsempfehlungen formuliert