Fellinger, T

Metnitz, P

Moreno, R

Posch, M

Zajic, P

English

Repositório do Centro Hospitalar de Lisboa Central, EPE

Intensive Care Med (2021) 47:910–912https://doi.org/10.1007/s00134-021-06436-9LETTEREvaluation and calibration of SAPS 3 in patients with COVID-19 admitted to intensive care unitsPhilipp G. H. Metnitz1, Rui P. Moreno2, Tobias Fellinger3, Martin Posch3 and Paul Zajic1* © 2021 Springer-Verlag GmbH Germany, part of Springer NatureDear Editor,Scoring systems are invaluable tools for research, quality assurance, and performance comparison in intensive care medicine. They allow for adjustment for underlying risk of unwanted outcomes, primarily mortality, and make comparisons between interventions, units, and systems possible. The Simplified Acute Physiology Score 3 (SAPS 3) is a well-validated scoring system used worldwide for the prediction of hospital mortality based on variables of acute physiologic derangements, current conditions and interventions, and previous health status [1, 2].Coronavirus disease 2019 (COVID-19) put pressure on intensive care units (ICU), health care systems, nations, and societies worldwide. Measurement of health care performance is therefore more important than ever. Pre-vious studies put the validity of other well-established intensive care scoring systems in COVID-19 cases in question [3], while others claimed good predictive capa-bilities [4]. Several outcome prognostication models for patient groups affected by COVID-19 have also been proposed. For these, a living systematic review and criti-cal appraisal of the available literature finds C-index esti-mates for the prognostication of mortality to range from 0.68 to 0.98 [5]. In this study, we seek to evaluate the per-formance of SAPS 3 in the prediction of hospital mortal-ity in COVID-19 patients admitted to ICUs.Anonymous data from the Austrian Centre for Statis-tics and Documentation in Intensive Care (ASDI) data-base on patients with documented SARS-CoV-2 infection admitted to participating ICUs from January 1st, 2020, to January 31st, 2021, were retrieved and used for retro-spective analyses. The anonymous fashion of the dataset precluded the need for ethical approval.Discriminative performance of SAPS 3 for hospital mortality was evaluated by calculation of the area under the receiver operating curve (AUC) and 95% confidence intervals (95% CI) based on the DeLong approach [6]. Hosmer–Lemeshow test in deciles of observed-to-expected ratios (O/E ratios) and the calibration belt method [7] were used to assess goodness of fit of the for-mula to calculate predicted hospital mortality. Recalibra-tion was conducted using the structure of the formula for regional customisations in ref. [2]; parameters were esti-mated by maximum likelihood method. Weights of indi-vidual SAPS 3 items remained unchanged. Analyses were conducted using R version 4.0.0 with packages pROC and givitiR.1464 patients with COVID-19 admitted to 90 par-ticipating ICUs were identified (electronic supplement Table  1). Of these, 501 (34%) died during their hospital stay. AUC for discrimination of hospital mortality was 0.745 (95% CI 0.719–0.770). Standard calibration [2] led to under-estimation of hospital mortality [O/E ratio (95% CI) 1.20 (1.12–1.27), Ĥ = 41.10 (p < 0.001), Ĉ = 40.92 (p < 0.001)], especially in lower risk groups (Fig.  1, elec-tronic supplement Fig.  2). Dedicated calibration for COVID-19 using the formula Probability of death = elogit/(1 + elogit), where logit = − 14.451 + 3.666 * ln(SAPS3 + − 12.092) led to improved goodness of fit [Ĥ = 7.15 (p = 0.71), Ĉ = 5.01 (p = 0.89)]] (Fig. 1, electronic supple-ment Fig. 3).We find the SAPS 3 to be of satisfactory performance in the prognostication of hospital mortality in patients *Correspondence:  paul.zajic@medunigraz.at 1 Division of General Anaesthesiology, Emergency- and Intensive Care Medicine, Medical University of Graz, Graz, AustriaFull author information is available at the end of the article911with COVID-19 admitted to intensive care units. Use of a general prediction model such as the SAPS 3 allows for the evaluation of outcomes in patient cohort, units, and systems irrespective of the underlying disease and is, therefore, preferable. Recalibration of the SAPS 3 can be used to allow for more precise performance evaluation in COVID-19 cohorts.Supplementary InformationThe online version contains supplementary material available at https:// doi. org/ 10. 1007/ s00134- 021- 06436-9.Author details1 Division of General Anaesthesiology, Emergency- and Intensive Care Medicine, Medical University of Graz, Graz, Austria. 2 Unidade de Cuidados Intensivos Neurocríticos e Trauma, Centro Hospitalar Universitário de Lisboa Central, Lisbon, Portugal. 3 Center for Medical Statistics, Informatics, and Intel-ligent Systems, Medical University of Vienna, Vienna, Austria. DeclarationsConflicts of interestMP reports institutional grants from the Austrian Center for Documentation and Quality Assurance in Intensive Care Medicine. TF reports his employment at the Center for Medical Statistics, Informatics, and Intelligent Systems to be funded by the Austrian Center for Documentation and Quality Assurance in Intensive Care Medicine.Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims in pub-lished maps and institutional affiliations.Accepted: 12 May 2021Published online: 19 May 2021References 1. Metnitz PG, Moreno RP, Almeida E, Jordan B, Bauer P, Campos RA, Iapi-chino G, Edbrooke D, Capuzzo M, Le Gall JR, SAPS 3 Investigators (2005) SAPS 3–from evaluation of the patient to evaluation of the intensive care unit. Part 1: objectives, methods and cohort description. Intensive Care Med 31(10):1336–1344. https:// doi. org/ 10. 1007/ s00134- 005- 2762-6COVID−19 Calibration Standard Calibration[0,10] (10,20] (20,30] (30,40] (40,50] (50,60] (60,70] (70,80] (80,90](90,100] [0,10] (10,20] (20,30] (30,40] (40,50] (50,60] (60,70] (70,80] (80,90](90,100]0%25%50%75%100%0100200300Predicted MortalityMortality, %Number of PatientsObserved Mortality Predicted MortalityFig. 1 Binned plots of hospital mortality predicted by SAPS 3 in deciles (x-axis), observed mortality (primary y-axis) for COVID-19 calibration (left) and standard calibration (right) [dot-and-line graphs, unadjusted confidence intervals], number of patients (secondary y-axis) [bar graphs]912 2. Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, Iapi-chino G, Edbrooke D, Capuzzo M, Le Gall JR, SAPS 3 Investigators (2005) SAPS 3–from evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 31(10):1345–1355. https:// doi. org/ 10. 1007/ s00134- 005- 2763-5 3. Stephens JR, Stümpfle R, Patel P, Brett S, Broomhead R, Baharlo B, Soni S (2021) Analysis of Critical Care Severity of Illness Scoring Systems in Patients With Coronavirus Disease 2019: A Retrospective Analysis of Three U.K. ICUs. Crit Care Med 49(1): e105–e107. https:// doi. org/ 10. 1097/ CCM. 00000 00000 004674 4. Zou X, Li S, Fang M, Hu M, Bian Y, Ling J, Yu S, Jing L, Li D, Huang J (2020) Acute physiology and chronic health evaluation II score as a predictor of hospital mortality in patients of coronavirus disease 2019. Crit Care Med 48(8):e657–e665. https:// doi. org/ 10. 1097/ CCM. 00000 00000 004411 5. Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G et al (2020) Predic-tion models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ 369:m1328 6. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845 7. Finazzi S, Poole D, Luciani D, Cogo PE, Bertolini G (2011) Calibration belt for quality-of-care assessment based on dichotomous outcomes. PLoS ONE 6(2):e16110

Evaluation and Calibration of SAPS 3 in Patients with COVID-19 Admitted to Intensive Care Units

http://repositorio.chlc.min-saude.pt/bitstream/10400.17/4424/1/ICM%202021%20910.pdf

Evaluation and Calibration of SAPS 3 in Patients with COVID-19 Admitted to Intensive Care Units

Abstract

Similar works

Full text

Available Versions

Repositório do Centro Hospitalar de Lisboa Central, EPE