Claasen, M.

Cowling, T.

Erdman, L.

Gravely, A.

Ivanics, Tommy

Patel, M.

Sapisochin, G.

So, D.

Walker, K.

Wallace, D.

English

Henry Ford Health System Scholarly Commons

Henry Ford Health Henry Ford Health Scholarly Commons Surgery Meeting Abstracts Surgery 8-1-2022 Evaluating the performance and external validity of machine learning-based prediction models in liver transplantation: an international study Tommy Ivanics D. So M. Claasen D. Wallace M. Patel See next page for additional authors Follow this and additional works at: https://scholarlycommons.henryford.com/surgery_mtgabstracts Authors Tommy Ivanics, D. So, M. Claasen, D. Wallace, M. Patel, A. Gravely, K. Walker, T. Cowling, L. Erdman, and G. Sapisochin Plenary Abstract Session I62022 Joint International Congress of ILTS, ELITA & LICAGE  May 4—7, 2022 | Istanbul, Turkey & OnlinePlenary Abstract Session IO-005  Evaluating the performance and external validity of machine learning-based prediction models in liver transplantation: an international study  T. Ivanics1,2,3, D. So4, M. Claasen1,5, D. Wallace6, M. Patel7, A. Gravely1, K. Walker6, T. Cowling6, L. Erdman4, G. Sapisochin11University Health Network, Multi-Organ Transplant Program, Toronto, Canada, 2Henry Ford Hospital, Department of Surgery, Detroit, United States, 3Uppsala University, Department of Surgical Sciences, Uppsala, Sweden, 4The Hospital for Sick Children, The Centre of Computational Medicine, Toronto, Canada, 5Erasmus MC, Department of Surgery, Rotterdam, Netherlands, 6London School of Hygiene and Tropical Medicine, Department of Health Services Research and Policy, London, United Kingdom, 7University of Texas Southwestern Medical Center, Division of Surgical Transplantation, Department of Surgery, Dallas, United StatesBackground: National liver transplant (LT) registry data are curated in many countries. We compared data from three national registries and developed machine learning algorithm (MLA)-based models to predict post-LT 90-day mortality within and across different countries. Predictive performance and external validity of each model was assessed to contextualize the applicability of MLA in LT.Methods: We studied adult (≥18-years) primary LTs between Jan-2008 and Dec-2018 from United Network for Organ Sharing (UNOS-US), National Health Service Blood and Transplantation (NHSBT-UK), and the Canadian Organ Replacement Registry (CORR-Canada). MLA models for 90-day post-LT mortality were built firstly on each individual registry (based on variables inherent to the individual database) and then using all 3 registries (based on harmonized variables). The predictive abilities of the models were evaluated across countries using area-under-the-receiver-operator-curve (AUROC) and area-under-the-precision-recall-curve (AUPRC).Results: Patients included were as follows: Canada n=1,214, UK n=5,287, and US n=59,558. ElasticNet had the best performance across both individual registries and harmonized datasets. Model performance diminished from the individualized registries to the harmonized registry (only using variables in common between the three registries), especially in the UK (individualized ElasticNet:AUROC:0.54;Range 0.52-0.56 to harmonized AUROC:0.48; Range:0.48-0.50) and the US (individualized ElasticNet:AUROC:0.70;Range:0.70-0.71 to harmonized AUROC:0.65;Range:0.64-0.65). Model performance after external validation across countries was overall poor.Conclusions: MLA-based models can be constructed using international LT registries, with independent ElasticNet models demonstrating optimal predictive performance. While MLA-based models yield fair discriminatory potential when used within individual databases, the external validity is poor when applied to different registries across countries. This is likely due to inherent limitations and variability within each dataset. It is conceivable that these limitations may be overcome by increasing the granularity of datasets (e.g., with linkages to other administrative datasets) and placing an increased emphasis on consensus for variable standardization.O-006  A population-based analysis of long-term outcomes following pediatric acute liver failure highlights high risk populations in access to transplant  J. Ascher Bartlett1, M. Lo2, K. Etesami3, L. Sher3, R. Kohli1, J. Emamaullee31Children‘s Hospital Los Angeles, Gastroenterology, Hepatology and Nutrition, Los Angeles, United States, 2University of Southern California, Los Angeles, United States, 3Keck Medicine of University of Southern California, Surgery, Los Angeles, United StatesBackground: Pediatric acute liver failure (PALF) is a devastating illness that affects otherwise healthy children. Nearly 70% of children with PALF recover, while ≥25% require emergent liver transplant (LT). PALF has an estimated overall mortality of 15%. Beyond short-term outcomes, the long-term clinical course of PALF has not been established. This study aims to characterize long-term outcomes and identify prognostic factors of PALF using registry data.Methods: Children (<18 years) admitted with PALF were identified by ICD codes in the California Office of Statewide Health Planning and Development Patient Discharge Dataset (1/2005-12/2018). Multivariable Cox proportional hazards models were used to identify risk factors for liver transplant and death.Results: Among 2162 inpatients with PALF, the mean age at presentation was 9.7±6.2 years, with a median follow-up of 4.6 (IQR 0.06 -10.4) years. 50.1% were female and 44.2% Hispanic. Most deaths occurred within 4 months of presentation. Children <2 years (18.2% of patients) were more likely to undergo LT (HR 2.74 [95% CI 1.96-3.83], p<0.001) and more likely to die (HR 1.66 [95% CI 1.33-2.07], p<0.001) when compared to children >10 years old (52.3% of patients) (Table 1). Mortality was higher among females (HR 1.47 [95% CI 1.17-1.85], p<0.001). No statistically significant difference was observed based on race/ethnicity. 

Evaluating the performance and external validity of machine learning-based prediction models in liver transplantation: an international study

https://scholarlycommons.henryford.com/cgi/viewcontent.cgi?article=1246&context=surgery_mtgabstracts

Evaluating the performance and external validity of machine learning-based prediction models in liver transplantation: an international study

Abstract

Similar works

Full text

Available Versions

Henry Ford Health System Scholarly Commons