Multiple Score Comparison: a network meta-analysis approach to comparison and external validation of prognostic scores

BACKGROUND Prediction models and prognostic scores have been increasingly popular in both clinical practice and clinical research settings, for example to aid in risk-based decision making or control for confounding. In many medical fields, a large number of prognostic scores are available, but practitioners may find it difficult to choose between them due to lack of external validation as well as lack of comparisons between them. METHODS Borrowing methodology from network meta-analysis, we describe an approach to Multiple Score Comparison meta-analysis (MSC) which permits concurrent external validation and comparisons of prognostic scores using individual patient data (IPD) arising from a large-scale international collaboration. We describe the challenges in adapting network meta-analysis to the MSC setting, for instance the need to explicitly include correlations between the scores on a cohort level, and how to deal with many multi-score studies. We propose first using IPD to make cohort-level aggregate discrimination or calibration scores, comparing all to a common comparator. Then, standard network meta-analysis techniques can be applied, taking care to consider correlation structures in cohorts with multiple scores. Transitivity, consistency and heterogeneity are also examined. RESULTS We provide a clinical application, comparing prognostic scores for 3-year mortality in patients with chronic obstructive pulmonary disease using data from a large-scale collaborative initiative. We focus on the discriminative properties of the prognostic scores. Our results show clear differences in performance, with ADO and eBODE showing higher discrimination with respect to mortality than other considered scores. The assumptions of transitivity and local and global consistency were not violated. Heterogeneity was small. CONCLUSIONS We applied a network meta-analytic methodology to externally validate and concurrently compare the prognostic properties of clinical scores. Our large-scale external validation indicates that the scores with the best discriminative properties to predict 3 year mortality in patients with COPD are ADO and eBODE

Global benchmarks in primary robotic bariatric surgery redefine quality standards for Roux-en-Y gastric bypass and sleeve gastrectomy

BACKGROUND Whether the benefits of the robotic platform in bariatric surgery translate into superior surgical outcomes remains unclear. The aim of this retrospective study was to establish the 'best possible' outcomes for robotic bariatric surgery and compare them with the established laparoscopic benchmarks. METHODS Benchmark cut-offs were established for consecutive primary robotic bariatric surgery patients of 17 centres across four continents (13 expert centres and 4 learning phase centres) using the 75th percentile of the median outcome values until 90 days after surgery. The benchmark patients had no previous laparotomy, diabetes, sleep apnoea, cardiopathy, renal insufficiency, inflammatory bowel disease, immunosuppression, history of thromboembolic events, BMI greater than 50 kg/m2, or age greater than 65 years. RESULTS A total of 9097 patients were included, who were mainly female (75.5%) and who had a mean(s.d.) age of 44.7(11.5) years and a mean(s.d.) baseline BMI of 44.6(7.7) kg/m2. In expert centres, 13.74% of the 3020 patients who underwent primary robotic Roux-en-Y gastric bypass and 5.9% of the 4078 patients who underwent primary robotic sleeve gastrectomy presented with greater than or equal to one complication within 90 postoperative days. No patient died and 1.1% of patients had adverse events related to the robotic platform. When compared with laparoscopic benchmarks, robotic Roux-en-Y gastric bypass had lower benchmark cut-offs for hospital stay, postoperative bleeding, and marginal ulceration, but the duration of the operation was 42 min longer. For most surgical outcomes, robotic sleeve gastrectomy outperformed laparoscopic sleeve gastrectomy with a comparable duration of the operation. In robotic learning phase centres, outcomes were within the established benchmarks only for low-risk robotic Roux-en-Y gastric bypass. CONCLUSION The newly established benchmarks suggest that robotic bariatric surgery may enhance surgical safety compared with laparoscopic bariatric surgery; however, the duration of the operation for robotic Roux-en-Y gastric bypass is longer

Global Benchmarks in Primary Robotic Bariatric Surgery Redefine Quality Standards for Roux-En-Y Gastric Bypass and Sleeve Gastrectomy

BACKGROUND: Whether the benefits of the robotic platform in bariatric surgery translate into superior surgical outcomes remains unclear. The aim of this retrospective study was to establish the \u27best possible\u27 outcomes for robotic bariatric surgery and compare them with the established laparoscopic benchmarks. METHODS: Benchmark cut-offs were established for consecutive primary robotic bariatric surgery patients of 17 centres across four continents (13 expert centres and 4 learning phase centres) using the 75th percentile of the median outcome values until 90 days after surgery. The benchmark patients had no previous laparotomy, diabetes, sleep apnoea, cardiopathy, renal insufficiency, inflammatory bowel disease, immunosuppression, history of thromboembolic events, BMI greater than 50 kg/m2, or age greater than 65 years. RESULTS: A total of 9097 patients were included, who were mainly female (75.5%) and who had a mean(s.d.) age of 44.7(11.5) years and a mean(s.d.) baseline BMI of 44.6(7.7) kg/m2. In expert centres, 13.74% of the 3020 patients who underwent primary robotic Roux-en-Y gastric bypass and 5.9% of the 4078 patients who underwent primary robotic sleeve gastrectomy presented with greater than or equal to one complication within 90 postoperative days. No patient died and 1.1% of patients had adverse events related to the robotic platform. When compared with laparoscopic benchmarks, robotic Roux-en-Y gastric bypass had lower benchmark cut-offs for hospital stay, postoperative bleeding, and marginal ulceration, but the duration of the operation was 42 min longer. For most surgical outcomes, robotic sleeve gastrectomy outperformed laparoscopic sleeve gastrectomy with a comparable duration of the operation. In robotic learning phase centres, outcomes were within the established benchmarks only for low-risk robotic Roux-en-Y gastric bypass. CONCLUSION: The newly established benchmarks suggest that robotic bariatric surgery may enhance surgical safety compared with laparoscopic bariatric surgery; however, the duration of the operation for robotic Roux-en-Y gastric bypass is longer

Large-scale external validation and comparison of prognostic models: an application to chronic obstructive pulmonary disease

Background: External validations and comparisons of prognostic models or scores are a prerequisite for their use in routine clinical care but are lacking in most medical fields including chronic obstructive pulmonary disease (COPD). Our aim was to externally validate and concurrently compare prognostic scores for 3-year all-cause mortality in mostly multimorbid patients with COPD. Methods: We relied on 24 cohort studies of the COPD Cohorts Collaborative International Assessment consortium, corresponding to primary, secondary, and tertiary care in Europe, the Americas, and Japan. These studies include globally 15,762 patients with COPD (1871 deaths and 42,203 person years of follow-up). We used network meta-analysis adapted to multiple score comparison (MSC), following a frequentist two-stage approach; thus, we were able to compare all scores in a single analytical framework accounting for correlations among scores within cohorts. We assessed transitivity, heterogeneity, and inconsistency and provided a performance ranking of the prognostic scores. Results: Depending on data availability, between two and nine prognostic scores could be calculated for each cohort. The BODE score (body mass index, airflow obstruction, dyspnea, and exercise capacity) had a median area under the curve (AUC) of 0.679 [1st quartile-3rd quartile = 0.655-0.733] across cohorts. The ADO score (age, dyspnea, and airflow obstruction) showed the best performance for predicting mortality (difference AUC(ADO) - AUC(BODE) = 0.015 [95% confidence interval (CI) = - 0.002 to 0.032]; p = 0.08) followed by the updated BODE (AUCBODE updated - AUCBODE = 0.008 [95% CI = -0.005 to +0.022]; p = 0.23). The assumption of transitivity was not violated. Heterogeneity across direct comparisons was small, and we did not identify any local or global inconsistency. Conclusions: Our analyses showed best discriminatory performance for the ADO and updated BODE scores in patients with COPD. A limitation to be addressed in future studies is the extension of MSC network meta-analysis to measures of calibration. MSC network meta-analysis can be applied to prognostic scores in any medical field to identify the best scores, possibly paving the way for stratified medicine, public health, and research

Prediction models for exacerbations in patients with COPD

Personalised medicine aims to tailor medical decisions to the individual patient. A possible approach is to stratify patients according to the risk of adverse outcomes such as exacerbations in chronic obstructive pulmonary disease (COPD). Risk-stratified approaches are particularly attractive for drugs like inhaled corticosteroids or phosphodiesterase-4 inhibitors that reduce exacerbations but are associated with harms. However, it is currently not clear which models are best to predict exacerbations in patients with COPD. Therefore, our aim was to identify and critically appraise studies on models that predict exacerbations in COPD patients. Out of 1382 studies, 25 studies with 27 prediction models were included. The prediction models showed great heterogeneity in terms of number and type of predictors, time horizon, statistical methods and measures of prediction model performance. Only two out of 25 studies validated the developed model, and only one out of 27 models provided estimates of individual exacerbation risk, only three out of 27 prediction models used high-quality statistical approaches for model development and evaluation. Overall, none of the existing models fulfilled the requirements for risk-stratified treatment to personalise COPD care. A more harmonised approach to develop and validate high- quality prediction models is needed to move personalised COPD medicine forward

Axisymmetric Hybrid Plasma Model for Hall Effect Thrusters

Hall Effect Thrusters (HETs) are nowadays widely used for satellite applications because of their efficiency and robustness compared to other electric propulsion devices. Computational modelling of plasma in HETs is interesting for several reasons: it can be used to predict thrusters’ operative life; moreover, it provides a better understanding of the physical behaviour of this device and can be used to optimize the next generation of thrusters. In this work, the discharge within the accelerating channel and near-plume of HETs has been modelled by means of an axisymmetric hybrid approach: a set of fluid equations for electrons has been solved to get electron temperatures, plasma potential and the discharge current, whereas a Particle-In-Cell (PIC) sub-model has been developed to capture the behaviour of neutrals and ions. A two-region electron mobility model has been incorporated. It includes electron–neutral/ion collisions and uses empirical constants, that vary as a continuous function of axial coordinates, to take into account electron–wall collisions and Bohm diffusion/SEE effects. An SPT-100 thruster has been selected for the verification of the model because of the availability of reliable numerical and experimental data. The results of the presented simulations show that the code is able to describe plasma discharge reproducing, with consistency, the physics within the accelerating channel of HETs. A small discrepancy in the experimental magnitude of ions’ expansion, due probably to boundary condition effects, has been found

Correction to: Multiple Score Comparison: a network meta-analysis approach to comparison and external validation of prognostic scores

Correction Following publication of the original article [1], a member of the writing group reported that his name is misspelt. The paper should appear in Pubmed under “Ter Riet G”, bot as “Riet GT”

Additional file 1: of Multiple Score Comparison: a network meta-analysis approach to comparison and external validation of prognostic scores

Multiple Score Comparison: A network meta-analysis approach to comparison and external validation of prognostic scores. Additional statistical background is provided, as well as further details on heterogeneity, transitivity, inconsistency, indirect evidence, and multiple imputation, as well as a table summarizing the scores compared. (DOCX 750 kb