Increased endorsement of TRIPOD and other reporting guidelines by high impact factor journals: survey of instructions to authors

OBJECTIVES: To assess the endorsement of reporting guidelines by high impact factor journals over the period 2017-2022, with a specific focus on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. STUDY DESIGN AND SETTING: We searched the online 'instructions to authors' of high impact factor medical journals in February 2017 and in January 2022 for any reference to reporting guidelines and TRIPOD in particular. RESULTS: In 2017, 205 out of 337 (61%) journals mentioned any reporting guideline in their instructions to authors and in 2022 this increased to 245 (73%) journals. A reference to TRIPOD was provided by 27 (8%) journals in 2017 and 67 (20%) in 2022. Of those journals mentioning TRIPOD in 2022, 22% provided a link to the TRIPOD website and 60% linked to TRIPOD information on the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network website. Twenty-five percent of the journals required adherence to TRIPOD. CONCLUSION: About three-quarters of high-impact medical journals endorse the use of reporting guidelines and 20% endorse TRIPOD. Transparent reporting is important in enhancing the usefulness of health research and endorsement by journals plays a critical role in this

The comparative and added prognostic value of biomarkers to the Revised Cardiac Risk Index for preoperative prediction of major adverse cardiac events and all-cause mortality in patients who undergo noncardiac surgery

BACKGROUND: The Revised Cardiac Risk Index (RCRI) is a widely acknowledged prognostic model to estimate preoperatively the probability of developing in-hospital major adverse cardiac events (MACE) in patients undergoing noncardiac surgery. However, the RCRI does not always make accurate predictions, so various studies have investigated whether biomarkers added to or compared with the RCRI could improve this. OBJECTIVES: Primary: To investigate the added predictive value of biomarkers to the RCRI to preoperatively predict in-hospital MACE and other adverse outcomes in patients undergoing noncardiac surgery. Secondary: To investigate the prognostic value of biomarkers compared to the RCRI to preoperatively predict in-hospital MACE and other adverse outcomes in patients undergoing noncardiac surgery. Tertiary: To investigate the prognostic value of other prediction models compared to the RCRI to preoperatively predict in-hospital MACE and other adverse outcomes in patients undergoing noncardiac surgery. SEARCH METHODS: We searched MEDLINE and Embase from 1 January 1999 (the year that the RCRI was published) until 25 June 2020. We also searched ISI Web of Science and SCOPUS for articles referring to the original RCRI development study in that period. SELECTION CRITERIA: We included studies among adults who underwent noncardiac surgery, reporting on (external) validation of the RCRI and: - the addition of biomarker(s) to the RCRI; or - the comparison of the predictive accuracy of biomarker(s) to the RCRI; or - the comparison of the predictive accuracy of the RCRI to other models. Besides MACE, all other adverse outcomes were considered for inclusion. DATA COLLECTION AND ANALYSIS: We developed a data extraction form based on the CHARMS checklist. Independent pairs of authors screened references, extracted data and assessed risk of bias and concerns regarding applicability according to PROBAST. For biomarkers and prediction models that were added or compared to the RCRI in ≥ 3 different articles, we described study characteristics and findings in further detail. We did not apply GRADE as no guidance is available for prognostic model reviews. MAIN RESULTS: We screened 3960 records and included 107 articles. Over all objectives we rated risk of bias as high in ≥ 1 domain in 90% of included studies, particularly in the analysis domain. Statistical pooling or meta-analysis of reported results was impossible due to heterogeneity in various aspects: outcomes used, scale by which the biomarker was added/compared to the RCRI, prediction horizons and studied populations. Added predictive value of biomarkers to the RCRI Fifty-one studies reported on the added value of biomarkers to the RCRI. Sixty-nine different predictors were identified derived from blood (29%), imaging (33%) or other sources (38%). Addition of NT-proBNP, troponin or their combination improved the RCRI for predicting MACE (median delta c-statistics: 0.08, 0.14 and 0.12 for NT-proBNP, troponin and their combination, respectively). The median total net reclassification index (NRI) was 0.16 and 0.74 after addition of troponin and NT-proBNP to the RCRI, respectively. Calibration was not reported. To predict myocardial infarction, the median delta c-statistic when NT-proBNP was added to the RCRI was 0.09, and 0.06 for prediction of all-cause mortality and MACE combined. For BNP and copeptin, data were not sufficient to provide results on their added predictive performance, for any of the outcomes. Comparison of the predictive value of biomarkers to the RCRI Fifty-one studies assessed the predictive performance of biomarkers alone compared to the RCRI. We identified 60 unique predictors derived from blood (38%), imaging (30%) or other sources, such as the American Society of Anesthesiologists (ASA) classification (32%). Predictions were similar between the ASA classification and the RCRI for all studied outcomes. In studies different from those identified in objective 1, the median delta c-statistic was 0.15 and 0.12 in favour of BNP and NT-proBNP alone, respectively, when compared to the RCRI, for the prediction of MACE. For C-reactive protein, the predictive performance was similar to the RCRI. For other biomarkers and outcomes, data were insufficient to provide summary results. One study reported on calibration and none on reclassification. Comparison of the predictive value of other prognostic models to the RCRI Fifty-two articles compared the predictive ability of the RCRI to other prognostic models. Of these, 42% developed a new prediction model, 22% updated the RCRI, or another prediction model, and 37% validated an existing prediction model. None of the other prediction models showed better performance in predicting MACE than the RCRI. To predict myocardial infarction and cardiac arrest, ACS-NSQIP-MICA had a higher median delta c-statistic of 0.11 compared to the RCRI. To predict all-cause mortality, the median delta c-statistic was 0.15 higher in favour of ACS-NSQIP-SRS compared to the RCRI. Predictive performance was not better for CHADS2, CHA2DS2-VASc, R2CHADS2, Goldman index, Detsky index or VSG-CRI compared to the RCRI for any of the outcomes. Calibration and reclassification were reported in only one and three studies, respectively. AUTHORS' CONCLUSIONS: Studies included in this review suggest that the predictive performance of the RCRI in predicting MACE is improved when NT-proBNP, troponin or their combination are added. Other studies indicate that BNP and NT-proBNP, when used in isolation, may even have a higher discriminative performance than the RCRI. There was insufficient evidence of a difference between the predictive accuracy of the RCRI and other prediction models in predicting MACE. However, ACS-NSQIP-MICA and ACS-NSQIP-SRS outperformed the RCRI in predicting myocardial infarction and cardiac arrest combined, and all-cause mortality, respectively. Nevertheless, the results cannot be interpreted as conclusive due to high risks of bias in a majority of papers, and pooling was impossible due to heterogeneity in outcomes, prediction horizons, biomarkers and studied populations. Future research on the added prognostic value of biomarkers to existing prediction models should focus on biomarkers with good predictive accuracy in other settings (e.g. diagnosis of myocardial infarction) and identification of biomarkers from omics data. They should be compared to novel biomarkers with so far insufficient evidence compared to established ones, including NT-proBNP or troponins. Adherence to recent guidance for prediction model studies (e.g. TRIPOD; PROBAST) and use of standardised outcome definitions in primary studies is highly recommended to facilitate systematic review and meta-analyses in the future

A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes

It is widely recommended that any developed—diagnostic or prognostic—prediction model is externally validated in terms of its predictive performance measured by calibration and discrimination. When multiple validations have been performed, a systematic review followed by a formal meta-analysis helps to summarize overall performance across multiple settings, and reveals under which circumstances the model performs suboptimal (alternative poorer) and may need adjustment. We discuss how to undertake meta-analysis of the performance of prediction models with either a binary or a time-to-event outcome. We address how to deal with incomplete availability of study-specific results (performance estimates and their precision), and how to produce summary estimates of the c -statistic, the observed:expected ratio and the calibration slope. Furthermore, we discuss the implementation of frequentist and Bayesian meta-analysis methods, and propose novel empirically-based prior distributions to improve estimation of between-study heterogeneity in small samples. Finally, we illustrate all methods using two examples: meta-analysis of the predictive performance of EuroSCORE II and of the Framingham Risk Score. All examples and meta-analysis models have been implemented in our newly developed R package “metamisc”