12 research outputs found

    Does poor methodological quality of prediction modeling studies translate to poor model performance?: An illustration in traumatic brain injury

    No full text
    BACKGROUND: Prediction modeling studies often have methodological limitations, which may compromise model performance in new patients and settings. We aimed to examine the relation between methodological quality of model development studies and their performance at external validation. METHODS: We systematically searched for externally validated multivariable prediction models that predict functional outcome following moderate or severe traumatic brain injury. Risk of bias and applicability of development studies was assessed with the Prediction model Risk Of Bias Assessment Tool (PROBAST). Each model was rated for its presentation with sufficient detail to be used in practice. Model performance was described in terms of discrimination (AUC), and calibration. Delta AUC (dAUC) was calculated to quantify the percentage change in discrimination between development and validation for all models. Generalized estimation equations (GEE) were used to examine the relation between methodological quality and dAUC while controlling for clustering. RESULTS: We included 54 publications, presenting ten development studies of 18 prediction models, and 52 external validation studies, including 245 unique validations. Two development studies (four models) were found to have low risk of bias (RoB). The other eight publications (14 models) showed high or unclear RoB. The median dAUC was positive in low RoB models (dAUC 8%, [IQR - 4% to 21%]) and negative in high RoB models (dAUC - 18%, [IQR - 43% to 2%]). The GEE showed a larger average negative change in discrimination for high RoB models (- 32% (95% CI: - 48 to - 15) and unclear RoB models (- 13% (95% CI: - 16 to - 10)) compared to that seen in low RoB models. CONCLUSION: Lower methodological quality at model development associates with poorer model performance at external validation. Our findings emphasize the importance of adherence to methodological principles and reporting guidelines in prediction modeling studies

    Does poor methodological quality of prediction modeling studies translate to poor model performance?:An illustration in traumatic brain injury

    Get PDF
    BACKGROUND: Prediction modeling studies often have methodological limitations, which may compromise model performance in new patients and settings. We aimed to examine the relation between methodological quality of model development studies and their performance at external validation. METHODS: We systematically searched for externally validated multivariable prediction models that predict functional outcome following moderate or severe traumatic brain injury. Risk of bias and applicability of development studies was assessed with the Prediction model Risk Of Bias Assessment Tool (PROBAST). Each model was rated for its presentation with sufficient detail to be used in practice. Model performance was described in terms of discrimination (AUC), and calibration. Delta AUC (dAUC) was calculated to quantify the percentage change in discrimination between development and validation for all models. Generalized estimation equations (GEE) were used to examine the relation between methodological quality and dAUC while controlling for clustering. RESULTS: We included 54 publications, presenting ten development studies of 18 prediction models, and 52 external validation studies, including 245 unique validations. Two development studies (four models) were found to have low risk of bias (RoB). The other eight publications (14 models) showed high or unclear RoB. The median dAUC was positive in low RoB models (dAUC 8%, [IQR − 4% to 21%]) and negative in high RoB models (dAUC − 18%, [IQR − 43% to 2%]). The GEE showed a larger average negative change in discrimination for high RoB models (− 32% (95% CI: − 48 to − 15) and unclear RoB models (− 13% (95% CI: − 16 to − 10)) compared to that seen in low RoB models. CONCLUSION: Lower methodological quality at model development associates with poorer model performance at external validation. Our findings emphasize the importance of adherence to methodological principles and reporting guidelines in prediction modeling studies. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s41512-022-00122-0

    Does poor methodological quality of prediction modeling studies translate to poor model performance? An illustration in traumatic brain injury

    No full text
    Background Prediction modeling studies often have methodological limitations, which may compromise model performance in new patients and settings. We aimed to examine the relation between methodological quality of model development studies and their performance at external validation. Methods We systematically searched for externally validated multivariable prediction models that predict functional outcome following moderate or severe traumatic brain injury. Risk of bias and applicability of development studies was assessed with the Prediction model Risk Of Bias Assessment Tool (PROBAST). Each model was rated for its presentation with sufficient detail to be used in practice. Model performance was described in terms of discrimination (AUC), and calibration. Delta AUC (dAUC) was calculated to quantify the percentage change in discrimination between development and validation for all models. Generalized estimation equations (GEE) were used to examine the relation between methodological quality and dAUC while controlling for clustering. Results We included 54 publications, presenting ten development studies of 18 prediction models, and 52 external validation studies, including 245 unique validations. Two development studies (four models) were found to have low risk of bias (RoB). The other eight publications (14 models) showed high or unclear RoB. The median dAUC was positive in low RoB models (dAUC 8%, [IQR − 4% to 21%]) and negative in high RoB models (dAUC − 18%, [IQR − 43% to 2%]). The GEE showed a larger average negative change in discrimination for high RoB models (− 32% (95% CI: − 48 to − 15) and unclear RoB models (− 13% (95% CI: − 16 to − 10)) compared to that seen in low RoB models. Conclusion Lower methodological quality at model development associates with poorer model performance at external validation. Our findings emphasize the importance of adherence to methodological principles and reporting guidelines in prediction modeling studies

    Tosap ed occupazioni di aree destinate ad attività mercatali: gli effetti di un’attività comunale di riordino degli spazi per attività commerciali sull’applicazione della tassa per occupazione di spazi ed aree pubbliche”.

    Get PDF
    The majority of traumatic brain injuries (TBIs) are categorized as mild, according to a baseline Glasgow Coma Scale (GCS) score of 13-15. Prognostic models that were developed to predict functional outcome and persistent post-concussive symptoms (PPCS) after mild TBI have rarely been externally validated. We aimed to externally validate models predicting 3-12-month Glasgow Outcome Scale Extended (GOSE) or PPCS in adults with mild TBI. We analyzed data from the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) project, which included 2862 adults with mild TBI, with 6-month GOSE available for 2374 and Rivermead Post-Concussion Symptoms Questionnaire (RPQ) results available for 1605 participants. Model performance was evaluated based on calibration (graphically and characterized by slope and intercept) and discrimination (C-index). We validated five published models for 6-month GOSE and three for 6-month PPCS scores. The models used different cutoffs for outcome and some included symptoms measured 2 weeks post-injury. Discriminative ability varied substantially (C-index between 0.58 and 0.79). The models developed in the Corticosteroid Randomisation After Significant Head Injury (CRASH) trial for prediction of GOSE <5 discriminated best (C-index 0.78 and 0.79), but were poorly calibrated. The best performing models for PPCS included 2-week symptoms (C-index 0.75 and 0.76). In conclusion, none of the prognostic models for early prediction of GOSE and PPCS has both good calibration and discrimination in persons with mild TBI. In future studies, prognostic models should be tailored to the population with mild TBI, predicting relevant end-points based on readily available predictors

    Prediction of Global Functional Outcome and Post-Concussive Symptoms after Mild Traumatic Brain Injury: External Validation of Prognostic Models in the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) Study

    No full text
    The majority of traumatic brain injuries (TBIs) are categorized as mild, according to a baseline Glasgow Coma Scale (GCS) score of 13-15. Prognostic models that were developed to predict functional outcome and persistent post-concussive symptoms (PPCS) after mild TBI have rarely been externally validated. We aimed to externally validate models predicting 3-12-month Glasgow Outcome Scale Extended (GOSE) or PPCS in adults with mild TBI. We analyzed data from the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) project, which included 2862 adults with mild TBI, with 6-month GOSE available for 2374 and Rivermead Post-Concussion Symptoms Questionnaire (RPQ) results available for 1605 participants. Model performance was evaluated based on calibration (graphically and characterized by slope and intercept) and discrimination (C-index). We validated five published models for 6-month GOSE and three for 6-month PPCS scores. The models used different cutoffs for outcome and some included symptoms measured 2 weeks post-injury. Discriminative ability varied substantially (C-index between 0.58 and 0.79). The models developed in the Corticosteroid Randomisation After Significant Head Injury (CRASH) trial for prediction of GOS

    Serum metabolome associated with severity of acute traumatic brain injury.

    Get PDF
    Complex metabolic disruption is a crucial aspect of the pathophysiology of traumatic brain injury (TBI). Associations between this and systemic metabolism and their potential prognostic value are poorly understood. Here, we aimed to describe the serum metabolome (including lipidome) associated with acute TBI within 24 h post-injury, and its relationship to severity of injury and patient outcome. We performed a comprehensive metabolomics study in a cohort of 716 patients with TBI and non-TBI reference patients (orthopedic, internal medicine, and other neurological patients) from the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) cohort. We identified panels of metabolites specifically associated with TBI severity and patient outcomes. Choline phospholipids (lysophosphatidylcholines, ether phosphatidylcholines and sphingomyelins) were inversely associated with TBI severity and were among the strongest predictors of TBI patient outcomes, which was further confirmed in a separate validation dataset of 558 patients. The observed metabolic patterns may reflect different pathophysiological mechanisms, including protective changes of systemic lipid metabolism aiming to maintain lipid homeostasis in the brain

    Effect of frailty on 6-month outcome after traumatic brain injury:a multicentre cohort study with external validation

    No full text
    Background: Frailty is known to be associated with poorer outcomes in individuals admitted to hospital for medical conditions requiring intensive care. However, little evidence is available for the effect of frailty on patients’ outcomes after traumatic brain injury. Many frailty indices have been validated for clinical practice and show good performance to predict clinical outcomes. However, each is specific to a particular clinical context. We aimed to develop a frailty index to predict 6-month outcomes in patients after a traumatic brain injury. Methods: A cumulative deficit approach was used to create a novel frailty index based on 30 items dealing with disease states, current medications, and laboratory values derived from data available from CENTER-TBI, a prospective, longitudinal observational study of patients with traumatic brain injury presenting within 24 h of injury and admitted to a ward or an intensive care unit at 65 centres in Europe between Dec 19, 2014, and Dec 17, 2017. From the individual cumulative CENTER-TBI frailty index (range 0–30), we obtained a standardised value (range 0–1), with high scores indicating higher levels of frailty. The effect of frailty on 6-month outcome evaluated with the extended Glasgow Outcome Scale (GOSE) was assessed through a proportional odds logistic model adjusted for known outcome predictors. An unfavourable outcome was defined as death or severe disability (GOSE score ≤4). External validation was performed on data from TRACK-TBI, a prospective observational study co-designed with CENTER-TBI, which enrolled patients with traumatic brain injury at 18 level I trauma centres in the USA from Feb 26, 2014, to July 27, 2018. CENTER-TBI is registered with ClinicalTrials.gov, NCT02210221; TRACK-TBI is registered at ClinicalTrials.gov, NCT02119182. Findings: 2993 participants (median age was 51 years [IQR 30–67], 2058 [69%] were men) were included in this analysis. The overall median CENTER-TBI frailty index score was 0·07 (IQR 0·03–0·15), with a median score of 0·17 (0·08–0·27) in older adults (aged ≥65 years). The CENTER-TBI frailty index score was significantly associated with the probability of an increasingly unfavourable outcome (cumulative odds ratio [OR] 1·03, 95% CI 1·02–1·04; p&lt;0·0001), and the association was stronger for participants admitted to hospital wards (1·04, 1·03–1·06, p&lt;0·0001) compared with those admitted to the intensive care unit (1·02, 1·01–1·03 p&lt;0·0001). External validation of the CENTER-TBI frailty index in data from the TRACK-TBI (n=1667) cohort supported the robustness and reliability of these findings. The overall median TRACK-TBI frailty index score was 0·03 (IQR 0–0·10), with the frailty index score significantly associated with the risk of an increasingly unfavourable outcome in patients admitted to hospital wards (cumulative OR 1·05, 95% CI 1·03–1·08; p&lt;0·0001), but not in those admitted to the intensive care unit (1·01, 0·99–1·03; p=0·43). Interpretation: We developed and externally validated a frailty index specific to traumatic brain injury. Risk of unfavourable outcome was significantly increased in participants with a higher CENTER-TBI frailty index score, regardless of age. Frailty identification could help to individualise rehabilitation approaches aimed at mitigating effects of frailty in patients with traumatic brain injury. Funding: European Union, Hannelore Kohl Stiftung, OneMind, Integra LifeSciences Corporation, NeuroTrauma Sciences, NIH-NINDS–TRACK-TBI, US Department of Defense.</p

    Frequency and predictors of headache in the first 12 months after traumatic brain injury: results from CENTER-TBI

    Get PDF
    Background: Headache is a prevalent and debilitating symptom following traumatic brain injury (TBI). Large-scale, prospective cohort studies are needed to establish long-term headache prevalence and associated factors after TBI. This study aimed to assess the frequency and severity of headache after TBI and determine whether sociodemographic factors, injury severity characteristics, and pre- and post-injury comorbidities predicted changes in headache frequency and severity during the first 12 months after injury. Methods: A large patient sample from the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) prospective observational cohort study was used. Patients were stratified based on their clinical care pathway: admitted to an emergency room (ER), a ward (ADM) or an intensive care unit (ICU) in the acute phase. Headache was assessed using a single item from the Rivermead Post-Concussion Symptoms Questionnaire measured at baseline, 3, 6 and 12 months after injury. Mixed-effect logistic regression analyses were applied to investigate changes in headache frequency and associated predictors. Results: A total of 2,291 patients responded to the headache item at baseline. At study enrolment, 59.3% of patients reported acute headache, with similar frequencies across all strata. Female patients and those aged up to 40 years reported a higher frequency of headache at baseline compared to males and older adults. The frequency of severe headache was highest in patients admitted to the ICU. The frequency of headache in the ER stratum decreased substantially from baseline to 3 months and remained from 3 to 6 months. Similar trajectory trends were observed in the ICU and ADM strata across 12 months. Younger age, more severe TBI, fatigue, neck pain and vision problems were among the predictors of more severe headache over time. More than 25% of patients experienced headache at 12 months after injury. Conclusions: Headache is a common symptom after TBI, especially in female and younger patients. It typically decreases in the first 3 months before stabilising. However, more than a quarter of patients still experienced headache at 12 months after injury. Translational research is needed to advance the clinical decision-making process and improve targeted medical treatment for headache. Trial registration: ClinicalTrials.gov NCT02210221.</p
    corecore