Developing risk models for multicenter data using standard logistic regression produced suboptimal predictions: A simulation study

Although multicenter data are common, many prediction model studies ignore this during model development. The objective of this study is to evaluate the predictive performance of regression methods for developing clinical risk prediction models using multicenter data, and provide guidelines for practice. We compared the predictive performance of standard logistic regression, generalized estimating equations, random intercept logistic regression, and fixed effects logistic regression. First, we presented a case study on the diagnosis of ovarian cancer. Subsequently, a simulation study investigated the performance of the different models as a function of the amount of clustering, development sample size, distribution of center-specific intercepts, the presence of a center-predictor interaction, and the presence of a dependency between center effects and predictors. The results showed that when sample sizes were sufficiently large, conditional models yielded calibrated predictions, whereas marginal models yielded miscalibrated predictions. Small sample sizes led to overfitting and unreliable predictions. This miscalibration was worse with more heavily clustered data. Calibration of random intercept logistic regression was better than that of standard logistic regression even when center-specific intercepts were not normally distributed, a center-predictor interaction was present, center effects and predictors were dependent, or when themodel was applied in a new center. Therefore, to make reliable predictions in a specific center, we recommend random intercept logistic regression.status: publishe

Developing risk models for multicenter data using standard logistic regression produced suboptimal predictions:A simulation study

Although multicenter data are common, many prediction model studies ignore this during model development. The objective of this study is to evaluate the predictive performance of regression methods for developing clinical risk prediction models using multicenter data, and provide guidelines for practice. We compared the predictive performance of standard logistic regression, generalized estimating equations, random intercept logistic regression, and fixed effects logistic regression. First, we presented a case study on the diagnosis of ovarian cancer. Subsequently, a simulation study investigated the performance of the different models as a function of the amount of clustering, development sample size, distribution of center-specific intercepts, the presence of a center-predictor interaction, and the presence of a dependency between center effects and predictors. The results showed that when sample sizes were sufficiently large, conditional models yielded calibrated predictions, whereas marginal models yielded miscalibrated predictions. Small sample sizes led to overfitting and unreliable predictions. This miscalibration was worse with more heavily clustered data. Calibration of random intercept logistic regression was better than that of standard logistic regression even when center-specific intercepts were not normally distributed, a center-predictor interaction was present, center effects and predictors were dependent, or when the model was applied in a new center. Therefore, to make reliable predictions in a specific center, we recommend random intercept logistic regression.</p

Burnout, well-being and defensive medical practice among obstetricians and gynaecologists in the UK: cross-sectional survey study

OBJECTIVES: To determine the prevalence of burnout in doctors practising obstetrics and gynaecology, and assess the association with defensive medical practice and self-reported well-being. DESIGN: Nationwide online cross-sectional survey study; December 2017-March 2018. SETTING: Hospitals in the UK. PARTICIPANTS: 5661 practising obstetrics and gynaecology consultants, specialty and associate specialist doctors and trainees registered with the Royal College of Obstetricians and Gynaecologists. PRIMARY AND SECONDARY OUTCOME MEASURES: Prevalence of burnout using the Maslach Burnout Inventory and defensive medical practice (avoiding cases or procedures, overprescribing, over-referral) using a 12-item questionnaire. The odds ratios (OR) of burnout with defensive medical practice and self-reported well-being. RESULTS: 3102/5661 doctors (55%) completed the survey. 3073/3102 (99%) met the inclusion criteria (1462 consultants, 1357 trainees and 254 specialty and associate specialist doctors). 1116/3073 (36%) doctors met the burnout criteria, with levels highest amongst trainees (580/1357 (43%)). 258/1116 (23%) doctors with burnout reported increased defensive practice compared with 142/1957 (7%) without (adjusted OR 4.35, 95% CI 3.46 to 5.49). ORs of burnout with well-being items varied between 1.38 and 6.37, and were highest for anxiety (3.59, 95% CI 3.07 to 4.21), depression (4.05, 95% CI 3.26 to 5.04) and suicidal thoughts (6.37, 95% CI 95% CI 3.95 to 10.7). In multivariable logistic regression, being of younger age, white or 'other' ethnicity, and graduating with a medical degree from the UK or Ireland had the strongest associations with burnout. CONCLUSIONS: High levels of burnout were observed in obstetricians and gynaecologists and particularly among trainees. Burnout was associated with both increased defensive medical practice and worse doctor well-being. These findings have implications for the well-being and retention of doctors as well as the quality of patient care, and may help to inform the content of future interventions aimed at preventing burnout and improving patient safety.status: publishe

Posttraumatic stress, anxiety and depression following miscarriage and ectopic pregnancy: a multicenter, prospective, cohort study

BACKGROUND: Early pregnancy losses are common, but their psychologic sequelae are often overlooked. Previous studies have established links between miscarriage and early symptoms of anxiety and depression. However, the incidence of posttraumatic stress symptoms and the psychologic response specifically to ectopic pregnancies have not been investigated. OBJECTIVE: The purpose of this study was to investigate levels of posttraumatic stress, depression, and anxiety in women in the 9 months after early pregnancy loss, with a focus on miscarriage and ectopic pregnancy. Morbidity at 1 month was compared with a control group in healthy pregnancy. STUDY DESIGN: This was a prospective cohort study. Consecutive women were recruited from the early pregnancy and antenatal clinics at 3 London hospitals and received emailed surveys that contained standardized psychologic assessments that included the Hospital Anxiety and Depression Scale and Posttraumatic stress Diagnostic Scale, at 1, 3, and 9 months after loss. Control subjects were assessed after a dating scan. We assessed the proportion of participants who met the screening criteria for posttraumatic stress and moderate/severe anxiety or depression. We used logistic regression to calculate adjusted odds ratios. RESULTS: Seven hundred thirty-seven of 1098 women (67%) with early pregnancy loss (including 537 miscarriages and 116 ectopic pregnancies) and 171 of 187 control subjects (91%) agreed to participate. Four hundred ninety-two of the women with losses (67%) completed the Hospital Anxiety and Depression Scale after 1 month; 426 women (58%) completed it after 3 months, and 338 women (46%) completed it after 9 months. Eighty-seven control subjects (51%) participated. Criteria for posttraumatic stress were met in 29% of women with early pregnancy loss after 1 month and in 18% after 9 months (odds ratio per month, 0.80; 95% confidence interval, 0.72-0.89). Moderate/severe anxiety was reported in 24% after 1 month and in 17% after 9 months (odds ratio per month, 0.69; 95% confidence interval, 0.50-0.94). Moderate/severe depression was reported in 11% of the women after 1 month and 6% of the women after 9 months (odds ratio per month, 0.87; 95% confidence interval, 0.53-1.44). After miscarriage, proportions after 9 months were 16% for posttraumatic stress, 17% for anxiety, and 5% for depression. Corresponding figures after ectopic pregnancy were 21%, 23%, and 11%, respectively. In contrast, among control women with viable pregnancies, 13% reported moderate-to-severe anxiety (odds ratio loss at 1 month vs controls: 2.14; 95% confidence interval, 1.14-4.36), and 2% reported moderate-to-severe depression (odds ratio loss at 1 month vs control subjects: 3.88; 95% confidence interval, 1.27-19.2). CONCLUSION: Women experience high levels of posttraumatic stress, anxiety, and depression after early pregnancy loss. Distress declines over time but remains at clinically important levels at 9 months.status: publishe