Practical guidance for applying the ADNEX model from the IOTA group to discriminate between different subtypes of adnexal tumors.

All gynecologists are faced with ovarian tumors on a regular basis, and the accurate preoperative diagnosis of these masses is important because appropriate management depends on the type of tumor. Recently, the International Ovarian Tumor Analysis (IOTA) consortium published the Assessment of Different NEoplasias in the adneXa (ADNEX) model, the first risk model that differentiates between benign and four types of malignant ovarian tumors: borderline, stage I cancer, stage II-IV cancer, and secondary metastatic cancer. This approach is novel compared to existing tools that only differentiate between benign and malignant tumors, and therefore questions may arise on how ADNEX can be used in clinical practice. In the present paper, we first provide an in-depth discussion about the predictors used in ADNEX and the ability for risk prediction with different tumor histologies. Furthermore, we formulate suggestions about the selection and interpretation of risk cut-offs for patient stratification and choice of appropriate clinical management. This is illustrated with a few example patients. We cannot propose a generally applicable algorithm with fixed cut-offs, because (as with any risk model) this depends on the specific clinical setting in which the model will be used. Nevertheless, this paper provides a guidance on how the ADNEX model may be adopted into clinical practice

Random-effects meta-analysis of the clinical utility of tests and prediction models

The use of data from multiple studies or centers for the validation of a clinical test or a multivariable prediction model allows researchers to investigate the test's/model's performance in multiple settings and populations. Recently, meta-analytic techniques have been proposed to summarize discrimination and calibration across study populations. Here, we rather consider performance in terms of net benefit, which is a measure of clinical utility that weighs the benefits of true positive classifications against the harms of false positives. We posit that it is important to examine clinical utility across multiple settings of interest. This requires a suitable meta-analysis method, and we propose a Bayesian trivariate random-effects meta-analysis of sensitivity, specificity, and prevalence. Across a range of chosen harm-to-benefit ratios, this provides a summary measure of net benefit, a prediction interval, and an estimate of the probability that the test/model is clinically useful in a new setting. In addition, the prediction interval and probability of usefulness can be calculated conditional on the known prevalence in a new setting. The proposed methods are illustrated by 2 case studies: one on the meta-analysis of published studies on ear thermometry to diagnose fever in children and one on the validation of a multivariable clinical risk prediction model for the diagnosis of ovarian cancer in a multicenter dataset. Crucially, in both case studies the clinical utility of the test/model was heterogeneous across settings, limiting its usefulness in practice. This emphasizes that heterogeneity in clinical utility should be assessed before a test/model is routinely implemented

How to develop, externally validate, and update multinomial prediction models

Multinomial prediction models (MPMs) have a range of potential applications across healthcare where the primary outcome of interest has multiple nominal or ordinal categories. However, the application of MPMs is scarce, which may be due to the added methodological complexities that they bring. This article provides a guide of how to develop, externally validate, and update MPMs. Using a previously developed and validated MPM for treatment outcomes in rheumatoid arthritis as an example, we outline guidance and recommendations for producing a clinical prediction model using multinomial logistic regression. This article is intended to supplement existing general guidance on prediction model research. This guide is split into three parts: 1) Outcome definition and variable selection, 2) Model development, and 3) Model evaluation (including performance assessment, internal and external validation, and model recalibration). We outline how to evaluate and interpret the predictive performance of MPMs. R code is provided. We recommend the application of MPMs in clinical settings where the prediction of a nominal polytomous outcome is of interest. Future methodological research could focus on MPM-specific considerations for variable selection and sample size criteria for external validation

Burnout, well-being and defensive medical practice among obstetricians and gynaecologists in the UK: cross-sectional survey study

Objectives: To determine the prevalence of burnout in doctors practising obstetrics and gynaecology, and assess the association with defensive medical practice and self-reported wellbeing. Design: Nationwide online cross-sectional survey study; December 2017-March 2018. Setting: Hospitals in the United Kingdom Participants: 5661 practising Obstetrics and Gynaecology consultants, specialty and associate specialist doctors and trainees registered with the Royal College of Obstetricians and Gynaecologists Primary and Secondary Outcome Measures: Prevalence of burnout using the Maslach Burnout Inventory and defensive medical practice (avoiding cases or procedures, overprescribing, over-referral) using a 12-item questionnaire. The odds ratios of burnout with defensive medical practice and self-reported wellbeing. Results: 3102/5661 doctors (55%) completed the survey. 3073/3102 (99%) met the inclusion criteria (1462 consultants, 1357 trainees and 254 specialty and associate specialist doctors). 1116/3073 (36%) doctors met the burnout criteria, with levels highest amongst trainees (580/1357 [43%]). 258/1116 (23%) doctors with burnout reported increased defensive practice compared to 142/1957 (7%) without (adjusted odds ratio 4.35, 95% CI 3.46 to 5.49). Odds ratios of burnout with wellbeing items varied between 1.38 and 6.37, and were highest for anxiety (3.59, 95% CI 3.07 to 4.21), depression (4.05, 95% CI 3.26 to 5.04), and suicidal thoughts (6.37, 95% CI 95% CI 3.95 to 10.7). In multivariable logistic regression, being of younger age, white or ‘other’ ethnicity, and graduating with a medical degree from the UK or Ireland had the strongest associations with burnout. Conclusions: High levels of burnout were observed in obstetricians and gynaecologists and particularly amongst trainees. Burnout was associated with both increased defensive medical practice and worse doctor wellbeing. These findings have implications for the wellbeing and retention of doctors as well as the quality of patient care, and may help to inform the content of future interventions aimed at preventing burnout and improving patient safety

Does ignoring clustering in multicenter data influence the performance of prediction models? A simulation study

Clinical risk prediction models are increasingly being developed and validated on multicenter datasets. In this article, we present a comprehensive framework for the evaluation of the predictive performance of prediction models at the center level and the population level, considering population-averaged predictions, center-specific predictions, and predictions assuming an average random center effect. We demonstrated in a simulation study that calibration slopes do not only deviate from one because of over- or underfitting of patterns in the development dataset, but also as a result of the choice of the model (standard versus mixed effects logistic regression), the type of predictions (marginal versus conditional versus assuming an average random effect), and the level of model validation (center versus population). In particular, when data is heavily clustered (ICC 20%), center-specific predictions offer the best predictive performance at the population level and the center level. We recommend that models should reflect the data structure, while the level of model validation should reflect the research question

Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study

Objectives To develop a risk prediction model to preoperatively discriminate between benign, borderline, stage I invasive, stage II-IV invasive, and secondary metastatic ovarian tumours. Design Observational diagnostic study using prospectively collected clinical and ultrasound data. Setting 24 ultrasound centres in 10 countries. Participants Women with an ovarian (including para-ovarian and tubal) mass and who underwent a standardised ultrasound examination before surgery. The model was developed on 3506 patients recruited between 1999 and 2007, temporally validated on 2403 patients recruited between 2009 and 2012, and then updated on all 5909 patients. Main outcome measures Histological classification and surgical staging of the mass. Results The Assessment of Different NEoplasias in the adneXa (ADNEX) model contains three clinical and six ultrasound predictors: age, serum CA-125 level, type of centre (oncology centres v other hospitals), maximum diameter of lesion, proportion of solid tissue, more than 10 cyst locules, number of papillary projections, acoustic shadows, and ascites. The area under the receiver operating characteristic curve (AUC) for the classic discrimination between benign and malignant tumours was 0.94 (0.93 to 0.95) on temporal validation. The AUC was 0.85 for benign versus borderline, 0.92 for benign versus stage I cancer, 0.99 for benign versus stage II-IV cancer, and 0.95 for benign versus secondary metastatic. AUCs between malignant subtypes varied between 0.71 and 0.95, with an AUC of 0.75 for borderline versus stage I cancer and 0.82 for stage II-IV versus secondary metastatic. Calibration curves showed that the estimated risks were accurate. Conclusions The ADNEX model discriminates well between benign and malignant tumours and offers fair to excellent discrimination between four types of ovarian malignancy. The use of ADNEX has the potential to improve triage and management decisions and so reduce morbidity and mortality associated with adnexal pathology