A spline-based tool to assess and visualize the calibration of multiclass risk predictions

When validating risk models (or probabilistic classifiers), calibration is often overlooked. Calibration refers to the reliability of the predicted risks, i.e. whether the predicted risks correspond to observed probabilities. In medical applications this is important because treatment decisions often rely on the estimated risk of disease. The aim of this paper is to present generic tools to assess the calibration of multiclass risk models. We describe a calibration framework based on a vector spline multinomial logistic regression model. This framework can be used to generate calibration plots and calculate the estimated calibration index (ECI) to quantify lack of calibration. We illustrate these tools in relation to risk models used to characterize ovarian tumors. The outcome of the study is the surgical stage of the tumor when relevant and the final histological outcome, which is divided into five classes: benign, borderline malignant, stage I, stage II-IV, and secondary metastatic cancer. The 5909 patients included in the study are randomly split into equally large training and test sets. We developed and tested models using the following algorithms: logistic regression, support vector machines, k nearest neighbors, random forest, naive Bayes and nearest shrunken centroids. Multiclass calibration plots are interesting as an approach to visualizing the reliability of predicted risks. The ECI is a convenient tool for comparing models, but is less informative and interpretable than calibration plots. In our case study, logistic regression and random forest showed the highest degree of calibration, and the naive Bayes the lowest.publisher: Elsevier articletitle: A spline-based tool to assess and visualize the calibration of multiclass risk predictions journaltitle: Journal of Biomedical Informatics articlelink: http://dx.doi.org/10.1016/j.jbi.2014.12.016 content_type: article copyright: Copyright © 2015 Elsevier Inc. All rights reserved.status: publishe

Practical guidance for applying the ADNEX model from the IOTA group to discriminate between different subtypes of adnexal tumors

All gynecologists are faced with ovarian tumors on a regular basis, and the accurate preoperative diagnosis of these masses is important because appropriate management depends on the type of tumor. Recently, the International Ovarian Tumor Analysis (IOTA) consortium published the Assessment of Different NEoplasias in the adneXa (ADNEX) model, the first risk model that differentiates between benign and four types of malignant ovarian tumors: borderline, stage I cancer, stage II-IV cancer, and secondary metastatic cancer. This approach is novel compared to existing tools that only differentiate between benign and malignant tumors, and therefore questions may arise on how ADNEX can be used in clinical practice. In the present paper, we first provide an in-depth discussion about the predictors used in ADNEX and the ability for risk prediction with different tumor histologies. Furthermore, we formulate suggestions about the selection and interpretation of risk cut-offs for patient stratification and choice of appropriate clinical management. This is illustrated with a few example patients. We cannot propose a generally applicable algorithm with fixed cut-offs, because (as with any risk model) this depends on the specific clinical setting in which the model will be used. Nevertheless, this paper provides a guidance on how the ADNEX model may be adopted into clinical practice.status: publishe

Retrospective surveillance of viable Bacillus cereus group contaminations in commercial food and feed vitamin B2 products sold on the Belgian market using whole-genome sequencing

Bacillus cereus&nbsp;is a spore-forming bacterium that occurs as a contaminant in food and feed, occasionally resulting in food poisoning through the production of various toxins. In this study, we retrospectively characterized viable&nbsp;B. cereus sensu lato&nbsp;(s.l.) isolates originating from commercial vitamin B2&nbsp;feed and food additives collected between 2016 and 2022 by the Belgian Federal Agency for the Safety of the Food Chain from products sold on the Belgian market. In total, 75 collected product samples were cultured on a general medium and, in case of bacterial growth, two isolates per product sample were collected and characterized using whole-genome sequencing (WGS) and subsequently characterized in terms of sequence type (ST), virulence gene profile, antimicrobial resistance (AMR) gene profile, plasmid content, and phylogenomic relationships. Viable&nbsp;B. cereus&nbsp;was identified in 18 of the 75 (24%) tested products, resulting in 36 WGS datasets, which were classified into eleven different STs, with ST165 (n&nbsp;= 10) and ST32 (n&nbsp;= 8) being the most common. All isolates carried multiple genes encoding virulence factors, including cytotoxin K-2 (52.78%) and cereulide (22.22%). Most isolates were predicted to be resistant to beta-lactam antibiotics (100%) and fosfomycin (88.89%), and a subset was predicted to be resistant to streptothricin (30.56%). Phylogenomic analysis revealed that some isolates obtained from different products were closely related or even identical indicating a likely common origin, whereas for some products the two isolates obtained did not show any close relationship to each other or other isolates found in other products. This study reveals that potentially pathogenic and drug-resistant&nbsp;B. cereus s.l.&nbsp;can be present in food and feed vitamin B2&nbsp;additives that are commercially available, and that more research is warranted to assess whether their presence in these types of products poses a threat to&nbsp;consumers.</p

Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study

OBJECTIVES: To develop a risk prediction model to preoperatively discriminate between benign, borderline, stage I invasive, stage II-IV invasive, and secondary metastatic ovarian tumours. DESIGN: Observational diagnostic study using prospectively collected clinical and ultrasound data. SETTING: 24 ultrasound centres in 10 countries. PARTICIPANTS: Women with an ovarian (including para-ovarian and tubal) mass and who underwent a standardised ultrasound examination before surgery. The model was developed on 3506 patients recruited between 1999 and 2007, temporally validated on 2403 patients recruited between 2009 and 2012, and then updated on all 5909 patients. MAIN OUTCOME MEASURES: Histological classification and surgical staging of the mass. RESULTS: The Assessment of Different NEoplasias in the adneXa (ADNEX) model contains three clinical and six ultrasound predictors: age, serum CA-125 level, type of centre (oncology centres v other hospitals), maximum diameter of lesion, proportion of solid tissue, more than 10 cyst locules, number of papillary projections, acoustic shadows, and ascites. The area under the receiver operating characteristic curve (AUC) for the classic discrimination between benign and malignant tumours was 0.94 (0.93 to 0.95) on temporal validation. The AUC was 0.85 for benign versus borderline, 0.92 for benign versus stage I cancer, 0.99 for benign versus stage II-IV cancer, and 0.95 for benign versus secondary metastatic. AUCs between malignant subtypes varied between 0.71 and 0.95, with an AUC of 0.75 for borderline versus stage I cancer and 0.82 for stage II-IV versus secondary metastatic. Calibration curves showed that the estimated risks were accurate. CONCLUSIONS: The ADNEX model discriminates well between benign and malignant tumours and offers fair to excellent discrimination between four types of ovarian malignancy. The use of ADNEX has the potential to improve triage and management decisions and so reduce morbidity and mortality associated with adnexal pathology.status: publishe