A comparison of machine learning models for predicting urinary incontinence in men with localized prostate cancer

IntroductionUrinary incontinence (UI) is a common side effect of prostate cancer treatment, but in clinical practice, it is difficult to predict. Machine learning (ML) models have shown promising results in predicting outcomes, yet the lack of transparency in complex models known as “black-box” has made clinicians wary of relying on them in sensitive decisions. Therefore, finding a balance between accuracy and explainability is crucial for the implementation of ML models. The aim of this study was to employ three different ML classifiers to predict the probability of experiencing UI in men with localized prostate cancer 1-year and 2-year after treatment and compare their accuracy and explainability. MethodsWe used the ProZIB dataset from the Netherlands Comprehensive Cancer Organization (Integraal Kankercentrum Nederland; IKNL) which contained clinical, demographic, and PROM data of 964 patients from 65 Dutch hospitals. Logistic Regression (LR), Random Forest (RF), and Support Vector Machine (SVM) algorithms were applied to predict (in)continence after prostate cancer treatment. ResultsAll models have been externally validated according to the TRIPOD Type 3 guidelines and their performance was assessed by accuracy, sensitivity, specificity, and AUC. While all three models demonstrated similar performance, LR showed slightly better accuracy than RF and SVM in predicting the risk of UI one year after prostate cancer treatment, achieving an accuracy of 0.75, a sensitivity of 0.82, and an AUC of 0.79. All models for the 2-year outcome performed poorly in the validation set, with an accuracy of 0.6 for LR, 0.65 for RF, and 0.54 for SVM. ConclusionThe outcomes of our study demonstrate the promise of using non-black box models, such as LR, to assist clinicians in recognizing high-risk patients and making informed treatment choices. The coefficients of the LR model show the importance of each feature in predicting results, and the generated nomogram provides an accessible illustration of how each feature impacts the predicted outcome. Additionally, the model’s simplicity and interpretability make it a more appropriate option in scenarios where comprehending the model’s predictions is essential

External validation of a prediction model for timely implementation of innovations in radiotherapy

Background and purpose: The aim of this study was to externally validate a model that predicts timely innovation implementation, which can support radiotherapy professionals to be more successful in innovation implementation. Materials and methods: A multivariate prediction model was built based on the TRIPOD (Transparent Reporting of a multivariate prediction model for Individual Prognosis Or Diagnosis) criteria for a type 4 study (1). The previously built internally validated model had an AUC of 0.82, and was now validated using a completely new multicentre dataset. Innovation projects that took place between 2017–2019 were included in this study. Semi-structured interviews were performed to retrieve the prognostic variables of the previously built model. Projects were categorized according to the size of the project; the success of the project and the presence of pre-defined success factors were analysed. Results: Of the 80 included innovation projects (32.5% technological, 35% organisational and 32.5% treatment innovations), 55% were successfully implemented within the planned timeframe. Comparing the outcome predictions with the observed outcomes of all innovations resulted in an AUC of the external validation of the prediction model of 0.72 (0.60–0.84, 95% CI). Factors related to successful implementation included in the model are sufficient and competent employees, desirability and feasibility, clear goals and processes and the complexity of a project. Conclusion: For the first time, a prediction model focusing on the timely implementation of innovations has been successfully built and externally validated. This model can now be widely used to enable more successful innovation in radiotherapy

External validation of a prediction model for timely implementation of innovations in radiotherapy

Background and purpose: The aim of this study was to externally validate a model that predicts timely innovation implementation, which can support radiotherapy professionals to be more successful in innovation implementation. Materials and methods: A multivariate prediction model was built based on the TRIPOD (Transparent Reporting of a multivariate prediction model for Individual Prognosis Or Diagnosis) criteria for a type 4 study (1). The previously built internally validated model had an AUC of 0.82, and was now validated using a completely new multicentre dataset. Innovation projects that took place between 2017–2019 were included in this study. Semi-structured interviews were performed to retrieve the prognostic variables of the previously built model. Projects were categorized according to the size of the project; the success of the project and the presence of pre-defined success factors were analysed. Results: Of the 80 included innovation projects (32.5% technological, 35% organisational and 32.5% treatment innovations), 55% were successfully implemented within the planned timeframe. Comparing the outcome predictions with the observed outcomes of all innovations resulted in an AUC of the external validation of the prediction model of 0.72 (0.60–0.84, 95% CI). Factors related to successful implementation included in the model are sufficient and competent employees, desirability and feasibility, clear goals and processes and the complexity of a project. Conclusion: For the first time, a prediction model focusing on the timely implementation of innovations has been successfully built and externally validated. This model can now be widely used to enable more successful innovation in radiotherapy

Development and external validation of multivariate prediction models for erectile dysfunction in men with localized prostate cancer.

While the 10-year survival rate for localized prostate cancer patients is very good (>98%), side effects of treatment may limit quality of life significantly. Erectile dysfunction (ED) is a common burden associated with increasing age as well as prostate cancer treatment. Although many studies have investigated the factors affecting erectile dysfunction (ED) after prostate cancer treatment, only limited studies have investigated whether ED can be predicted before the start of treatment. The advent of machine learning (ML) based prediction tools in oncology offers a promising approach to improve the accuracy of prediction and quality of care. Predicting ED may help aid shared decision-making by making the advantages and disadvantages of certain treatments clear, so that a tailored treatment for an individual patient can be chosen. This study aimed to predict ED at 1-year and 2-year post-diagnosis based on patient demographics, clinical data and patient-reported outcomes (PROMs) measured at diagnosis. We used a subset of the ProZIB dataset collected by the Netherlands Comprehensive Cancer Organization (Integraal Kankercentrum Nederland; IKNL) that contained information on 964 localized prostate cancer cases from 69 Dutch hospitals for model training and external validation. Two models were generated using a logistic regression algorithm coupled with Recursive Feature Elimination (RFE). The first predicted ED 1 year post-diagnosis and required 10 pre-treatment variables; the second predicted ED 2 years post-diagnosis with 9 pre-treatment variables. The validation AUCs were 0.84 and 0.81 for 1 year and 2 years post-diagnosis respectively. To immediately allow patients and clinicians to use these models in the clinical decision-making process, nomograms were generated. In conclusion, we successfully developed and validated two models that predicted ED in patients with localized prostate cancer. These models will allow physicians and patients alike to make informed evidence-based decisions about the most suitable treatment with quality of life in mind

DataSheet_1_A comparison of machine learning models for predicting urinary incontinence in men with localized prostate cancer.docx

IntroductionUrinary incontinence (UI) is a common side effect of prostate cancer treatment, but in clinical practice, it is difficult to predict. Machine learning (ML) models have shown promising results in predicting outcomes, yet the lack of transparency in complex models known as “black-box” has made clinicians wary of relying on them in sensitive decisions. Therefore, finding a balance between accuracy and explainability is crucial for the implementation of ML models. The aim of this study was to employ three different ML classifiers to predict the probability of experiencing UI in men with localized prostate cancer 1-year and 2-year after treatment and compare their accuracy and explainability. MethodsWe used the ProZIB dataset from the Netherlands Comprehensive Cancer Organization (Integraal Kankercentrum Nederland; IKNL) which contained clinical, demographic, and PROM data of 964 patients from 65 Dutch hospitals. Logistic Regression (LR), Random Forest (RF), and Support Vector Machine (SVM) algorithms were applied to predict (in)continence after prostate cancer treatment. ResultsAll models have been externally validated according to the TRIPOD Type 3 guidelines and their performance was assessed by accuracy, sensitivity, specificity, and AUC. While all three models demonstrated similar performance, LR showed slightly better accuracy than RF and SVM in predicting the risk of UI one year after prostate cancer treatment, achieving an accuracy of 0.75, a sensitivity of 0.82, and an AUC of 0.79. All models for the 2-year outcome performed poorly in the validation set, with an accuracy of 0.6 for LR, 0.65 for RF, and 0.54 for SVM. ConclusionThe outcomes of our study demonstrate the promise of using non-black box models, such as LR, to assist clinicians in recognizing high-risk patients and making informed treatment choices. The coefficients of the LR model show the importance of each feature in predicting results, and the generated nomogram provides an accessible illustration of how each feature impacts the predicted outcome. Additionally, the model’s simplicity and interpretability make it a more appropriate option in scenarios where comprehending the model’s predictions is essential.</p