Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform

From Springer Nature via Jisc Publications RouterHistory: received 2020-02-26, rev-recd 2020-03-28, accepted 2020-05-14, registration 2020-05-14, pub-electronic 2020-06-01, online 2020-06-01, pub-print 2020-11Publication status: PublishedFunder: Cancer Research UK; Grant(s): C147/A18083, C147/A25254, C19221/A22746Funder: Manchester Biomedical Research Centre; doi: http://dx.doi.org/10.13039/100014653Abstract: Objective: To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome. Methods: The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset. Results: The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios. Conclusion: IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics. Key Points: • Reliability of radiomic features varies between feature calculation platforms and with choice of software version. • Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised. • IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features

Radiomics as a personalized medicine tool in lung cancer: separating the hope from the hype

Radiomics has become a popular image analysis method in the last few years. Its key hypothesis is that medical images harbor biological, prognostic and predictive information that is not revealed upon visual inspection. In contrast to previous work with a priori defined imaging biomarkers, radiomics instead calculates image features at scale and uses statistical methods to identify those most strongly associated to outcome. This builds on years of research into computer aided diagnosis and pattern recognition. While the potential of radiomics to aid personalized medicine is widely recognized, several technical limitations exist which hinder biomarker translation. Aspects of the radiomic workflow lack repeatability or reproducibility under particular circumstances, which is a key requirement for the translation of imaging biomarkers into clinical practice. One of the most commonly studied uses of radiomics is for personalized medicine applications in Non-Small Cell Lung Cancer (NSCLC). In this review, we summarize reported methodological limitations in CT based radiomic analyses together with suggested solutions. We then evaluate the current NSCLC radiomics literature to assess the risk associated with accepting the published conclusions with respect to these limitations. We review different complementary scoring systems and initiatives that can be used to critically appraise data from radiomics studies. Wider awareness should improve the quality of ongoing and future radiomics studies and advance their potential as clinically relevant biomarkers for personalized medicine in patients with NSCLC

Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform

Objective To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome.Methods The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset.Results The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios.Conclusion IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics.Key points • Reliability of radiomic features varies between feature calculation platforms and with choice of software version. • Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised. • IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features

A study demonstrating users’ preference for the adapted-REQUITE patient-reported outcome questionnaire over PRO-CTCAE in patients with lung cancer

Introduction The use of patient-reported outcomes (PROs) has been shown to enhance the accuracy of symptom collection and improve overall survival and quality of life. This is the first study comparing concordance and patient preference for two PRO tools: Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) and the adapted-REQUITE Lung Questionnaire. Materials and Methods Patients with lung cancer were recruited to the study while attending outpatient clinics at a tertiary cancer centre. Clinician-reported outcomes were generated through initial patient assessment with CTCAE v4.03. Participants then completed the PRO-CTCAE and adapted-REQUITE questionnaires. Concordance between the 2 questionnaires was assessed by calculating Pearson correlation coefficient. PRO-CTCAE and CTCAE concordance was demonstrated by calculating Pearson correlation coefficient from the linear predictors of an ordinal logistic regression. P-values were also calculated. Results Out of 74 patients approached, 65 provided written informed consent to participate in the study. 63 (96.9%) patients completed both PRO-CTCAE and adapted-REQUITE questionnaires. Pearson correlation coefficient between PRO tools was 0.8-0.83 (p &lt; 001). Correlation between CTCAE and PRO-CTCAE ranged between 0.66-0.82 (p &lt; 001). Adapted-REQUITE and CTCAE correlation was higher for all symptoms ranging between 0.79-0.91 (p &amp;lt;.001). Acceptable discrepancies within one grade were present in 96.8%-100% of symptom domains for REQUITE and in 92.1%-96.8% for all domains in the PRO-CTCAE. 54% of the total participant cohort favored the adapted-REQUITE questionnaire due to reduced subjectivity in the questions and ease of use. Conclusion The adapted-REQUITE questionnaire has shown a superior correlation to clinician-reported outcomes and higher patient preference than the PRO-CTCAE. The results of this study suggest the use of the REQUITE questionnaire for patients with lung cancer in routine clinical practice.<br/

Impact of Introducing Intensity Modulated Radiotherapy on Curative Intent Radiotherapy and Survival for Lung Cancer

BACKGROUND: Lung cancer survival remains poor. The introduction of Intensity-Modulated Radiotherapy (IMRT) allows treatment of more complex tumours as it improves conformity around the tumour and greater normal tissue sparing. However, there is limited evidence assessing the clinical impact of IMRT. In this study, we evaluated whether the introduction of IMRT had an influence on the proportion of patients treated with curative-intent radiotherapy over time, and whether this had an effect on patient survival. MATERIALS AND METHODS: Patients treated with thoracic radiotherapy at our institute between 2005 and 2020 were retrospectively identified and grouped into three time periods: A) 2005-2008 (pre-IMRT), B) 2009-2012 (selective use of IMRT), and C) 2013-2020 (full access to IMRT). Data on performance status (PS), stage, age, gross tumour volume (GTV), planning target volume (PTV) and survival were collected. The proportion of patients treated with a curative dose between these periods was compared. Multivariable survival models were fitted to evaluate the hazard for patients treated in each time period, adjusting for PS, stage, age and tumour volume. RESULTS: 12,499 patients were included in the analysis (n=2675 (A), n=3127 (B), and n=6697 (C)). The proportion of patients treated with curative-intent radiotherapy increased between the 3 time periods, from 38.1% to 50.2% to 65.6% (p<0.001). When stage IV patients were excluded, this increased to 40.1% to 58.1% to 82.9% (p<0.001). This trend was seen across all PS and stages. The GTV size increased across the time periods and PTV size decreased. Patients treated with curative-intent during period C had a survival improvement compared to time period A when adjusting for clinical variables (HR=0.725 (0.632-0.831), p<0.001). CONCLUSION: IMRT was associated with to more patients receiving curative-intent radiotherapy. In addition, it facilitated the treatment of larger tumours that historically would have been treated palliatively. Despite treating larger, more complex tumours with curative-intent, a survival benefit was seen for patients treated when full access to IMRT was available (2013-2020). This study highlights the impact of IMRT on thoracic oncology practice, accepting that improved survival may also be attributed to a number of other contributing factors, including improvements in staging, other technological radiotherapy advances and changes to systemic treatment

DataSheet_1_A study demonstrating users’ preference for the adapted-REQUITE patient-reported outcome questionnaire over PRO-CTCAE® in patients with lung cancer.docx

IntroductionThe use of patient-reported outcomes (PROs) has been shown to enhance the accuracy of symptom collection and improve overall survival and quality of life. This is the first study comparing concordance and patient preference for two PRO tools: Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE®) and the adapted-REQUITE Lung Questionnaire.Materials and MethodsPatients with lung cancer were recruited to the study while attending outpatient clinics at a tertiary cancer centre. Clinician-reported outcomes were generated through initial patient assessment with CTCAE v4.03. Participants then completed the PRO-CTCAE® and adapted-REQUITE questionnaires. Concordance between the 2 questionnaires was assessed by calculating Pearson correlation coefficient. PRO-CTCAE® and CTCAE concordance was demonstrated by calculating Pearson correlation coefficient from the linear predictors of an ordinal logistic regression. P-values were also calculated.ResultsOut of 74 patients approached, 65 provided written informed consent to participate in the study. 63 (96.9%) patients completed both PRO-CTCAE® and adapted-REQUITE questionnaires. Pearson correlation coefficient between PRO tools was 0.8-0.83 (p ® ranged between 0.66-0.82 (p ®. 54% of the total participant cohort favored the adapted-REQUITE questionnaire due to reduced subjectivity in the questions and ease of use.ConclusionThe adapted-REQUITE questionnaire has shown a superior correlation to clinician-reported outcomes and higher patient preference than the PRO-CTCAE®. The results of this study suggest the use of the REQUITE questionnaire for patients with lung cancer in routine clinical practice.</p