Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform

From Springer Nature via Jisc Publications RouterHistory: received 2020-02-26, rev-recd 2020-03-28, accepted 2020-05-14, registration 2020-05-14, pub-electronic 2020-06-01, online 2020-06-01, pub-print 2020-11Publication status: PublishedFunder: Cancer Research UK; Grant(s): C147/A18083, C147/A25254, C19221/A22746Funder: Manchester Biomedical Research Centre; doi: http://dx.doi.org/10.13039/100014653Abstract: Objective: To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome. Methods: The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset. Results: The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios. Conclusion: IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics. Key Points: • Reliability of radiomic features varies between feature calculation platforms and with choice of software version. • Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised. • IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features

On the reporting and analysis of a cancer evolutionary adaptive dosing trial

There is growing interest in combining expertise from different academic fields when it comes to treating cancer. The combination of mathematics and cancer has led to the birth of mathematical oncology. In this short article the reporting of results of the first clinical mathematical oncology trial exploring a novel dosing algorithm based on evolutionary principles is discussed. In particular the lack of details on patient characteristics, acknowledgement of known confounders and improper comparisons to historical controls are discussed. The study by Zhang et al.1 reports on the initial results of the first cancer clinical trial employing an evolutionary based adaptive dosing algorithm within the clinic. The study was conducted using the drug Abiraterone within the metastatic castration resistant prostate cancer (mCRPC) setting. The initial results were based on 11 patients who had their treatment stopped and started based on a simple rule relating to Prostate Specific Antigen (PSA) falls and rises as follows. Patients had to have experienced a PSA fall to be eligible for the study, once a >50% fall was seen, patients had their treatment stopped and re-started once their PSA levels reached pre-treatment levels. The final conclusion from their initial analysis was that “The outcomes show significant improvement over published studies and a contemporaneous population.” This conclusion was based on comparing their results against a 16 patient contemporaneous cohort and a clinical trial cohort of 546 patients. This short article will highlight key issues with these comparisons which can be used to dispute the key conclusion from the paper. The authors compare the radiological progression times of their 11 patients on an adaptive dosing schedule with 16 patients who have received continuous therapy. However, the authors do not provide any information on key prognostic factors of Abiraterone for either their 11 patients or the 16 patients they compare to. In fact, there is no information on the patient characteristics of the 16 contemporaneous cohort. There are numerous prognostic factors for Abiraterone, lactate dehydrogenase, alkaline phosphatase, hemoglobin to name but a few, which were released in the statistical review held at the FDA. Therefore, without information on these prognostic factors the authors cannot exclude that the difference in progression times was not attributable to differences in prognostic factors between the two cohorts. The second comparison made by the authors is with the results of a Phase III clinical trial.2 In the authors study patients were only eligible for the study if they experienced a PSA fall >50%. However, this was not the case in the trial they compare their results too where patients recruited had not yet had any doses of Abiraterone. Therefore, the comparison made by the authors is incorrect as the two cohorts of patients are not necessarily the same. In summary, the points raised within this short article highlight that the evidence for adaptive therapy being superior to continuous therapy still does not exist. By highlighting the flaws in reporting and comparisons it is hoped the community raises its standards for future trials of this type, especially when making comparisons with historical data. References 1. Zhang, J., Cunningham, J. J., Brown, J. S. & Gatenby, R. A. Integrating evolutionary dynamics into treatment of metastatic castrate-resistant prostate cancer. Nature Communications 8, 1–9 (2017). 2. Ryan, C. J. et al. Abiraterone acetate plus prednisone versus placebo plus prednisone in chemotherapy-naive men with metastatic castration-resistant prostate cancer (COU-AA-302): final overall survival analysis of a randomised, double-blind, placebo-controlled phase 3 study. Lancet Oncol. 16, 152–160 (2015)