Performance Validity and Outcome of Cognitive Behavior Therapy in Patients with Chronic Fatigue Syndrome

Objective: There is limited research examining the impact of the validity of cognitive test performance on treatment outcome. All known studies to date have operationalized performance validity dichotomously, leading to the loss of predictive information. Using the range of scores on a performance validity test (PVT), we hypothesized that lower performance at baseline was related to a worse treatment outcome following cognitive behavioral therapy (CBT) in patients with Chronic Fatigue Syndrome (CFS) and to lower adherence to treatment. Method: Archival data of 1081 outpatients treated with CBT for CFS were used in this study. At baseline, all patients were assessed with a PVT, the Amsterdam Short-Term Memory test (ASTM). Questionnaires assessing fatigue, physical disabilities, psychological distress, and level of functional impairment were administered before and after CBT. Results: Our main hypothesis was not confirmed: the total ASTM score was not significantly associated with outcomes at follow-up. However, patients with a missing follow-up assessment had a lower ASTM performance at baseline, reported higher levels of physical limitations, and completed fewer therapy sessions. Conclusions: CFS patients who scored low on the ASTM during baseline assessment are more likely to complete fewer therapy sessions and not to complete follow-up assessment, indicative of limited adherence to treatment. However, if these patients were retained in the intervention, their response to CBT for CFS was comparable with subjects who score high on the ASTM. This finding calls for more research to better understand the impact of performance validity on engagement with treatment and outcomes

Performance Validity Test Failure in the Clinical Population: A Systematic Review and Meta-Analysis of Prevalence Rates

Performance validity tests (PVTs) are used to measure the validity of the obtained neuropsychological test data. However, when an individual fails a PVT, the likelihood that failure truly reflects invalid performance (i.e., the positive predictive value) depends on the base rate in the context in which the assessment takes place. Therefore, accurate base rate information is needed to guide interpretation of PVT performance. This systematic review and meta-analysis examined the base rate of PVT failure in the clinical population (PROSPERO number: CRD42020164128). PubMed/MEDLINE, Web of Science, and PsychINFO were searched to identify articles published up to November 5, 2021. Main eligibility criteria were a clinical evaluation context and utilization of stand-alone and well-validated PVTs. Of the 457 articles scrutinized for eligibility, 47 were selected for systematic review and meta-analyses. Pooled base rate of PVT failure for all included studies was 16%, 95% CI [14, 19]. High heterogeneity existed among these studies (Cochran's Q = 697.97, p <.001; I 2 = 91%; τ 2 = 0.08). Subgroup analysis indicated that pooled PVT failure rates varied across clinical context, presence of external incentives, clinical diagnosis, and utilized PVT. Our findings can be used for calculating clinically applied statistics (i.e., positive and negative predictive values, and likelihood ratios) to increase the diagnostic accuracy of performance validity determination in clinical evaluation. Future research is necessary with more detailed recruitment procedures and sample descriptions to further improve the accuracy of the base rate of PVT failure in clinical practice

Performance Validity and Outcome of Cognitive Behavior Therapy in Patients with Chronic Fatigue Syndrome

OBJECTIVE: There is limited research examining the impact of the validity of cognitive test performance on treatment outcome. All known studies to date have operationalized performance validity dichotomously, leading to the loss of predictive information. Using the range of scores on a performance validity test (PVT), we hypothesized that lower performance at baseline was related to a worse treatment outcome following cognitive behavioral therapy (CBT) in patients with Chronic Fatigue Syndrome (CFS) and to lower adherence to treatment. METHOD: Archival data of 1081 outpatients treated with CBT for CFS were used in this study. At baseline, all patients were assessed with a PVT, the Amsterdam Short-Term Memory test (ASTM). Questionnaires assessing fatigue, physical disabilities, psychological distress, and level of functional impairment were administered before and after CBT. RESULTS: Our main hypothesis was not confirmed: the total ASTM score was not significantly associated with outcomes at follow-up. However, patients with a missing follow-up assessment had a lower ASTM performance at baseline, reported higher levels of physical limitations, and completed fewer therapy sessions. CONCLUSIONS: CFS patients who scored low on the ASTM during baseline assessment are more likely to complete fewer therapy sessions and not to complete follow-up assessment, indicative of limited adherence to treatment. However, if these patients were retained in the intervention, their response to CBT for CFS was comparable with subjects who score high on the ASTM. This finding calls for more research to better understand the impact of performance validity on engagement with treatment and outcomes

Machine learning versus logistic regression for the prediction of complications after pancreatoduodenectomy

BACKGROUND: Machine learning is increasingly advocated to develop prediction models for postoperative complications. It is, however, unclear if machine learning is superior to logistic regression when using structured clinical data. Postoperative pancreatic fistula and delayed gastric emptying are the two most common complications with the biggest impact on patient condition and length of hospital stay after pancreatoduodenectomy. This study aimed to compare the performance of machine learning and logistic regression in predicting pancreatic fistula and delayed gastric emptying after pancreatoduodenectomy. METHODS: This retrospective observational study used nationwide data from 16 centers in the Dutch Pancreatic Cancer Audit between January 2014 and January 2021. The area under the curve of a machine learning and logistic regression model for clinically relevant postoperative pancreatic fistula and delayed gastric emptying were compared. RESULTS: Overall, 799 (16.3%) patients developed a postoperative pancreatic fistula, and 943 developed (19.2%) delayed gastric emptying. For postoperative pancreatic fistula, the area under the curve of the machine learning model was 0.74, and the area under the curve of the logistic regression model was 0.73. For delayed gastric emptying, the area under the curve of the machine learning model and logistic regression was 0.59. CONCLUSION: Machine learning did not outperform logistic regression modeling in predicting postoperative complications after pancreatoduodenectomy

Deep learning models for automatic tumor segmentation and total tumor volume assessment in patients with colorectal liver metastases

BACKGROUND: We developed models for tumor segmentation to automate the assessment of total tumor volume (TTV) in patients with colorectal liver metastases (CRLM).METHODS: In this prospective cohort study, pre- and post-systemic treatment computed tomography (CT) scans of 259 patients with initially unresectable CRLM of the CAIRO5 trial (NCT02162563) were included. In total, 595 CT scans comprising 8,959 CRLM were divided into training (73%), validation (6.5%), and test sets (21%). Deep learning models were trained with ground truth segmentations of the liver and CRLM. TTV was calculated based on the CRLM segmentations. An external validation cohort was included, comprising 72 preoperative CT scans of patients with 112 resectable CRLM. Image segmentation evaluation metrics and intraclass correlation coefficient (ICC) were calculated.RESULTS: In the test set (122 CT scans), the autosegmentation models showed a global Dice similarity coefficient (DSC) of 0.96 (liver) and 0.86 (CRLM). The corresponding median per-case DSC was 0.96 (interquartile range [IQR] 0.95-0.96) and 0.80 (IQR 0.67-0.87). For tumor segmentation, the intersection-over-union, precision, and recall were 0.75, 0.89, and 0.84, respectively. An excellent agreement was observed between the reference and automatically computed TTV for the test set (ICC 0.98) and external validation cohort (ICC 0.98). In the external validation, the global DSC was 0.82 and the median per-case DSC was 0.60 (IQR 0.29-0.76) for tumor segmentation.CONCLUSIONS: Deep learning autosegmentation models were able to segment the liver and CRLM automatically and accurately in patients with initially unresectable CRLM, enabling automatic TTV assessment in such patients.RELEVANCE STATEMENT: Automatic segmentation enables the assessment of total tumor volume in patients with colorectal liver metastases, with a high potential of decreasing radiologist's workload and increasing accuracy and consistency.KEY POINTS: • Tumor response evaluation is time-consuming, manually performed, and ignores total tumor volume. • Automatic models can accurately segment tumors in patients with colorectal liver metastases. • Total tumor volume can be accurately calculated based on automatic segmentations.</p

Deep learning models for automatic tumor segmentation and total tumor volume assessment in patients with colorectal liver metastases

BACKGROUND: We developed models for tumor segmentation to automate the assessment of total tumor volume (TTV) in patients with colorectal liver metastases (CRLM).METHODS: In this prospective cohort study, pre- and post-systemic treatment computed tomography (CT) scans of 259 patients with initially unresectable CRLM of the CAIRO5 trial (NCT02162563) were included. In total, 595 CT scans comprising 8,959 CRLM were divided into training (73%), validation (6.5%), and test sets (21%). Deep learning models were trained with ground truth segmentations of the liver and CRLM. TTV was calculated based on the CRLM segmentations. An external validation cohort was included, comprising 72 preoperative CT scans of patients with 112 resectable CRLM. Image segmentation evaluation metrics and intraclass correlation coefficient (ICC) were calculated.RESULTS: In the test set (122 CT scans), the autosegmentation models showed a global Dice similarity coefficient (DSC) of 0.96 (liver) and 0.86 (CRLM). The corresponding median per-case DSC was 0.96 (interquartile range [IQR] 0.95-0.96) and 0.80 (IQR 0.67-0.87). For tumor segmentation, the intersection-over-union, precision, and recall were 0.75, 0.89, and 0.84, respectively. An excellent agreement was observed between the reference and automatically computed TTV for the test set (ICC 0.98) and external validation cohort (ICC 0.98). In the external validation, the global DSC was 0.82 and the median per-case DSC was 0.60 (IQR 0.29-0.76) for tumor segmentation.CONCLUSIONS: Deep learning autosegmentation models were able to segment the liver and CRLM automatically and accurately in patients with initially unresectable CRLM, enabling automatic TTV assessment in such patients.RELEVANCE STATEMENT: Automatic segmentation enables the assessment of total tumor volume in patients with colorectal liver metastases, with a high potential of decreasing radiologist's workload and increasing accuracy and consistency.KEY POINTS: • Tumor response evaluation is time-consuming, manually performed, and ignores total tumor volume. • Automatic models can accurately segment tumors in patients with colorectal liver metastases. • Total tumor volume can be accurately calculated based on automatic segmentations.</p

The Prognostic Value of Total Tumor Volume Response Compared With RECIST1.1 in Patients With Initially Unresectable Colorectal Liver Metastases Undergoing Systemic Treatment

OBJECTIVES: Compare total tumor volume (TTV) response after systemic treatment to Response Evaluation Criteria in Solid Tumors (RECIST1.1) and assess the prognostic value of TTV change and RECIST1.1 for recurrence-free survival (RFS) in patients with colorectal liver-only metastases (CRLM). BACKGROUND: RECIST1.1 provides unidimensional criteria to evaluate tumor response to systemic therapy. Those criteria are accepted worldwide but are limited by interobserver variability and ignore potentially valuable information about TTV. METHODS: Patients with initially unresectable CRLM receiving systemic treatment from the randomized, controlled CAIRO5 trial (NCT02162563) were included. TTV response was assessed using software specifically developed together with SAS analytics. Baseline and follow-up computed tomography (CT) scans were used to calculate RECIST1.1 and TTV response to systemic therapy. Different thresholds (10%, 20%, 40%) were used to define response of TTV as no standard currently exists. RFS was assessed in a subgroup of patients with secondarily resectable CRLM after induction treatment. RESULTS: A total of 420 CT scans comprising 7820 CRLM in 210 patients were evaluated. In 30% to 50% (depending on chosen TTV threshold) of patients, discordance was observed between RECIST1.1 and TTV change. A TTV decrease of &gt;40% was observed in 47 (22%) patients who had stable disease according to RECIST1.1. In 118 patients with secondarily resectable CRLM, RFS was shorter for patients with less than 10% TTV decrease compared with patients with more than 10% TTV decrease ( = 0.015), while RECIST1.1 was not prognostic ( = 0.821). CONCLUSIONS: TTV response assessment shows prognostic potential in the evaluation of systemic therapy response in patients with CRLM

Deep learning models for automatic tumor segmentation and total tumor volume assessment in patients with colorectal liver metastases

Abstract Background We developed models for tumor segmentation to automate the assessment of total tumor volume (TTV) in patients with colorectal liver metastases (CRLM). Methods In this prospective cohort study, pre- and post-systemic treatment computed tomography (CT) scans of 259 patients with initially unresectable CRLM of the CAIRO5 trial (NCT02162563) were included. In total, 595 CT scans comprising 8,959 CRLM were divided into training (73%), validation (6.5%), and test sets (21%). Deep learning models were trained with ground truth segmentations of the liver and CRLM. TTV was calculated based on the CRLM segmentations. An external validation cohort was included, comprising 72 preoperative CT scans of patients with 112 resectable CRLM. Image segmentation evaluation metrics and intraclass correlation coefficient (ICC) were calculated. Results In the test set (122 CT scans), the autosegmentation models showed a global Dice similarity coefficient (DSC) of 0.96 (liver) and 0.86 (CRLM). The corresponding median per-case DSC was 0.96 (interquartile range [IQR] 0.95–0.96) and 0.80 (IQR 0.67–0.87). For tumor segmentation, the intersection-over-union, precision, and recall were 0.75, 0.89, and 0.84, respectively. An excellent agreement was observed between the reference and automatically computed TTV for the test set (ICC 0.98) and external validation cohort (ICC 0.98). In the external validation, the global DSC was 0.82 and the median per-case DSC was 0.60 (IQR 0.29–0.76) for tumor segmentation. Conclusions Deep learning autosegmentation models were able to segment the liver and CRLM automatically and accurately in patients with initially unresectable CRLM, enabling automatic TTV assessment in such patients. Relevance statement Automatic segmentation enables the assessment of total tumor volume in patients with colorectal liver metastases, with a high potential of decreasing radiologist’s workload and increasing accuracy and consistency. Key points • Tumor response evaluation is time-consuming, manually performed, and ignores total tumor volume. • Automatic models can accurately segment tumors in patients with colorectal liver metastases. • Total tumor volume can be accurately calculated based on automatic segmentations. Graphical Abstrac