Methods for Implementing and Reporting Patient-reported Outcome (PRO) Measures of Symptomatic Adverse Events in Cancer Clinical Trials

Purpose There is increasing interest to use patient-reported outcome (PRO) measures to evaluate symptomatic adverse events (AEs) in cancer treatment trials. However, there are currently no standard recommended approaches for integrating patient-reported AE measures into trials. Methods Approaches are identified from previous trials for selecting AEs for solicited patient reporting, administering patient-reported AE measures, and analyzing and reporting results. Findings Approaches for integrating patient-reported AE measures into cancer trials generally combine current standard methods for clinician-reported AEs and established best practices for using PRO measures. Specific AEs can be selected for a PRO questionnaire based on common and expected reactions in a given trial context, derived from literature review and qualitative/mixed-methods evaluations and should be the same set administered across all arms of a trial. A mechanism for collecting unsolicited patient-reported AEs will also ideally be included. Patients will preferably report at baseline and at the end of active treatment as well as on a frequent standardized schedule during active treatment, such as weekly from home, with a recall period corresponding to the frequency of reporting (eg, past 7 days). Less frequent reporting may be considered after an initial intensive monitoring period for trials of prolonged treatments and during long-term follow-up. Electronic PRO data collection is preferred. Backup data collection for missed PRO reports is advisable to boost response rates. Analysis can use a combination of approaches to AE and PRO data. If a high proportion of patients is experiencing baseline symptoms, systematic subtraction of these from on-study AEs should be considered to improve reporting of symptoms related to treatment. More granular longitudinal analyses of individual symptoms can also be useful. Implications Methods are evolving for integrating patient-reported symptomatic AEs into cancer trials. These methods are expected to further evolve as more data from trials become available

What Do “None,” “Mild,” “Moderate,” “Severe,” and “Very Severe” Mean to Patients With Cancer? Content Validity of PRO-CTCAE™ Response Scales

Accurate capture of the symptom experience is essential to gauging efficacy, safety, and tolerability of cancer treatments. The Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) was developed by the National Cancer Institute to allow direct patient self-reporting of symptomatic adverse events in cancer clinical trials. Its content validity has been established in accordance with recommended practices for novel patient-reported outcome (PRO) instruments

An Exploratory Analysis of the “Was It Worth It?” Questionnaire as a Novel Metric to Capture Patient Perceptions of Cancer Treatment

Objectives: Asking “Was it worth it?” (WIWI) potentially captures the patient perception of a treatment's benefit weighed against its harms. This exploratory analysis evaluates the WIWI questionnaire as a metric of patients’ perspectives on the worthwhileness of cancer treatment. Methods: A 3-item WIWI questionnaire was assessed at end of treatment in patients with cancer on the COMET-2 trial (NCT01522443). WIWI items were evaluated to determine their association with quality of life (QOL), treatment duration, end-of-treatment reason, patient-reported adverse events (AEs), and disease response. Results: A total of 65 patients completed the questionnaire; 40 (62%), 16 (25%), and 9 (14%) patients replied yes, uncertain, and no to “Was it worthwhile for you to receive the cancer treatment given in this study?” (item 1), respectively; 39 (60%), 12 (18%), and 14 (22%) to “If you had to do it over again, would you choose to have this cancer treatment?”; and 40 (62%), 14 (22%), and 11 (17%) to “Would you recommend this cancer treatment to others?” Patients responding yes to item 1 remained on treatment longer than those responding uncertain or no (mean 23.0 vs 11.3 weeks, P<.001). Patients responding uncertain/no to item 1 discontinued treatment because of AEs more frequently than those responding yes (36% vs 7.5%, P=.004) and demonstrated meaningful decline in QOL from baseline (−2.5 vs −0.2 mean change, P<.001). Associations between WIWI responses and most patient-reported AEs or treatment efficacy did not reach statistical significance. Conclusions: Patients who responded affirmatively on WIWI items remained on therapy longer, were less likely to stop treatment because of AEs, and demonstrated superior QOL. The WIWI may inform clinical practice, oncology research, and value frameworks

Evaluating Treatment Tolerability Using the Toxicity Index With Patient-Reported Outcomes Data

Context: Summarizing longitudinal symptomatic adverse events during clinical trials is necessary for understanding treatment tolerability. The Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) provides insight for capturing treatment tolerability within trials. Tolerability summary measures, such as the maximum score, are often used to communicate the potential negative symptoms both in the medical literature and directly to patients. Commonly, the proportions of present and severe symptomatic adverse events are used and reported between treatment arms among adverse event types. The toxicity index is also a summary measure previously applied to clinician-reported CTCAE data. Objectives: Apply the toxicity index to PRO-CTCAE data from the COMET-2 trial alongside the maximum score, then present and discuss considerations for using the toxicity index as a summary measure for communicating tolerability to patients and clinicians. Methods: Proportions of maximum PRO-CTCAE severity levels and median toxicity index were computed by arm using all trial data and adjusting for baseline symptoms. Results: Group-wise statistical differences were similar whether using severity level proportions or the toxicity index. The impact of adjusting for baseline symptoms was equivalently seen when comparing arms using severity rates or the toxicity index. Conclusion: The toxicity index is a useful method when ranking patients from those with the least to most symptomatic adverse event burden. This study showed the toxicity index can be applied to PRO-CTCAE data. Though as a tolerability summary measure, further study is needed to provide a clear clinical or patient-facing interpretation of the toxicity index

Differential effects of an electronic symptom monitoring intervention based on the age of patients with advanced cancer

Background: Symptom monitoring interventions enhance patient outcomes, including quality of life (QoL), health care utilization, and survival, but it remains unclear whether older and younger patients with cancer derive similar benefits. We explored whether age moderates the improved outcomes seen with an outpatient electronic symptom monitoring intervention. Patients and methods: We carried out a secondary analysis of data from a randomized trial of 766 patients receiving chemotherapy for metastatic solid tumors. Patients received an electronic symptom monitoring intervention integrated with oncology care or usual oncology care alone. The intervention consisted of patients reporting their symptoms, which were provided to their physicians at clinic visits, and nurses receiving alerts for severe/worsening symptoms. We used regression models to determine whether age (older or younger than 70 years) moderated the effects of the intervention on QoL (EuroQol EQ-5D), emergency room (ER) visits, hospitalizations, and survival outcomes. Results: Enrollment rates for younger (589/777 = 75.8%) and older (177/230 = 77.0%) patients did not differ. Older patients (median age = 75 years, range 70–91 years) were more likely to have an education level of high school or less (26.6% versus 20.9%, P = 0.029) and to be computer inexperienced (50.3% versus 23.4%, P < 0.001) compared with younger patients (median age = 58 years, range 26–69 years). Younger patients receiving the symptom monitoring intervention experienced lower risk of ER visits [hazard ratio (HR) = 0.74, P = 0.011] and improved survival (HR = 0.76, P = 0.011) compared with younger patients receiving usual care. However, older patients did not experience significantly lower risk of ER visits (HR = 0.90, P = 0.613) or improved survival (HR = 1.06, P = 0.753) with the intervention. We found no moderation effects based on age for QoL and risk of hospitalizations. Conclusions: Among patients with advanced cancer, age moderated the effects of an electronic symptom monitoring intervention on the risk of ER visits and survival, but not QoL. Symptom monitoring interventions may need to be tailored to the unique needs of older adults with cancer

Composite grading algorithm for the National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)

Background: The Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events is an item library designed for eliciting patient-reported adverse events in oncology. For each adverse event, up to three individual items are scored for frequency, severity, and interference with daily activities. To align the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events with other standardized tools for adverse event assessment including the Common Terminology Criteria for Adverse Events, an algorithm for mapping individual items for any given adverse event to a single composite numerical grade was developed and tested. Methods: A five-step process was used: (1) All 179 possible Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events score combinations were presented to 20 clinical investigators to subjectively map combinations to single numerical grades ranging from 0 to 3. (2) Combinations with <75% agreement were presented to investigator committees at a National Clinical Trials Network cooperative group meeting to gain majority consensus via anonymous voting. (3) The resulting algorithm was refined via graphical and tabular approaches to assure directional consistency. (4) Validity, reliability, and sensitivity were assessed in a national study dataset. (5) Accuracy for delineating adverse events between study arms was measured in two Phase III clinical trials (NCT02066181 and NCT01522443). Results: In Step 1, 12/179 score combinations had <75% initial agreement. In Step 2, majority consensus was reached for all combinations. In Step 3, five grades were adjusted to assure directional consistency. In Steps 4 and 5, composite grades performed well and comparably to individual item scores on validity, reliability, sensitivity, and between-arm delineation. Conclusion: A composite grading algorithm has been developed and yields single numerical grades for adverse events assessed via the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events, and can be useful in analyses and reporting

Missing data strategies for the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) in Alliance A091105 and COMET-2

Purpose: Missing scores complicate analysis of the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) because patients with and without missing scores may systematically differ. We focus on optimal analysis methods for incomplete PRO-CTCAE items, with application to two randomized, double-blind, placebo-controlled, phase III trials. Methods: In Alliance A091105 and COMET-2, patients completed PRO-CTCAE items before randomization and several times post-randomization (N = 64 and 107, respectively). For each trial, we conducted between-arm comparisons on the PRO-CTCAE via complete-case two-sample t-tests, mixed modeling with contrast, and multiple imputation followed by two-sample t-tests. Because interest lies in whether CTCAE grades can inform missing PRO-CTCAE scores, we performed multiple imputation with and without CTCAE grades as auxiliary variables to assess the added benefit of including them in the imputation model relative to only including PRO-CTCAE scores across all cycles. Results: PRO-CTCAE completion rates ranged from 100.0 to 71.4% and 100.0 to 77.1% across time in A091105 and COMET-2, respectively. In both trials, mixed modeling and multiple imputation provided the most similar estimates of the average treatment effects. Including CTCAE grades in the imputation model did not consistently narrow confidence intervals of the average treatment effects because correlations for the same PRO-CTCAE item between different cycles were generally stronger than correlations between each PRO-CTCAE item and its corresponding CTCAE grade at the same cycle. Conclusion: For between-arm comparisons, mixed modeling and multiple imputation are informative techniques for handling missing PRO-CTCAE scores. CTCAE grades do not provide added benefit for informing missing PRO-CTCAE scores. ClinicalTrials.gov Identifiers: NCT02066181 (Alliance A091105); NCT01522443 (COMET-2)

Patient free text reporting of symptomatic adverse events in cancer clinical research using the National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)

Objective: The study sought to describe patient-entered supplemental information on symptomatic adverse events (AEs) in cancer clinical research reported via a National Cancer Institute software system and examine the feasibility of mapping these entries to established terminologies. Materials and Methods: Patients in 3 multicenter trials electronically completed surveys during cancer treatment. Each survey included a prespecified subset of items from the National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Upon completion of the survey items, patients could add supplemental symptomatic AE information in a free text box. As patients typed into the box, structured dropdown terms could be selected from the PRO-CTCAE item library or Medical Dictionary for Regulatory Activities (MedDRA), or patients could type unstructured free text for submission. Results: Data were pooled from 1760 participants (48% women; 78% White) who completed 8892 surveys, of which 2387 (26.8%) included supplemental symptomatic AE information. Overall, 1024 (58%) patients entered supplemental information at least once, with an average of 2.3 per patient per study. This encompassed 1474 of 8892 (16.6%) dropdowns and 913 of 8892 (10.3%) unstructured free text entries. One-third of the unstructured free text entries (32%) could be mapped post hoc to a PRO-CTCAE term and 68% to a MedDRA term. Discussion: Participants frequently added supplemental information beyond study-specific survey items. Almost half selected a structured dropdown term, although many opted to submit unstructured free text entries. Most free text entries could be mapped post hoc to PRO-CTCAE or MedDRA terms, suggesting opportunities to enhance the system to perform real-time mapping for AE reporting. Conclusions: Patient reporting of symptomatic AEs using a text box functionality with mapping to existing terminologies is both feasible and informative

Software for administering the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events: Usability study

Background: The US National Cancer Institute (NCI) developed software to gather symptomatic adverse events directly from patients participating in clinical trials. The software administers surveys to patients using items from the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) through Web-based or automated telephone interfaces and facilitates the management of survey administration and the resultant data by professionals (clinicians and research associates). Objective: The purpose of this study was to iteratively evaluate and improve the usability of the PRO-CTCAE software. Methods: Heuristic evaluation of the software functionality was followed by semiscripted, think-aloud protocols in two consecutive rounds of usability testing among patients with cancer, clinicians, and research associates at 3 cancer centers. We conducted testing with patients both in clinics and at home (remotely) for both Web-based and telephone interfaces. Furthermore, we refined the software between rounds and retested. Results: Heuristic evaluation identified deviations from the best practices across 10 standardized categories, which informed initial software improvement. Subsequently, we conducted user-based testing among 169 patients and 47 professionals. Software modifications between rounds addressed identified issues, including difficulty using radio buttons, absence of survey progress indicators, and login problems (for patients) as well as scheduling of patient surveys (for professionals). The initial System Usability Scale (SUS) score for the patient Web-based interface was 86 and 82 (P=.22) before and after modifications, respectively, whereas the task completion score was 4.47, which improved to 4.58 (P=.39) after modifications. Following modifications for professional users, the SUS scores improved from 71 to 75 (P=.47), and the mean task performance improved significantly (4.40 vs 4.02; P=.001). Conclusions: Software modifications, informed by rigorous assessment, rendered a usable system, which is currently used in multiple NCI-sponsored multicenter cancer clinical trials

Feasibility of implementing the patient-reported outcomes version of the common terminology criteria for adverse events in a multicenter trial: NCCTG N1048

Purpose The US National Cancer Institute (NCI) Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) was developed to enable patient reporting of symptomatic adverse events in oncology clinical research. This study was designed to assess the feasibility and resource requirements associated with implementing PRO-CTCAE in a multicenter trial. Methods Patients with locally advanced rectal cancer enrolled in the National Cancer Institute-sponsored North Central Cancer Treatment Group (Alliance) Preoperative Radiation or Selective Preoperative Radiation and Evaluation before Chemotherapy and Total Mesorectal Excision trial were asked to self-report 30 PRO-CTCAE items weekly from home during preoperative therapy, and every 6 months after surgery, via either the Web or an automated telephone system. If participants did not self-report within 3 days, a central coordinator called them to complete the items. Compliance was defined as the proportion of participants who completed PRO-CTCAE assessments at expected time points. Results The prespecified PRO-CTCAE analysis was conducted after the 500th patient completed the 6-month follow-up (median age, 56 years; 33% female; 12% nonwhite; 43% high school education or less; 5% Spanish speaking), across 165 sites. PRO-CTCAE was reported by participants at 4,491 of 4,882 expected preoperative time points (92.0% compliance), of which 3,771 (77.2%) were selfreported by participants and 720 (14.7%) were collected via central coordinator backup. Compliance at 6-month post-treatment follow-up was 333 of 468 (71.2%), with 122 (26.1%) via backup. Site research associates spent a median of 15 minutes on PRO-CTCAE work for each patient visit. Work by a central coordinator required a 50% time commitment. Conclusion Home-based reporting of PRO-CTCAE in a multicenter trial is feasible, with high patient compliance and low site administrative requirements. PRO-CTCAE data capture is improved through centralized backup calls