Monitoring metrics over time: Why clinical trialists need to systematically collect site performance metrics

Background: Over the last decade, there has been an increasing interest in risk-based monitoring (RBM) in clinical trials, resulting in a number of guidelines from regulators and its inclusion in ICH GCP. However, there is a lack of detail on how to approach RBM from a practical perspective, and insufficient understanding of best practice. Purpose: We present a method for clinical trials units to track their metrics within clinical trials using descriptive statistics and visualisations. Research Design: We suggest descriptive statistics and visualisations within a SWAT methodology. Study Sample: We illustrate this method using the metrics from TEMPER, a monitoring study carried out in three trials at the MRC Clinical Trials Unit at UCL. Data Collection: The data collection for TEMPER is described in DOI: 10.1177/1740774518793379. Results: We show the results and discuss a protocol for a Study-Within-A-Trial (SWAT 167) for those wishing to use the method. Conclusions: The potential benefits metric tracking brings to clinical trials include enhanced assessment of sites for potential corrective action, improved evaluation and contextualisation of the influence of metrics and their thresholds, and the establishment of best practice in RBM. The standardisation of the collection of such monitoring data would benefit both individual trials and the clinical trials community

Lack of transparent reporting of trial monitoring approaches in randomised controlled trials: A systematic review of contemporary protocol papers

Background: Monitoring is essential to ensure patient safety and data integrity in clinical trials as per Good Clinical Practice. The Standard Protocol Items: Recommendations for Interventional Trials Statement and its checklist guides authors to include monitoring in their protocols. We investigated how well monitoring was reported in published ‘protocol papers’ for contemporary randomised controlled trials. Methods: A systematic search was conducted in PubMed to identify eligible protocol papers published in selected journals between 1 January 2020 and 31 May 2020. Protocol papers were classified by whether they reported monitoring and, if so, by the details of monitoring. Data were summarised descriptively. Results: Of 811 protocol papers for randomised controlled trials, 386 (48%; 95% CI: 44%–51%) explicitly reported some monitoring information. Of these, 20% (77/386) reported monitoring information consistent with an on-site monitoring approach, and 39% (152/386) with central monitoring, 26% (101/386) with a mixed approach, while 14% (54/386) did not provide sufficient information to specify an approach. Only 8% (30/386) of randomised controlled trials reported complete details about all of scope, frequency and organisation of monitoring; frequency of monitoring was the least reported. However, 6% (25/386) of papers used the term ‘audit’ to describe ‘monitoring’. Discussion: Monitoring information was reported in only approximately half of the protocol papers. Suboptimal reporting of monitoring hinders the clinical community from having the full information on which to judge the validity of a trial and jeopardises the value of protocol papers and the credibility of the trial itself. Greater efforts are needed to promote the transparent reporting of monitoring to journal editors and authors

Site staff perspectives on communicating trial results to participants : Cost and feasibility results from the Show RESPECT cluster randomised, factorial, mixed-methods trial

We are grateful to the patients and site staff who participated in this study. We dedicate this article to the memory of Amanda Hunn, who was part of the steering group for this study until she passed away in 2022. We thank other members of the steering group for their input into the study, especially Conor Tweed and Liz James. We acknowledge the hard work and diligence of Cara Purvis, the trial manager, and the data managers for Show RESPECT. We also thank the ICON8 trial team for their support, particularly Babasola Popoola, Francesca Schiavone, Jonathan Badrock, Andrew Clamp and Rick Kaplan.Peer reviewe

Access to routinely collected health data for clinical trials - review of successful data requests to UK registries.

BACKGROUND: Clinical trials generally each collect their own data despite routinely collected health data (RCHD) increasing in quality and breadth. Our aim is to quantify UK-based randomised controlled trials (RCTs) accessing RCHD for participant data, characterise how these data are used and thereby recommend how more trials could use RCHD. METHODS: We conducted a systematic review of RCTs accessing RCHD from at least one registry in the UK between 2013 and 2018 for the purposes of informing or supplementing participant data. A list of all registries holding RCHD in the UK was compiled. In cases where registries published release registers, these were searched for RCTs accessing RCHD. Where no release register was available, registries were contacted to request a list of RCTs. For each identified RCT, information was collected from all publicly available sources (release registers, websites, protocol etc.). The search and data extraction were undertaken between January and May 2019. RESULTS: We identified 160 RCTs accessing RCHD between 2013 and 2018 from a total of 22 registries; this corresponds to only a very small proportion of all UK RCTs (about 3%). RCTs accessing RCHD were generally large (median sample size 1590), commonly evaluating treatments for cancer or cardiovascular disease. Most of the included RCTs accessed RCHD from NHS Digital (68%), and the most frequently accessed datasets were mortality (76%) and hospital visits (55%). RCHD was used to inform the primary trial (82%) and long-term follow-up (57%). There was substantial variation in how RCTs used RCHD to inform participant outcome measures. A limitation was the lack of information and transparency from registries and RCTs with respect to which datasets have been accessed and for what purposes. CONCLUSIONS: In the last five years, only a small minority of UK-based RCTs have accessed RCHD to inform participant data. We ask for improved accessibility, confirmed data quality and joined-up thinking between the registries and the regulatory authorities. TRIAL REGISTRATION: PROSPERO CRD42019123088

Site staff perspectives on communicating trial results to participants: Cost and feasibility results from the Show RESPECT cluster randomised, factorial, mixed-methods trial

BACKGROUND/AIMS: Sharing trial results with participants is an ethical imperative but often does not happen. Show RESPECT (ISRCTN96189403) tested ways of sharing results with participants in an ovarian cancer trial (ISRCTN10356387). Sharing results via a printed summary improved patient satisfaction. Little is known about staff experience and the costs of communicating results with participants. We report the costs of communication approaches used in Show RESPECT and the views of site staff on these approaches. METHODS: We allocated 43 hospitals (sites) to share results with trial participants through one of eight intervention combinations (2 × 2 × 2 factorial; enhanced versus basic webpage, printed summary versus no printed summary, email list invitation versus no invitation). Questionnaires elicited data from staff involved in sharing results. Open- and closed-ended questions covered resources used to share results and site staff perspectives on the approaches used. Semi-structured interviews were conducted. Interview and free-text data were analysed thematically. The mean additional site costs per participant from each intervention were estimated jointly as main effects by linear regression. RESULTS: We received questionnaires from 68 staff from 41 sites and interviewed 11 site staff. Sites allocated to the printed summary had mean total site costs of sharing results £13.71/patient higher (95% confidence interval (CI): -3.19, 30.60; p = 0.108) than sites allocated no printed summary. Sites allocated to the enhanced webpage had mean total site costs £1.91/patient higher (95% CI: -14, 18.74; p = 0.819) than sites allocated to the basic webpage. Sites allocated to the email list had costs £2.87/patient lower (95% CI: -19.70, 13.95; p = 0.731) than sites allocated to no email list. Most of these costs were staff time for mailing information and handling patients' queries. Most site staff reported no concerns about how they had shared results (88%) and no challenges (76%). Most (83%) found it easy to answer queries from patients about the results and thought the way they were allocated to share results with participants would be an acceptable standard approach (76%), with 79% saying they would follow the same approach for future trials. There were no significant effects of the randomised interventions on these outcomes. Site staff emphasised the importance of preparing patients to receive the results, including giving opt-in/opt-out options, and the need to offer further support, particularly if the results could confuse or distress some patients. CONCLUSIONS: Adding a printed summary to a webpage (which significantly improved participant satisfaction) may increase costs to sites by ~£14/patient, which is modest in relation to the cost of trials. The Show RESPECT communication interventions were feasible to implement. This information could help future trials ensure they have sufficient resources to share results with participants

Testing approaches to sharing trial results with participants : The Show RESPECT cluster randomised, factorial, mixed methods trial

Funding: The Show RESPECT study was funded by the Medical Research Council through core grants to MRS at the MRC CTU at UCL for Trial Conduct Methodology (MC_UU12023/24 and MC_UU_00004/08) https://mrc.ukri.org/. The funder had no role in the study design, the collection, analysis and interpretation of data, the writing of the report and the decision to submit the article for publication. All authors had full access to the study data, including statistical reports and tables, and can take responsibility for the integrity of the data and the accuracy of the data analysis. Acknowledgments We are very thankful to the patients who participated in this study and the people who contributed to our Patient and Public Involvement activities. We are very grateful for the work of the Show RESPECT site teams in carrying out this study. A list of team members from sites can be found in S10 Table. We thank Eva Burnett, who is a patient representative on the Study Steering Group for Show RESPECT and ICON8, for her input on the design and conduct of this study and for her comments on this manuscript, and Amanda Hunn for her contributions to the steering group. We also acknowledge the hard work and diligence of Sierra Santana and Ania Spurdens, who were data managers for Show RESPECT. We also wish to thank the ICON8 trial team for their support of this project, particularly Andrew Clamp, Babasola Popoola, Francesca Schiavone, Jonathan Badrock, and Rick Kaplan. We also thank Julia Bailey for her advice and guidance on the qualitative aspect of this study.Peer reviewedPublisher PD

Changing platforms without stopping the train: experiences of data management and data management systems when adapting platform protocols by adding and closing comparisons.

BACKGROUND: There is limited research and literature on the data management challenges encountered in multi-arm, multi-stage platform and umbrella protocols. These trial designs allow both (1) seamless addition of new research comparisons and (2) early stopping of accrual to individual comparisons that do not show sufficient activity. FOCUS4 (colorectal cancer) and STAMPEDE (prostate cancer), run from the Medical Research Council Clinical Trials Unit (CTU) at UCL, are two leading UK examples of clinical trials implementing adaptive platform protocol designs. To date, STAMPEDE has added five new research comparisons, closed two research comparisons following pre-planned interim analysis (lack of benefit), adapted the control arm following results from STAMPEDE and other relevant trials, and completed recruitment to six research comparisons. FOCUS4 has closed one research comparison following pre-planned interim analysis (lack of benefit) and added one new research comparison, with a number of further comparisons in the pipeline. We share our experiences from the operational aspects of running these adaptive trials, focusing on data management. METHODS: We held discussion groups with STAMPEDE and FOCUS4 CTU data management staff to identify data management challenges specific to adaptive platform protocols. We collated data on a number of case report form (CRF) changes, database amendments and database growth since each trial began. DISCUSSION: We found similar adaptive protocol-specific challenges in both trials. Adding comparisons to and removing them from open trials provides extra layers of complexity to CRF and database development. At the start of an adaptive trial, CRFs and databases must be designed to be flexible and scalable in order to cope with the continuous changes, ensuring future data requirements are considered where possible. When adding or stopping a comparison, the challenge is to incorporate new data requirements while ensuring data collection within ongoing comparisons is unaffected. Some changes may apply to all comparisons; others may be comparison-specific or applicable only to patients recruited during a specific time period. We discuss the advantages and disadvantages of the different approaches to CRF and database design we implemented in these trials, particularly in relation to use and maintenance of generic versus comparison-specific CRFs and databases. The work required to add or remove a comparison, including the development and testing of changes, updating of documentation, and training of sites, must be undertaken alongside data management of ongoing comparisons. Adequate resource is required for these competing data management tasks, especially in trials with long follow-up. A plan is needed for regular and pre-analysis data cleaning for multiple comparisons that could recruit at different rates and periods of time. Data-cleaning activities may need to be split and prioritised, especially if analyses for different comparisons overlap in time. CONCLUSIONS: Adaptive trials offer an efficient model to run randomised controlled trials, but setting up and conducting the data management activities in these trials can be operationally challenging. Trialists and funders must plan for scalability in data collection and the resource required to cope with additional competing data management tasks

Abiraterone acetate plus prednisolone for metastatic patients starting hormone therapy: 5-year follow-up results from the STAMPEDE randomised trial (NCT00268476)

Abiraterone acetate plus prednisolone (AAP) previously demonstrated improved survival in STAMPEDE, a multiarm, multistage platform trial in men starting long-term hormone therapy for prostate cancer. This long-term analysis in metastatic patients was planned for 3 years after the first results. Standard-of-care (SOC) was androgen deprivation therapy. The comparison randomised patients 1:1 to SOC-alone with or without daily abiraterone acetate 1000 mg + prednisolone 5 mg (SOC + AAP), continued until disease progression. The primary outcome measure was overall survival. Metastatic disease risk group was classified retrospectively using baseline CT and bone scans by central radiological review and pathology reports. Analyses used Cox proportional hazards and flexible parametric models, accounting for baseline stratification factors. One thousand and three patients were contemporaneously randomised (November 2011 to January 2014): median age 67 years; 94% newly-diagnosed; metastatic disease risk group: 48% high, 44% low, 8% unassessable; median PSA 97 ng/mL. At 6.1 years median follow-up, 329 SOC-alone deaths (118 low-risk, 178 high-risk) and 244 SOC + AAP deaths (75 low-risk, 145 high-risk) were reported. Adjusted HR = 0.60 (95% CI: 0.50-0.71; P = 0.31 × 10−9) favoured SOC + AAP, with 5-years survival improved from 41% SOC-alone to 60% SOC + AAP. This was similar in low-risk (HR = 0.55; 95% CI: 0.41-0.76) and high-risk (HR = 0.54; 95% CI: 0.43-0.69) patients. Median and current maximum time on SOC + AAP was 2.4 and 8.1 years. Toxicity at 4 years postrandomisation was similar, with 16% patients in each group reporting grade 3 or higher toxicity. A sustained and substantial improvement in overall survival of all metastatic prostate cancer patients was achieved with SOC + abiraterone acetate + prednisolone, irrespective of metastatic disease risk group

Abiraterone acetate plus prednisolone for metastatic patients starting hormone therapy: 5‐year follow‐up results from the STAMPEDE randomised trial (NCT00268476)

This is an accepted manuscript of a paper published by Wiley in International Journal of Cancer on 12/04/2022, available online: https://doi.org/10.1002/ijc.34018 The accepted manuscript of the publication may differ from the final published versionAbiraterone acetate plus prednisolone (AAP) previously demonstrated improved survival in STAMPEDE, a multi-arm, multi-stage platform trial in men starting long-term hormone therapy for prostate cancer. This long-term analysis in metastatic patients was planned for 3 yrs after the first results. Standard-of-care (SOC) was androgen deprivation therapy. The comparison randomized patients 1:1 to SOC-alone with or without daily abiraterone acetate 1000 mg + prednisolone 5 mg (SOC + AAP), continued until disease progression. The primary outcome measure was overall survival. Metastatic disease risk group was classified retrospectively using baseline CT and bone scans by central radiological review and pathology reports. Analyses used Cox proportional hazards & flexible parametric models, adjusted for baseline stratification factors. 1003 patients were contemporaneously randomized (Nov-2011--Jan-2014): median age 67 yr; 94% newly-diagnosed; metastatic disease risk group: 48% high, 44% low, 8% un-assessable; median PSA 97 ng/mL. At 6.1 yr median follow-up, 329 SOC-alone deaths (118 low-risk, 178 high-risk) and 244 SOC + AAP deaths (75 low-risk, 145 high-risk) were reported. Adjusted HR = 0·60 (95%CI:0·50—0·71; P = 0.31x10−9) favoured SOC + AAP, with 5-yr survival improved from 41% SOC-alone to 60% SOC + AAP. This was similar in low-risk (HR = 0·55; 95%CI:0·41—0·76) and high-risk (HR = 0·54; 95%CI:0·43—0·69) patients. Median and current maximum time on SOC + AAP was 2.4 yr and 8.1 yr. Toxicity at 4 yr post-randomisation was similar, with 16% patients in each group reporting grade 3 or higher toxicity. A sustained and substantial improvement in overall survival of all metastatic prostate cancer patients was achieved with SOC + abiraterone acetate + prednisolone, irrespective of metastatic disease risk group.Cancer Research UK, (CRUK_A12459), Medical Research Council (MRC_MC_UU_12023/25, MC_UU_00004/01), UK Clinical Research Network, and the Swiss Group for Cancer Clinical Research (SAKK).Published onlin

Abiraterone acetate plus prednisolone with or without enzalutamide for patients with metastatic prostate cancer starting androgen deprivation therapy: final results from two randomised phase 3 trials of the STAMPEDE platform protocol

Background: Abiraterone acetate plus prednisolone (herein referred to as abiraterone) or enzalutamide added at the start of androgen deprivation therapy improves outcomes for patients with metastatic prostate cancer. Here, we aimed to evaluate long-term outcomes and test whether combining enzalutamide with abiraterone and androgen deprivation therapy improves survival. Methods: We analysed two open-label, randomised, controlled, phase 3 trials of the STAMPEDE platform protocol, with no overlapping controls, conducted at 117 sites in the UK and Switzerland. Eligible patients (no age restriction) had metastatic, histologically-confirmed prostate adenocarcinoma; a WHO performance status of 0–2; and adequate haematological, renal, and liver function. Patients were randomly assigned (1:1) using a computerised algorithm and a minimisation technique to either standard of care (androgen deprivation therapy; docetaxel 75 mg/m2 intravenously for six cycles with prednisolone 10 mg orally once per day allowed from Dec 17, 2015) or standard of care plus abiraterone acetate 1000 mg and prednisolone 5 mg (in the abiraterone trial) orally or abiraterone acetate and prednisolone plus enzalutamide 160 mg orally once a day (in the abiraterone and enzalutamide trial). Patients were stratified by centre, age, WHO performance status, type of androgen deprivation therapy, use of aspirin or non-steroidal anti-inflammatory drugs, pelvic nodal status, planned radiotherapy, and planned docetaxel use. The primary outcome was overall survival assessed in the intention-to-treat population. Safety was assessed in all patients who started treatment. A fixed-effects meta-analysis of individual patient data was used to compare differences in survival between the two trials. STAMPEDE is registered with ClinicalTrials.gov (NCT00268476) and ISRCTN (ISRCTN78818544). Findings: Between Nov 15, 2011, and Jan 17, 2014, 1003 patients were randomly assigned to standard of care (n=502) or standard of care plus abiraterone (n=501) in the abiraterone trial. Between July 29, 2014, and March 31, 2016, 916 patients were randomly assigned to standard of care (n=454) or standard of care plus abiraterone and enzalutamide (n=462) in the abiraterone and enzalutamide trial. Median follow-up was 96 months (IQR 86–107) in the abiraterone trial and 72 months (61–74) in the abiraterone and enzalutamide trial. In the abiraterone trial, median overall survival was 76·6 months (95% CI 67·8–86·9) in the abiraterone group versus 45·7 months (41·6–52·0) in the standard of care group (hazard ratio [HR] 0·62 [95% CI 0·53–0·73]; p<0·0001). In the abiraterone and enzalutamide trial, median overall survival was 73·1 months (61·9–81·3) in the abiraterone and enzalutamide group versus 51·8 months (45·3–59·0) in the standard of care group (HR 0·65 [0·55–0·77]; p<0·0001). We found no difference in the treatment effect between these two trials (interaction HR 1·05 [0·83–1·32]; pinteraction=0·71) or between-trial heterogeneity (I2 p=0·70). In the first 5 years of treatment, grade 3–5 toxic effects were higher when abiraterone was added to standard of care (271 [54%] of 498 vs 192 [38%] of 502 with standard of care) and the highest toxic effects were seen when abiraterone and enzalutamide were added to standard of care (302 [68%] of 445 vs 204 [45%] of 454 with standard of care). Cardiac causes were the most common cause of death due to adverse events (five [1%] with standard of care plus abiraterone and enzalutamide [two attributed to treatment] and one (<1%) with standard of care in the abiraterone trial). Interpretation: Enzalutamide and abiraterone should not be combined for patients with prostate cancer starting long-term androgen deprivation therapy. Clinically important improvements in survival from addition of abiraterone to androgen deprivation therapy are maintained for longer than 7 years. Funding: Cancer Research UK, UK Medical Research Council, Swiss Group for Clinical Cancer Research, Janssen, and Astellas