Monitoring metrics over time: Why clinical trialists need to systematically collect site performance metrics

Background: Over the last decade, there has been an increasing interest in risk-based monitoring (RBM) in clinical trials, resulting in a number of guidelines from regulators and its inclusion in ICH GCP. However, there is a lack of detail on how to approach RBM from a practical perspective, and insufficient understanding of best practice. Purpose: We present a method for clinical trials units to track their metrics within clinical trials using descriptive statistics and visualisations. Research Design: We suggest descriptive statistics and visualisations within a SWAT methodology. Study Sample: We illustrate this method using the metrics from TEMPER, a monitoring study carried out in three trials at the MRC Clinical Trials Unit at UCL. Data Collection: The data collection for TEMPER is described in DOI: 10.1177/1740774518793379. Results: We show the results and discuss a protocol for a Study-Within-A-Trial (SWAT 167) for those wishing to use the method. Conclusions: The potential benefits metric tracking brings to clinical trials include enhanced assessment of sites for potential corrective action, improved evaluation and contextualisation of the influence of metrics and their thresholds, and the establishment of best practice in RBM. The standardisation of the collection of such monitoring data would benefit both individual trials and the clinical trials community

Lack of transparent reporting of trial monitoring approaches in randomised controlled trials: A systematic review of contemporary protocol papers

Background: Monitoring is essential to ensure patient safety and data integrity in clinical trials as per Good Clinical Practice. The Standard Protocol Items: Recommendations for Interventional Trials Statement and its checklist guides authors to include monitoring in their protocols. We investigated how well monitoring was reported in published ‘protocol papers’ for contemporary randomised controlled trials. Methods: A systematic search was conducted in PubMed to identify eligible protocol papers published in selected journals between 1 January 2020 and 31 May 2020. Protocol papers were classified by whether they reported monitoring and, if so, by the details of monitoring. Data were summarised descriptively. Results: Of 811 protocol papers for randomised controlled trials, 386 (48%; 95% CI: 44%–51%) explicitly reported some monitoring information. Of these, 20% (77/386) reported monitoring information consistent with an on-site monitoring approach, and 39% (152/386) with central monitoring, 26% (101/386) with a mixed approach, while 14% (54/386) did not provide sufficient information to specify an approach. Only 8% (30/386) of randomised controlled trials reported complete details about all of scope, frequency and organisation of monitoring; frequency of monitoring was the least reported. However, 6% (25/386) of papers used the term ‘audit’ to describe ‘monitoring’. Discussion: Monitoring information was reported in only approximately half of the protocol papers. Suboptimal reporting of monitoring hinders the clinical community from having the full information on which to judge the validity of a trial and jeopardises the value of protocol papers and the credibility of the trial itself. Greater efforts are needed to promote the transparent reporting of monitoring to journal editors and authors

P203: Lack of transparent reporting of trial monitoring approaches in randomised controlled trials: a systematic review of contemporary protocol papers

Background: Monitoring is essential to ensure patient safety and data integrity in clinical trials as per Good Clinical Practice. The SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) checklist guides authors to include monitoring in their protocols. We investigated the reporting prevalence and detail of monitoring in published “protocol papers” for contemporary randomised controlled trials (RCTs). Methods: A systematic search was conducted in PubMed to identify eligible protocol papers published in key journals. Articles were further classified by whether they reported monitoring. Descriptive data were summarised for the reporting prevalence of monitoring and the reporting extent. / Results: Of 811 protocol papers, 386 RCTs (48%; 95% CI: 44% to 51%) explicitly reported monitoring information. In particular, 20% (77/386) of RCTs reporting monitoring information described an approach consistent with on-site monitoring, 39% (152/386) with central monitoring, 26% (101/386) with a mixed approach, whilst 14% (54/386) did not provide sufficient information to specify an approach. Only 8% (30/386) of RCTs reported complete details about scope, frequency and organisation, and the monitoring frequency was the least reported. Moreover, 6% (25/386) of protocol papers interchangeably used “audit” to describe “monitoring”. / Discussion: Monitoring information was only reported in half of the published protocols. Suboptimal reporting of monitoring, as shown in this study, hinders the clinical community from having the full information on which to judge the validity of a trial externally and jeopardises the value of protocol papers and trial credibility. Greater efforts are needed to promote the transparent reporting of monitoring to journal editors and authors

Site staff perspectives on communicating trial results to participants : Cost and feasibility results from the Show RESPECT cluster randomised, factorial, mixed-methods trial

We are grateful to the patients and site staff who participated in this study. We dedicate this article to the memory of Amanda Hunn, who was part of the steering group for this study until she passed away in 2022. We thank other members of the steering group for their input into the study, especially Conor Tweed and Liz James. We acknowledge the hard work and diligence of Cara Purvis, the trial manager, and the data managers for Show RESPECT. We also thank the ICON8 trial team for their support, particularly Babasola Popoola, Francesca Schiavone, Jonathan Badrock, Andrew Clamp and Rick Kaplan.Peer reviewe

Site staff perspectives on communicating trial results to participants: Cost and feasibility results from the Show RESPECT cluster randomised, factorial, mixed-methods trial

BACKGROUND/AIMS: Sharing trial results with participants is an ethical imperative but often does not happen. Show RESPECT (ISRCTN96189403) tested ways of sharing results with participants in an ovarian cancer trial (ISRCTN10356387). Sharing results via a printed summary improved patient satisfaction. Little is known about staff experience and the costs of communicating results with participants. We report the costs of communication approaches used in Show RESPECT and the views of site staff on these approaches. METHODS: We allocated 43 hospitals (sites) to share results with trial participants through one of eight intervention combinations (2 × 2 × 2 factorial; enhanced versus basic webpage, printed summary versus no printed summary, email list invitation versus no invitation). Questionnaires elicited data from staff involved in sharing results. Open- and closed-ended questions covered resources used to share results and site staff perspectives on the approaches used. Semi-structured interviews were conducted. Interview and free-text data were analysed thematically. The mean additional site costs per participant from each intervention were estimated jointly as main effects by linear regression. RESULTS: We received questionnaires from 68 staff from 41 sites and interviewed 11 site staff. Sites allocated to the printed summary had mean total site costs of sharing results £13.71/patient higher (95% confidence interval (CI): -3.19, 30.60; p = 0.108) than sites allocated no printed summary. Sites allocated to the enhanced webpage had mean total site costs £1.91/patient higher (95% CI: -14, 18.74; p = 0.819) than sites allocated to the basic webpage. Sites allocated to the email list had costs £2.87/patient lower (95% CI: -19.70, 13.95; p = 0.731) than sites allocated to no email list. Most of these costs were staff time for mailing information and handling patients' queries. Most site staff reported no concerns about how they had shared results (88%) and no challenges (76%). Most (83%) found it easy to answer queries from patients about the results and thought the way they were allocated to share results with participants would be an acceptable standard approach (76%), with 79% saying they would follow the same approach for future trials. There were no significant effects of the randomised interventions on these outcomes. Site staff emphasised the importance of preparing patients to receive the results, including giving opt-in/opt-out options, and the need to offer further support, particularly if the results could confuse or distress some patients. CONCLUSIONS: Adding a printed summary to a webpage (which significantly improved participant satisfaction) may increase costs to sites by ~£14/patient, which is modest in relation to the cost of trials. The Show RESPECT communication interventions were feasible to implement. This information could help future trials ensure they have sufficient resources to share results with participants

Access to routinely collected health data for clinical trials - review of successful data requests to UK registries.

BACKGROUND: Clinical trials generally each collect their own data despite routinely collected health data (RCHD) increasing in quality and breadth. Our aim is to quantify UK-based randomised controlled trials (RCTs) accessing RCHD for participant data, characterise how these data are used and thereby recommend how more trials could use RCHD. METHODS: We conducted a systematic review of RCTs accessing RCHD from at least one registry in the UK between 2013 and 2018 for the purposes of informing or supplementing participant data. A list of all registries holding RCHD in the UK was compiled. In cases where registries published release registers, these were searched for RCTs accessing RCHD. Where no release register was available, registries were contacted to request a list of RCTs. For each identified RCT, information was collected from all publicly available sources (release registers, websites, protocol etc.). The search and data extraction were undertaken between January and May 2019. RESULTS: We identified 160 RCTs accessing RCHD between 2013 and 2018 from a total of 22 registries; this corresponds to only a very small proportion of all UK RCTs (about 3%). RCTs accessing RCHD were generally large (median sample size 1590), commonly evaluating treatments for cancer or cardiovascular disease. Most of the included RCTs accessed RCHD from NHS Digital (68%), and the most frequently accessed datasets were mortality (76%) and hospital visits (55%). RCHD was used to inform the primary trial (82%) and long-term follow-up (57%). There was substantial variation in how RCTs used RCHD to inform participant outcome measures. A limitation was the lack of information and transparency from registries and RCTs with respect to which datasets have been accessed and for what purposes. CONCLUSIONS: In the last five years, only a small minority of UK-based RCTs have accessed RCHD to inform participant data. We ask for improved accessibility, confirmed data quality and joined-up thinking between the registries and the regulatory authorities. TRIAL REGISTRATION: PROSPERO CRD42019123088

Testing approaches to sharing trial results with participants : The Show RESPECT cluster randomised, factorial, mixed methods trial

Funding: The Show RESPECT study was funded by the Medical Research Council through core grants to MRS at the MRC CTU at UCL for Trial Conduct Methodology (MC_UU12023/24 and MC_UU_00004/08) https://mrc.ukri.org/. The funder had no role in the study design, the collection, analysis and interpretation of data, the writing of the report and the decision to submit the article for publication. All authors had full access to the study data, including statistical reports and tables, and can take responsibility for the integrity of the data and the accuracy of the data analysis. Acknowledgments We are very thankful to the patients who participated in this study and the people who contributed to our Patient and Public Involvement activities. We are very grateful for the work of the Show RESPECT site teams in carrying out this study. A list of team members from sites can be found in S10 Table. We thank Eva Burnett, who is a patient representative on the Study Steering Group for Show RESPECT and ICON8, for her input on the design and conduct of this study and for her comments on this manuscript, and Amanda Hunn for her contributions to the steering group. We also acknowledge the hard work and diligence of Sierra Santana and Ania Spurdens, who were data managers for Show RESPECT. We also wish to thank the ICON8 trial team for their support of this project, particularly Andrew Clamp, Babasola Popoola, Francesca Schiavone, Jonathan Badrock, and Rick Kaplan. We also thank Julia Bailey for her advice and guidance on the qualitative aspect of this study.Peer reviewedPublisher PD

Changing platforms without stopping the train: experiences of data management and data management systems when adapting platform protocols by adding and closing comparisons.

BACKGROUND: There is limited research and literature on the data management challenges encountered in multi-arm, multi-stage platform and umbrella protocols. These trial designs allow both (1) seamless addition of new research comparisons and (2) early stopping of accrual to individual comparisons that do not show sufficient activity. FOCUS4 (colorectal cancer) and STAMPEDE (prostate cancer), run from the Medical Research Council Clinical Trials Unit (CTU) at UCL, are two leading UK examples of clinical trials implementing adaptive platform protocol designs. To date, STAMPEDE has added five new research comparisons, closed two research comparisons following pre-planned interim analysis (lack of benefit), adapted the control arm following results from STAMPEDE and other relevant trials, and completed recruitment to six research comparisons. FOCUS4 has closed one research comparison following pre-planned interim analysis (lack of benefit) and added one new research comparison, with a number of further comparisons in the pipeline. We share our experiences from the operational aspects of running these adaptive trials, focusing on data management. METHODS: We held discussion groups with STAMPEDE and FOCUS4 CTU data management staff to identify data management challenges specific to adaptive platform protocols. We collated data on a number of case report form (CRF) changes, database amendments and database growth since each trial began. DISCUSSION: We found similar adaptive protocol-specific challenges in both trials. Adding comparisons to and removing them from open trials provides extra layers of complexity to CRF and database development. At the start of an adaptive trial, CRFs and databases must be designed to be flexible and scalable in order to cope with the continuous changes, ensuring future data requirements are considered where possible. When adding or stopping a comparison, the challenge is to incorporate new data requirements while ensuring data collection within ongoing comparisons is unaffected. Some changes may apply to all comparisons; others may be comparison-specific or applicable only to patients recruited during a specific time period. We discuss the advantages and disadvantages of the different approaches to CRF and database design we implemented in these trials, particularly in relation to use and maintenance of generic versus comparison-specific CRFs and databases. The work required to add or remove a comparison, including the development and testing of changes, updating of documentation, and training of sites, must be undertaken alongside data management of ongoing comparisons. Adequate resource is required for these competing data management tasks, especially in trials with long follow-up. A plan is needed for regular and pre-analysis data cleaning for multiple comparisons that could recruit at different rates and periods of time. Data-cleaning activities may need to be split and prioritised, especially if analyses for different comparisons overlap in time. CONCLUSIONS: Adaptive trials offer an efficient model to run randomised controlled trials, but setting up and conducting the data management activities in these trials can be operationally challenging. Trialists and funders must plan for scalability in data collection and the resource required to cope with additional competing data management tasks

Abiraterone acetate plus prednisolone for metastatic patients starting hormone therapy: 5-year follow-up results from the STAMPEDE randomised trial (NCT00268476)

Abiraterone acetate plus prednisolone (AAP) previously demonstrated improved survival in STAMPEDE, a multiarm, multistage platform trial in men starting long-term hormone therapy for prostate cancer. This long-term analysis in metastatic patients was planned for 3 years after the first results. Standard-of-care (SOC) was androgen deprivation therapy. The comparison randomised patients 1:1 to SOC-alone with or without daily abiraterone acetate 1000 mg + prednisolone 5 mg (SOC + AAP), continued until disease progression. The primary outcome measure was overall survival. Metastatic disease risk group was classified retrospectively using baseline CT and bone scans by central radiological review and pathology reports. Analyses used Cox proportional hazards and flexible parametric models, accounting for baseline stratification factors. One thousand and three patients were contemporaneously randomised (November 2011 to January 2014): median age 67 years; 94% newly-diagnosed; metastatic disease risk group: 48% high, 44% low, 8% unassessable; median PSA 97 ng/mL. At 6.1 years median follow-up, 329 SOC-alone deaths (118 low-risk, 178 high-risk) and 244 SOC + AAP deaths (75 low-risk, 145 high-risk) were reported. Adjusted HR = 0.60 (95% CI: 0.50-0.71; P = 0.31 × 10−9) favoured SOC + AAP, with 5-years survival improved from 41% SOC-alone to 60% SOC + AAP. This was similar in low-risk (HR = 0.55; 95% CI: 0.41-0.76) and high-risk (HR = 0.54; 95% CI: 0.43-0.69) patients. Median and current maximum time on SOC + AAP was 2.4 and 8.1 years. Toxicity at 4 years postrandomisation was similar, with 16% patients in each group reporting grade 3 or higher toxicity. A sustained and substantial improvement in overall survival of all metastatic prostate cancer patients was achieved with SOC + abiraterone acetate + prednisolone, irrespective of metastatic disease risk group

Abiraterone acetate plus prednisolone for metastatic patients starting hormone therapy: 5‐year follow‐up results from the STAMPEDE randomised trial (NCT00268476)

This is an accepted manuscript of a paper published by Wiley in International Journal of Cancer on 12/04/2022, available online: https://doi.org/10.1002/ijc.34018 The accepted manuscript of the publication may differ from the final published versionAbiraterone acetate plus prednisolone (AAP) previously demonstrated improved survival in STAMPEDE, a multi-arm, multi-stage platform trial in men starting long-term hormone therapy for prostate cancer. This long-term analysis in metastatic patients was planned for 3 yrs after the first results. Standard-of-care (SOC) was androgen deprivation therapy. The comparison randomized patients 1:1 to SOC-alone with or without daily abiraterone acetate 1000 mg + prednisolone 5 mg (SOC + AAP), continued until disease progression. The primary outcome measure was overall survival. Metastatic disease risk group was classified retrospectively using baseline CT and bone scans by central radiological review and pathology reports. Analyses used Cox proportional hazards & flexible parametric models, adjusted for baseline stratification factors. 1003 patients were contemporaneously randomized (Nov-2011--Jan-2014): median age 67 yr; 94% newly-diagnosed; metastatic disease risk group: 48% high, 44% low, 8% un-assessable; median PSA 97 ng/mL. At 6.1 yr median follow-up, 329 SOC-alone deaths (118 low-risk, 178 high-risk) and 244 SOC + AAP deaths (75 low-risk, 145 high-risk) were reported. Adjusted HR = 0·60 (95%CI:0·50—0·71; P = 0.31x10−9) favoured SOC + AAP, with 5-yr survival improved from 41% SOC-alone to 60% SOC + AAP. This was similar in low-risk (HR = 0·55; 95%CI:0·41—0·76) and high-risk (HR = 0·54; 95%CI:0·43—0·69) patients. Median and current maximum time on SOC + AAP was 2.4 yr and 8.1 yr. Toxicity at 4 yr post-randomisation was similar, with 16% patients in each group reporting grade 3 or higher toxicity. A sustained and substantial improvement in overall survival of all metastatic prostate cancer patients was achieved with SOC + abiraterone acetate + prednisolone, irrespective of metastatic disease risk group.Cancer Research UK, (CRUK_A12459), Medical Research Council (MRC_MC_UU_12023/25, MC_UU_00004/01), UK Clinical Research Network, and the Swiss Group for Cancer Clinical Research (SAKK).Published onlin