A DELPHI study priority setting the remaining challenges for the use of routinely collected data in trials: COMORANT-UK

Background Researchers are increasingly seeking to use routinely collected data to support clinical trials. This approach has the potential to transform the way clinical trials are conducted in the future. The availability of routinely collected data for research, whether healthcare or administrative, has increased, and infrastructure funding has enabled much of this. However, challenges remain at all stages of a trial life cycle. This study, COMORANT-UK, aimed to systematically identify, with key stakeholders across the UK, the ongoing challenges related to trials that seek to use routinely collected data. Methods This three-step Delphi method consisted of two rounds of anonymous web-based surveys and a virtual consensus meeting. Stakeholders included trialists, data infrastructures, funders of trials, regulators, data providers and the public. Stakeholders identified research questions or challenges that they considered were of particular importance and then selected their top 10 in the second survey. The ranked questions were taken forward to the consensus meeting for discussion with representatives invited from the stakeholder groups. Results In the first survey, 66 respondents yielded over 260 questions or challenges. These were thematically grouped and merged into a list of 40 unique questions. Eighty-eight stakeholders then ranked their top ten from the 40 questions in the second survey. The most common 14 questions were brought to the virtual consensus meeting in which stakeholders agreed a top list of seven questions. We report these seven questions which are within the following domains: trial design, Patient and Public Involvement, trial set-up, trial open and trial data. These questions address both evidence gaps (requiring further methodological research) and implementation gaps (requiring training and/or service re-organisation). Conclusion This prioritised list of seven questions should inform the direction of future research in this area and should direct efforts to ensure that the benefits in major infrastructure for routinely collected data are achieved and translated. Without this and future work to address these questions, the potential societal benefits of using routinely collected data to help answer important clinical questions will not be realised

Bayesian design and analysis of external pilot trials for complex interventions.

External pilot trials of complex interventions are used to help determine if and how a confirmatory trial should be undertaken, providing estimates of parameters such as recruitment, retention, and adherence rates. The decision to progress to the confirmatory trial is typically made by comparing these estimates to pre-specified thresholds known as progression criteria, although the statistical properties of such decision rules are rarely assessed. Such assessment is complicated by several methodological challenges, including the simultaneous evaluation of multiple endpoints, complex multi-level models, small sample sizes, and uncertainty in nuisance parameters. In response to these challenges, we describe a Bayesian approach to the design and analysis of external pilot trials. We show how progression decisions can be made by minimizing the expected value of a loss function, defined over the whole parameter space to allow for preferences and trade-offs between multiple parameters to be articulated and used in the decision-making process. The assessment of preferences is kept feasible by using a piecewise constant parametrization of the loss function, the parameters of which are chosen at the design stage to lead to desirable operating characteristics. We describe a flexible, yet computationally intensive, nested Monte Carlo algorithm for estimating operating characteristics. The method is used to revisit the design of an external pilot trial of a complex intervention designed to increase the physical activity of care home residents

Results of a feasibility randomised controlled trial (RCT) for WATCH IT: a programme for obese children and adolescents

Background: In the evaluation of childhood obesity interventions, few researchers undertake a rigorous feasibility stage in which the design and procedures of the evaluation process are examined. Consequently, phase III studies often demonstrate methodological weaknesses. Purpose: Our aim was to conduct a feasibility trial of the evaluation of WATCH IT, a community obesity intervention for children and adolescents. We sought to determine an achievable recruitment rate; acceptability of randomisation, assessment procedures, and dropout rate; optimal outcome measures for the definitive trial; and a robust sample size calculation. Method: Our goal was to recruit 70 participants over 6 months, randomise them to intervention or control group, and retain participation for 12 months. Assessments were taken prior to randomisation and after 6 and 12 months. Procedures mirrored those intended for a full-scale trial, but multiple measures of similar outcomes were included as a means to determine those most appropriate for future research. Acceptability of the research and impact of the research on the programme were ascertained through interviewing participants and staff. Results: We recruited 70 participants and found that randomisation and data collection procedures were acceptable. Self-referral (via media promotion) was more effective than professional referral. Blinding of assessors was sustained to a reasonable degree, and optimal outcome measures for a full-scale trial were identified. Estimated sample size was significantly greater than sample sized reported in published trials. There was some negative impact on the existing programme as a result of the research, a lesson for designers of future trials. Limitations: We successfully recruited socially disadvantaged families, but the majority of families were of White British nationality. The composition of the participants was an added valuable lesson, suggesting that recruitment strategies to obtain a more heterogeneous ethnic sample warrant consideration in future research. Conclusions: This study provided us with confidence that we can run a phase III multi-centre trial to test the effectiveness of WATCH IT. Importantly, it was invaluable in informing the design not only of that trial but also of future evaluations of childhood obesity treatment interventions

Statistical challenges in assessing potential efficacy of complex interventions in pilot or feasibility studies

Early phase trials of complex interventions currently focus on assessing the feasibility of a large RCT and on conducting pilot work. Assessing the efficacy of the proposed intervention is generally discouraged, due to concerns of underpowered hypothesis testing. In contrast, early assessment of efficacy is common for drug therapies, where phase II trials are often used as a screening mechanism to identify promising treatments. In this paper we outline the challenges encountered in extending ideas developed in the phase II drug trial literature to the complex intervention setting. The prevalence of multiple endpoints and clustering of outcome data are identified as important considerations, having implications for timely and robust determination of optimal trial design parameters. The potential for Bayesian methods to help to identify robust trial designs and optimal decision rules is also explored