Efficient and flexible simulation-based sample size determination for clinical trials with multiple design parameters

Simulation offers a simple and flexible way to estimate the power of a clinical trial when analytic formulae are not available. The computational burden of using simulation has, however, restricted its application to only the simplest of sample size determination problems, often minimising a single parameter (the overall sample size) subject to power being above a target level. We describe a general framework for solving simulation-based sample size determination problems with several design parameters over which to optimise and several conflicting criteria to be minimised. The method is based on an established global optimisation algorithm widely used in the design and analysis of computer experiments, using a non-parametric regression model as an approximation of the true underlying power function. The method is flexible, can be used for almost any problem for which power can be estimated using simulation, and can be implemented using existing statistical software packages. We illustrate its application to a sample size determination problem involving complex clustering structures, two primary endpoints and small sample considerations

Bayesian design and analysis of external pilot trials for complex interventions

External pilot trials of complex interventions are used to help determine if and how a confirmatory trial should be undertaken, providing estimates of parameters such as recruitment, retention, and adherence rates. The decision to progress to the confirmatory trial is typically made by comparing these estimates to pre‐specified thresholds known as progression criteria, although the statistical properties of such decision rules are rarely assessed. Such assessment is complicated by several methodological challenges, including the simultaneous evaluation of multiple endpoints, complex multi‐level models, small sample sizes, and uncertainty in nuisance parameters. In response to these challenges, we describe a Bayesian approach to the design and analysis of external pilot trials. We show how progression decisions can be made by minimizing the expected value of a loss function, defined over the whole parameter space to allow for preferences and trade‐offs between multiple parameters to be articulated and used in the decision‐making process. The assessment of preferences is kept feasible by using a piecewise constant parametrization of the loss function, the parameters of which are chosen at the design stage to lead to desirable operating characteristics. We describe a flexible, yet computationally intensive, nested Monte Carlo algorithm for estimating operating characteristics. The method is used to revisit the design of an external pilot trial of a complex intervention designed to increase the physical activity of care home residents

Measuring commissioners’ willingness-to-pay for community based childhood obesity prevention programmes using a discrete choice experiment

Background: In the UK, rates of childhood obesity remain high. Community based programmes for child obesity prevention are available to be commissioned by local authorities. However, there is a lack of evidence regarding how programmes are commissioned and which attributes of programmes are valued most by commissioners. The aim of this study was to determine the factors that decision-makers prioritise when commissioning programmes that target childhood obesity prevention. Methods: An online discrete choice experiment (DCE) was used to survey commissioners and decision makers in the UK to assess their willingness-to-pay for childhood obesity programmes. Results: A total of 64 commissioners and other decision makers completed the DCE. The impact of programmes on behavioural outcomes was prioritised, with participants willing to pay an extra £16,600/year if average daily fruit and vegetable intake increased for each child by one additional portion. Participants also prioritised programmes that had greater number of parents fully completing them, and were willing to pay an extra £4810/year for every additional parent completing a programme. The number of parents enrolling in a programme (holding the number completing fixed) and hours of staff time required did not significantly influence choices. Conclusions: Emphasis on high programme completion rates and success increasing children’s fruit and vegetable intake has potential to increase commissioning of community based obesity prevention programmes

Introducing structured caregiver training in stroke care: findings from the TRACS process evaluation study

Objective: To evaluate the process of implementation of the modified London Stroke Carers Training Course (LSCTC) in the Training Caregivers After Stroke (TRACS) cluster randomised trial and contribute to the interpretation of the TRACS trial results. The LSCTC was a structured competency-based training programme designed to help develop the knowledge and skills (eg, patient handling or transfer skills) essential for the day-to-day management of disabled survivors of stroke. The LSCTC comprised 14 components, 6 were mandatory (and delivered to all) and 8 non-mandatory, to be delivered based on individual assessment of caregiver need. Design: Process evaluation using non-participant observation, documentary analysis and semistructured interviews. Participants: Patients with stroke (n=38), caregivers (n=38), stroke unit staff (n=53). Settings: 10 of the 36 stroke units participating in the TRACS trial in four English regions (Yorkshire, North West, South East and South West, Peninsula). Results: Preparatory cascade training on delivery of the LSCTC did not reach all staff and did not lead to multidisciplinary team (MDT) wide understanding of, engagement with or commitment to the LSCTC. Although senior therapists in most intervention units observed developed ownership of the LSCTC, MDT working led to separation rather than integration of delivery of LSCTC elements. Organisational features of stroke units and professionals’ patient-focused practices limited the involvement of caregivers. Caregivers were often invited to observe therapy or care being provided by professionals but had few opportunities to make sense of, or to develop knowledge and stroke-specific skills provided by the LSCTC. Where provided, caregiver training came very late in the inpatient stay. Assessment and development of caregiver competence was not commonly observed. Conclusions: Contextual factors including service improvement pressures and staff perceptions of the necessity for and work required in caregiver training impacted negatively on implementation of the caregiver training intervention. Structured caregiver training programmes such as the LSCTC are unlikely to be practical in settings with short inpatient stays. Stroke units where early supported discharge is in place potentially offer a more effective vehicle for introducing competency based caregiver training

e-Consent in UK academic-led clinical trials: current practice, challenges and the need for more evidence

BACKGROUND: During the COVID-19 pandemic, in-person healthcare visits were reduced. Consequently, trial teams needed to consider implementing remote methods for conducting clinical trials, including e-Consent. Although some clinical trials may have implemented e-Consent prior to the pandemic, anecdotes of uptake for this method increased within academic-led trials. When the increased use of this process emerged, representatives from several large academic clinical trial groups within the UK collaborated to discuss ways in which trialists can learn from one another when implementing e-Consent. METHODS: A survey of UKCRC-registered Clinical Trials Units (CTUs) was undertaken in April–June 2021 to understand the implementation of and their views on the use of e-Consent and experiences from the perspectives of systems programmers and quality assurance staff on the use of e-Consent. CTUs not using e-Consent were asked to provide any reasons/barriers (including no suitable trials) and any plans for implementing it in the future. Two events for trialists and patient and public involvement (PPI) representatives were then held to disseminate findings, foster discussion, share experiences and aid in the identification of areas that the academic CTU community felt required more research. RESULTS: Thirty-four (64%) of 53 CTUs responded to the survey, with good geographical representation across the UK. Twenty-one (62%) of the responding CTUs had implemented e-Consent in at least one of their trials, across different types of trials, including CTIMPs (Clinical Trial of Investigational Medicinal Product), ATIMPs (Advanced Therapy Medicinal Products) and non-CTIMPs. One hundred ninety-seven participants attended the two workshops for wide-ranging discussions. CONCLUSION: e-Consent is increasingly used in academic-led trials, yet uncertainties remain amongst trialists, patients and members of the public. Uncertainties include a lack of formal, practical guidance and a lack of evidence to demonstrate optimal or appropriate methods to use. We strongly encourage trialists to continue to share their own experiences of the implementation of e-Consent

Methodological bias in cluster randomised trials

Background: Cluster randomised trials can be susceptible to a range of methodological problems. These problems are not commonly recognised by many researchers. In this paper we discuss the issues that can lead to bias in cluster trials. Methods: We used a sample of cluster randomised trials from a recent review and from a systematic review of hip protectors. We compared the mean age of participants between intervention groups in a sample of 'good' cluster trials with a sample of potentially biased trials. We also compared the effect sizes, in a funnel plot, between hip protector trials that used individual randomisation compared with those that used cluster randomisation. Results: There is a tendency for cluster trials, with evidence methodological biases, to also show an age imbalance between treatment groups. In a funnel plot we show that all cluster trials show a large positive effect of hip protectors whilst individually randomised trials show a range of positive and negative effects, suggesting that cluster trials may be producing a biased estimate of effect. Conclusion: Methodological biases in the design and execution of cluster randomised trials is frequent. Some of these biases associated with the use of cluster designs can be avoided through careful attention to the design of cluster trials. Firstly, if possible, individual allocation should be used. Secondly, if cluster allocation is required, then ideally participants should be identified before random allocation of the clusters. Third, if prior identification is not possible, then an independent recruiter should be used to recruit participants

Longer-term effectiveness of systemic family therapy compared with treatment as usual for young people after self-harm: An extended follow up of pragmatic randomised controlled trial

Background: Self-harm in adolescents is common and repetition frequent. Evidence for effective interventions to reduce self-harm is limited. Long term follow-up of existing studies is rare. Methods: Extended follow up, from 18 to at least 36-months, of the SHIFT trial: a pragmatic, multi-centre, individually-randomised, controlled trial involving young people (11–17) who had self-harmed at least twice and presented to Child & Adolescent Mental Health Services (CAMHS). SHIFT evaluated manualised family therapy (FT) versus treatment as usual (TAU) in reducing repetition of self-harm leading to hospital attendance 18 months post-randomisation. We obtained ONS mortality data, adult mental health data, and further details of hospital attendance from routine Hospital Episode Statistics (HES) data plus researcher follow-up. We assessed longer-term differences in outcome using multivariable Cox Proportional Hazards regression analysis, and assessed all-cause mortality and morbidity relating to hospital attendances for reasons other than self-harm. Study registration: ISRCTN 59793150 Outcomes: The original sample of 832 were randomised between April 2010 and December 2013. Extended follow-up continued until February 2017 for a median 55·4 months (range 0–82·5 months), providing post 18-month data for 804 (96·6%) participants, of whom 785 (94·4%) had a minimum of 36-months follow-up. There was no evidence of a between-group difference in the primary outcome during the extended follow-up period (Hazard Ratio (HR) 1·03; 95% CI: 0·83, 1·28; p-value=0·78), consistent with our findings in the original trial with 18 months follow-up (HR 1·14, 95% CI 0·87, 1·49; p-value 0·33). There was a reduced rate of self-harm in older participants aged 15–17 (HR 0·7, 95% CI 0·56, 0·88), as compared with those aged 11–14; and significantly increased rates of self-harm in participants whose index episode combined self-injury and poisoning (HR 1·8, 95% CI 1·2, 2·7). Two deaths were reported during the extended follow up period. Interpretation: For adolescents referred to CAMHS after self-harm, having self-harmed at least once before, trial FT confers no benefits over TAU in reducing subsequent hospitalisation for self-harm over 18 months or 36 months. Funding: NIHR HTA Reference: 07/33/0

Lifestyle referral assessment in an acute cardiology setting: study protocol for a randomized controlled feasibility trial.

BACKGROUND: Lifestyle and behaviour change are important factors in the prevention of cardiovascular disease and reduction of premature mortality. Public health initiatives have focused on opportunities for healthcare staff to deliver lifestyle advice routinely in primary and secondary care but there is no consistent approach to onward referrals and the rate of uptake of advice remains low. We do not know if advice is more effective in supporting behaviour change when a systematic approach is taken that includes identification of barriers to change, directing patients toward services, referral to services, and feedback on outcome. METHODS AND DESIGN: This is a single-centre, randomized, unblinded feasibility trial in an acute hospital setting which aims to assess the feasibility of a definitive trial and provide proof of concept for the systematic delivery of individualized lifestyle advice in patients managed through an acute cardiology in-patient service.Patients will be recruited before discharge and randomized to two groups. A control group will receive the usual lifestyle assessment and referral, while an intervention group will receive the usual assessment plus the new individualized lifestyle assessment and referral. The new assessment will inform assignment of each patient to one of three categories based on personal barriers to change. Patients may be referred to a formal lifestyle-change programme, through the 'Leeds Let's Change' website, or they may be guided in self-management, using goal setting, or they may be assigned to a 'deferment' category, for reassessment at follow-up. These latter patients will be given a contact card for the 'Leeds Let's Change' service. DISCUSSION: Lifestyle change is an important mechanism for improving health and wellbeing across the population but there are widely acknowledged difficulties in addressing lifestyle factors with patients and supporting behaviour change. A systematic approach to assessment would facilitate audit and provide an indicator of the quality of care. The new assessment template has been designed to be quick and easy to use in practice and could, for example, be added to a primary care consultation or form part of a nursing discharge assessment in an acute setting. TRIAL REGISTRATION: Current Controlled Trials ISRCTN41781196

A non-randomised feasibility study of an intervention to optimise medicines at transitions of care for patients with heart failure

Background Heart failure affects 26 million people globally, and the optimal management of medicines is crucial for patients, particularly when their care is transferred between hospital and the community. Optimising clinical outcomes requires well-calibrated cross-organisational processes with staff and patients responding and adapting to medicines changes. The aim of this study was to assess the feasibility of implementing a complex intervention (the Medicines at Transitions Intervention; MaTI) co-designed by patients and healthcare staff. The purpose of the intervention was to optimise medicines management across the gaps between secondary and primary care when hospitals handover care. The study objectives were to (1) assess feasibility through meeting specified progression criteria to proceed to the trial, (2) assess if the intervention was acceptable to staff and patients, and (3) determine whether amendment or refinement would be needed to enhance the MaTI. Methods The feasibility of the MaTI was tested in three healthcare areas in the North of England between July and October 2017. Feasibility was measured and assessed through four agreed progression to trial criteria: (1) patient recruitment, (2) patient receipt of a medicines toolkit, (3) transfer of discharge information to community pharmacy, and (4) offer of a community pharmacy medicines review/discussion or medicines reconciliation. From the cardiology wards at each of the three NHS Acute Trusts (sites), 10 patients (aged ≥ 18 years) were recruited and introduced to the ‘My Medicines Toolkit’ (MMT). Patients were asked to identify their usual community pharmacy or nominate a pharmacy. Discharge information was transferred to the community pharmacy; pharmacists were asked to reconcile medicines and invited patients for a medicines use review (MUR) or discussion. At 1 month following discharge, all patients were sent three questionnaire sets: quality-of-life, healthcare utilisation, and a patient experience survey. In a purposive sample, 20 patients were invited to participate in a semi-structured interview about their experiences of the MaTI. Staff from hospital and primary care settings involved in patients’ care were invited to participate in a semi-structured interview. Patient and staff interviews were analysed using Framework Analysis. Questionnaire completion rates were recorded and data were descriptively analysed. Results Thirty-one patients were recruited across three sites. Eighteen staff and 18 patients took part in interviews, and 19 patients returned questionnaire sets. All four progression to trial criteria were met. We identified barriers to patient engagement with the intervention in hospital, which were compounded by patients’ focus on returning home. Some patients described not engaging in discussions with staff about medicines and lacking motivation to do so because they were preoccupied with returning home. Some patients were unable or unwilling to attend a community pharmacy in person for a medicines review. Roles and responsibilities for delivering the MaTI were different in the three sites, and staff reported variations in time spent on MaTI activities. Staff reported some work pressures and staff absences that limited the time they could spend talking to patients about their medicines. Clinical teams reported that recording a target dose for heart failure medicines in patient-held documentation was difficult as they did not always know the ideal or tolerable dose. The majority of patients reported receiving the patient-held documentation. More than two-thirds reported being offered a MUR by their community pharmacists. Conclusions Delivery of the Medicines at Transitions Intervention (MaTI) was feasible at all three sites, and progression to trial criteria were met. Refinements were found to be necessary to overcome identified barriers and strengthen delivery of all steps of the intervention. Necessary changes to the MaTI were identified along with amendments to the implementation plan for the subsequent trial. Future implementation needs to take into account the complexity of medicines management and adaptation to local context

Cluster randomised controlled feasibility study of HENRY: a community-based intervention aimed at reducing obesity rates in preschool children

Background Community-based obesity prevention interventions are often commissioned despite the limited evidence base. HENRY (Health, Exercise, Nutrition for the Really Young) is a programme delivered to parents of preschool children across the UK. Early evidence suggests that it may be effective, but a robust evaluation has not been conducted. We initiated a systematic evaluation of HENRY by studying the feasibility of conducting a multi-centre definitive trial to evaluate its effectiveness and cost-effectiveness to prevent obesity. Objectives were to assess the feasibility of recruiting local authorities, centres and parents; test processes and time required to train and certify intervention staff; explore HENRY commissioning processes; identify potential sources (and associated impact) of contamination; and consider the feasibility of trial procedures. Methods We conducted a multi-centre, open labelled, two group, prospective, cluster randomised, controlled, feasibility study, with embedded process evaluation and pre-defined criteria for progression to definitive trial. We sought to recruit 120 parents from 12 children’s centres, across two UK local authority (government) areas. Within each local authority, we planned to randomise three centres to HENRY and three to ‘standard care’ control. Our plan was to collect data in family homes at baseline and 12 months, including parent and child height and weight, and parent-reported questionnaires on self-efficacy, feeding, eating habits, quality of life and resource use. Contamination, implementation and study acceptability were explored using parent interviews. Results We recruited two local authorities and 12 children’s centres within eight months. One hundred and seventeen parents were recruited (average 3.9 parents per programme) and follow-up data were collected from 85% of participants. Process data from 20 parents and 24 members of staff indicate that both would benefit from more detail about their involvement as participants, but that methods were acceptable. Contamination was likely, though the impact of this on behaviour was unclear. Conclusion Our findings indicate that a cluster RCT of HENRY to assess its effect on childhood obesity prevention is feasible. This study has allowed us to design a pragmatic definitive trial with minimal bias, taking account of lessons learnt from conducting evaluation research in public health settings