Efficient and flexible simulation-based sample size determination for clinical trials with multiple design parameters

Simulation offers a simple and flexible way to estimate the power of a clinical trial when analytic formulae are not available. The computational burden of using simulation has, however, restricted its application to only the simplest of sample size determination problems, often minimising a single parameter (the overall sample size) subject to power being above a target level. We describe a general framework for solving simulation-based sample size determination problems with several design parameters over which to optimise and several conflicting criteria to be minimised. The method is based on an established global optimisation algorithm widely used in the design and analysis of computer experiments, using a non-parametric regression model as an approximation of the true underlying power function. The method is flexible, can be used for almost any problem for which power can be estimated using simulation, and can be implemented using existing statistical software packages. We illustrate its application to a sample size determination problem involving complex clustering structures, two primary endpoints and small sample considerations

A multicentre, pragmatic, cluster randomised, controlled feasibility trial of the POD system of care

Objective to provide a preliminary estimate of the effectiveness of the prevention of delirium (POD) system of care in reducing incident delirium in acute hospital wards and gather data for a future definitive randomised controlled trial. Design cluster randomised and controlled feasibility trial. Setting sixteen acute care of older people and orthopaedic trauma wards in eight hospitals in England and Wales. Participants patients 65 years and over admitted to participating wards during the trial period. Interventions participating wards were randomly assigned to either the POD programme or usual care, determined by existing local policies and practices. The POD programme is a manualised multicomponent delirium prevention intervention that targets 10 risk factors for delirium. The intervention wards underwent a 6-month implementation period before trial recruitment commenced. Main outcome measure incidence of new-onset delirium measured using the Confusion Assessment Method (CAM) measured daily for up to 10 days post consent. Results out of 4449, 3274 patients admitted to the wards were eligible. In total, 714 patients consented (713 registered) to the trial, thirty-three participants (4.6%) withdrew. Adherence to the intervention was classified as at least medium for seven wards. Rates of new-onset delirium were lower than expected and did not differ between groups (24 (7.0%) of participants in the intervention group versus 33 (8.9%) in the control group; odds ratio (95% confidence interval) 0.68 (0.37–1.26); P = 0.2225). Conclusions based on these findings, a definitive trial is achievable and would need to recruit 5220 patients in 26 two-ward hospital clusters. Trial registration: ISRCTN01187372. Registered 13 March 2014

The feasibility of using the EQ-5D-3L with adults with mild to moderate learning disabilities within a randomized control trial: a qualitative evaluation

This is the final version. Available on open access from BMC via the DOI in this recordAvailability of data and materials: The datasets used and analysed during the current study are available from the corresponding author on reasonable request. It is not possible to share qualitative data as individuals could be identifiable.Background In trials incorporating a health economic evaluation component, reliable validated measures for health-related quality of life (HRQOL) are essential. The EQ-5D is the preferred measure for cost-effectiveness analysis in UK trials. This paper presents a qualitative evaluation of the use of the EQ-5D-3L in a feasibility randomised control trial with participants who had a mild- to moderate learning disability and type 2 diabetes. Methods Researchers administered the EQ-5D-3L to 82 participants at baseline and 77 at follow-up. After each interview, researchers rated the ease of administering the EQ-5D-3L and made free-text entries on the administration experience. For a subset of 16 interviews, researchers audio-recorded more detailed journal entries. Ease of administration data were analysed using descriptive statistics. Free-text responses were subject to a basic content analysis. The EQ-5D-3L-related journal entries were transcribed, coded and analysed thematically. Results Over half of participants were perceived to experience difficulty answering some or all of the items in the EQ-5D-3L (60% at baseline; 54% at follow-up). Analysis of the free-text entries and audio journals identified four themes that question the use of the EQ-5D-3L in this population. The first theme is related to observations of participant intellectual ability and difficulties, for example, in understanding the wording of the measure. Theme 2 is related to the normalisation of adjustments for impairments, which rendered the measure less sensitive in this population. Theme 3 is related to researcher adaptation and non-standard administration. An overarching fourth theme was identified in that people with learning disabilities were viewed as ‘unreliable witnesses’ by both researchers and supporters. Conclusions It is recommended that the EQ-5D-3L should not be used in isolation to assess health-related quality of life outcomes in trials research in adults with a learning disability. Further research is required to develop and evaluate a version of the EQ-5D appropriate for this population in trials research. It is unrealistic to expect that adjustments to the wording alone will deliver an appropriate measure: supporter or researcher involvement will almost always be required. This requirement needs to be factored into the development and administration guidelines of any new version of the EQ-5D for adults with a learning disability.This work was undertaken by those working on the OK Diabetes study and contributed to the development of the project. OK Diabetes was funded by the National Institute for Health Research Health Technology Assessment Programme [project number 10/102/03]

Introducing structured caregiver training in stroke care: findings from the TRACS process evaluation study

Objective: To evaluate the process of implementation of the modified London Stroke Carers Training Course (LSCTC) in the Training Caregivers After Stroke (TRACS) cluster randomised trial and contribute to the interpretation of the TRACS trial results. The LSCTC was a structured competency-based training programme designed to help develop the knowledge and skills (eg, patient handling or transfer skills) essential for the day-to-day management of disabled survivors of stroke. The LSCTC comprised 14 components, 6 were mandatory (and delivered to all) and 8 non-mandatory, to be delivered based on individual assessment of caregiver need. Design: Process evaluation using non-participant observation, documentary analysis and semistructured interviews. Participants: Patients with stroke (n=38), caregivers (n=38), stroke unit staff (n=53). Settings: 10 of the 36 stroke units participating in the TRACS trial in four English regions (Yorkshire, North West, South East and South West, Peninsula). Results: Preparatory cascade training on delivery of the LSCTC did not reach all staff and did not lead to multidisciplinary team (MDT) wide understanding of, engagement with or commitment to the LSCTC. Although senior therapists in most intervention units observed developed ownership of the LSCTC, MDT working led to separation rather than integration of delivery of LSCTC elements. Organisational features of stroke units and professionals’ patient-focused practices limited the involvement of caregivers. Caregivers were often invited to observe therapy or care being provided by professionals but had few opportunities to make sense of, or to develop knowledge and stroke-specific skills provided by the LSCTC. Where provided, caregiver training came very late in the inpatient stay. Assessment and development of caregiver competence was not commonly observed. Conclusions: Contextual factors including service improvement pressures and staff perceptions of the necessity for and work required in caregiver training impacted negatively on implementation of the caregiver training intervention. Structured caregiver training programmes such as the LSCTC are unlikely to be practical in settings with short inpatient stays. Stroke units where early supported discharge is in place potentially offer a more effective vehicle for introducing competency based caregiver training

Methodological bias in cluster randomised trials

Background: Cluster randomised trials can be susceptible to a range of methodological problems. These problems are not commonly recognised by many researchers. In this paper we discuss the issues that can lead to bias in cluster trials. Methods: We used a sample of cluster randomised trials from a recent review and from a systematic review of hip protectors. We compared the mean age of participants between intervention groups in a sample of 'good' cluster trials with a sample of potentially biased trials. We also compared the effect sizes, in a funnel plot, between hip protector trials that used individual randomisation compared with those that used cluster randomisation. Results: There is a tendency for cluster trials, with evidence methodological biases, to also show an age imbalance between treatment groups. In a funnel plot we show that all cluster trials show a large positive effect of hip protectors whilst individually randomised trials show a range of positive and negative effects, suggesting that cluster trials may be producing a biased estimate of effect. Conclusion: Methodological biases in the design and execution of cluster randomised trials is frequent. Some of these biases associated with the use of cluster designs can be avoided through careful attention to the design of cluster trials. Firstly, if possible, individual allocation should be used. Secondly, if cluster allocation is required, then ideally participants should be identified before random allocation of the clusters. Third, if prior identification is not possible, then an independent recruiter should be used to recruit participants

Results of a feasibility randomised controlled trial (RCT) for WATCH IT: a programme for obese children and adolescents

Background: In the evaluation of childhood obesity interventions, few researchers undertake a rigorous feasibility stage in which the design and procedures of the evaluation process are examined. Consequently, phase III studies often demonstrate methodological weaknesses. Purpose: Our aim was to conduct a feasibility trial of the evaluation of WATCH IT, a community obesity intervention for children and adolescents. We sought to determine an achievable recruitment rate; acceptability of randomisation, assessment procedures, and dropout rate; optimal outcome measures for the definitive trial; and a robust sample size calculation. Method: Our goal was to recruit 70 participants over 6 months, randomise them to intervention or control group, and retain participation for 12 months. Assessments were taken prior to randomisation and after 6 and 12 months. Procedures mirrored those intended for a full-scale trial, but multiple measures of similar outcomes were included as a means to determine those most appropriate for future research. Acceptability of the research and impact of the research on the programme were ascertained through interviewing participants and staff. Results: We recruited 70 participants and found that randomisation and data collection procedures were acceptable. Self-referral (via media promotion) was more effective than professional referral. Blinding of assessors was sustained to a reasonable degree, and optimal outcome measures for a full-scale trial were identified. Estimated sample size was significantly greater than sample sized reported in published trials. There was some negative impact on the existing programme as a result of the research, a lesson for designers of future trials. Limitations: We successfully recruited socially disadvantaged families, but the majority of families were of White British nationality. The composition of the participants was an added valuable lesson, suggesting that recruitment strategies to obtain a more heterogeneous ethnic sample warrant consideration in future research. Conclusions: This study provided us with confidence that we can run a phase III multi-centre trial to test the effectiveness of WATCH IT. Importantly, it was invaluable in informing the design not only of that trial but also of future evaluations of childhood obesity treatment interventions

How, in what contexts, and why do quality dashboards lead to improvements in care quality in acute hospitals? Protocol for a realist feasibility evaluation

Introduction: National audits are used to monitor care quality and safety and are anticipated to reduce unexplained variations in quality by stimulating quality improvement (QI). However, variation within and between providers in the extent of engagement with national audits means that the potential for national audit data to inform QI is not being realised. This study will undertake a feasibility evaluation of QualDash, a quality dashboard designed to support clinical teams and managers to explore data from two national audits, the Myocardial Ischaemia National Audit Project (MINAP) and the Paediatric Intensive Care Audit Network (PICANet). Methods and analysis: Realist evaluation, which involves building, testing and refining theories of how an intervention works, provides an overall framework for this feasibility study. Realist hypotheses that describe how, in what contexts, and why QualDash is expected to provide benefit will be tested across five hospitals. A controlled interrupted time series analysis, using key MINAP and PICANet measures, will provide preliminary evidence of the impact of QualDash, while ethnographic observations and interviews over 12 months will provide initial insight into contexts and mechanisms that lead to those impacts. Feasibility outcomes include the extent to which MINAP and PICANet data are used, data completeness in the audits, and the extent to which participants perceive QualDash to be useful and express the intention to continue using it after the study period. Ethics and dissemination: The study has been approved by the University of Leeds School of Healthcare Research Ethics Committee. Study results will provide an initial understanding of how, in what contexts, and why quality dashboards lead to improvements in care quality. These will be disseminated to academic audiences, study participants, hospital IT departments and national audits. If the results show a trial is feasible, we will disseminate the QualDash software through a stepped wedge cluster randomised trial

Developing a pressure ulcer risk factor minimum data set and risk assessment framework

AIM: To agree a draft pressure ulcer risk factor Minimum Data Set to underpin the development of a new evidenced-based Risk Assessment Framework.BACKGROUND: A recent systematic review identified the need for a pressure ulcer risk factor Minimum Data Set and development and validation of an evidenced-based pressure ulcer Risk Assessment Framework. This was undertaken through the Pressure UlceR Programme Of reSEarch (RP-PG-0407-10056), funded by the National Institute for Health Research and incorporates five phases. This article reports phase two, a consensus study.DESIGN: Consensus study.METHOD: A modified nominal group technique based on the Research and Development/University of California at Los Angeles appropriateness method. This incorporated an expert group, review of the evidence and the views of a Patient and Public Involvement service user group. Data were collected December 2010-December 2011.FINDINGS: The risk factors and assessment items of the Minimum Data Set (including immobility, pressure ulcer and skin status, perfusion, diabetes, skin moisture, sensory perception and nutrition) were agreed. In addition, a draft Risk Assessment Framework incorporating all Minimum Data Set items was developed, comprising a two stage assessment process (screening and detailed full assessment) and decision pathways.CONCLUSION: The draft Risk Assessment Framework will undergo further design and pre-testing with clinical nurses to assess and improve its usability. It will then be evaluated in clinical practice to assess its validity and reliability. The Minimum Data Set could be used in future for large scale risk factor studies informing refinement of the Risk Assessment Framework