Appropriate design and reporting of superiority, equivalence and non-inferiority clinical trials incorporating a benefit risk assessment: the BRAINS study including expert workshop

Background Randomised controlled trials are designed to assess the superiority, equivalence or non-inferiority of a new health technology, but which trial design should be used is not always obvious in practice. In particular, when using equivalence or non-inferiority designs, multiple outcomes of interest may be important for the success of a trial, despite the fact that usually only a single primary outcome is used to design the trial. Benefit–risk methods are used in the regulatory clinical trial setting to assess multiple outcomes and consider the trade-off of the benefits against the risks, but are not regularly implemented in publicly funded trials. Objectives The aim of the project is to aid the design of clinical trials with multiple outcomes of interest by defining when each trial design is appropriate to use and identifying when to use benefit–risk methods to assess outcome trade-offs (qualitatively or quantitatively) in a publicly funded trial setting. Methods A range of methods was used to elicit expert opinion to answer the project objectives, including a web-based survey of relevant researchers, a rapid review of current literature and a 2-day consensus workshop of experts (in 2019). Results We created a list of 19 factors to aid researchers in selecting the most appropriate trial design, containing the following overarching sections: population, intervention, comparator, outcomes, feasibility and perspectives. Six key reasons that indicate a benefit–risk method should be considered within a trial were identified: (1) when the success of the trial depends on more than one outcome; (2) when important outcomes within the trial are in competing directions (i.e. a health technology is better for one outcome, but worse for another); (3) to allow patient preferences to be included and directly influence trial results; (4) to provide transparency on subjective recommendations from a trial; (5) to provide consistency in the approach to presenting results from a trial; and (6) to synthesise multiple outcomes into a single metric. Further information was provided to support the use of benefit–risk methods in appropriate circumstances, including the following: methods identified from the review were collated into different groupings and described to aid the selection of a method; potential implementation of methods throughout the trial process were provided and discussed (with examples); and general considerations were described for those using benefit–risk methods. Finally, a checklist of five pieces of information that should be present when reporting benefit–risk methods was defined, with two additional items specifically for reporting the results. Conclusions These recommendations will assist research teams in selecting which trial design to use and deciding whether or not a benefit–risk method could be included to ensure research questions are answered appropriately. Additional information is provided to support consistent use and clear reporting of benefit–risk methods in the future. The recommendations can also be used by funding committees to confirm that appropriate considerations of the trial design have been made. Limitations This research was limited in scope and should be considered in conjunction with other trial design methodologies to assess appropriateness. In addition, further research is needed to provide concrete information about which benefit–risk methods are best to use in publicly funded trials, along with recommendations that are specific to each method

Involving patients and the public In sTatistIcal Analysis pLans (INITIAL): A delphi survey

BACKGROUND: Patient and public involvement (PPI) in trials aims to enhance research by improving its relevance and transparency. Planning for statistical analysis begins at the design stage of a trial within the protocol and is refined and detailed in a Statistical Analysis Plan (SAP). While PPI is common in design and protocol development it is less common within SAPs. This study aimed to reach consensus on the most important and relevant statistical analysis items within an SAP to involve patients and the public. METHODS: We developed a UK-based, two-round Delphi survey through an iterative consultation with public partners, statisticians, and trialists. The consultation process started with 55 items from international guidance for statistical analysis plans. We aimed to recruit at least 20 participants per key stakeholder group for inclusion in the final analysis of the Delphi survey. Participants were asked to vote on each item using a Likert scale from 1 to 9, where a rating of 1 to 3 was labelled as having ‘limited importance’; 4 to 6 as ‘important but not critical’ and 7 to 9 as ‘critical’ to involve patients and the public. Results from the second round determined consensus on critical items for PPI. RESULTS: The consultation exercise led to the inclusion of 15 statistical items in the Delphi survey. We recruited 179 participants, of whom 72% (129: 36 statisticians, 29 patients or public partners, 25 clinical researchers or methodologists, 27 trial managers, and 12 PPI coordinators) completed both rounds. Participants were on average 48 years old, 60% were female, 84% were White, 64% were based in England and 84% had at least five years’ experience in trials. Four items reached consensus regarding critical importance for patient and public involvement: presentation of results to trial participants; summary and presentation of harms; interpretation and presentation of findings in an academic setting; factors impacting how well a treatment works. No consensus was reached for the remaining 11 items. In general, the results were consistent across stakeholder groups. DISCUSSION: We identified four critical items to involve patients and the public in statistical analysis plans. The remaining 11 items did not reach consensus and need to be considered in a case-by-case basis with most responders considering patient and public involvement important (but not critical). Our research provides a platform to enable focused future efforts to improve patient and public involvement in trials and enhance the relevance of statistical analyses to patients and the public

Changing practice in cystic fibrosis: Implementing objective medication adherence data at every consultation, a learning health system and quality improvement collaborative

Background: Medication adherence data are an important quality indicator in cystic fibrosis (CF) care, yet real‐time objective data are not routinely available. An online application (CFHealthHub) has been designed to deliver these data to people with CF and their clinical team. Adoption of this innovation is the focus of an National Health Service England‐funded learning health system and Quality Improvement Collaborative (QIC). This study applies the capability, opportunity, and motivation model of behavior change to assess whether the QIC had supported healthcare professionals' uptake of accessing patient adherence data. Method: This was a mixed‐method study, treating each multidisciplinary team as an individual case. Click analytic data from CFHealthHub were collected between January 1, 2018, and September 22, 2019. Thirteen healthcare practitioners participated in semi‐structured interviews, before and after establishing the QIC. Qualitative data were analyzed using the behavior change model. Results: The cases showed varied improvement trajectories. While two cases reported reduced barriers, one faced persistent challenges. Participation in the QIC led to enhanced confidence in the platform's utility. Reduced capability, opportunity, and motivation barriers correlated with increased uptake, demonstrating value in integrating behavior change theory into QICs. Conclusion: QICs can successfully reduce barriers and enable uptake of e‐health innovations such as adherence monitoring technology. However, ongoing multi‐level strategies are needed to embed changes. Further research should explore sustainability mechanisms and their impact on patient outcomes.</p

Appropriate design and reporting of superiority, equivalence and non-inferiority clinical trials incorporating a benefit risk assessment: the BRAINS study including expert workshop

Background Randomised controlled trials are designed to assess the superiority, equivalence or non-inferiority of a new health technology, but which trial design should be used is not always obvious in practice. In particular, when using equivalence or non-inferiority designs, multiple outcomes of interest may be important for the success of a trial, despite the fact that usually only a single primary outcome is used to design the trial. Benefit–risk methods are used in the regulatory clinical trial setting to assess multiple outcomes and consider the trade-off of the benefits against the risks, but are not regularly implemented in publicly funded trials. Objectives The aim of the project is to aid the design of clinical trials with multiple outcomes of interest by defining when each trial design is appropriate to use and identifying when to use benefit–risk methods to assess outcome trade-offs (qualitatively or quantitatively) in a publicly funded trial setting. Methods A range of methods was used to elicit expert opinion to answer the project objectives, including a web-based survey of relevant researchers, a rapid review of current literature and a 2-day consensus workshop of experts (in 2019). Results We created a list of 19 factors to aid researchers in selecting the most appropriate trial design, containing the following overarching sections: population, intervention, comparator, outcomes, feasibility and perspectives. Six key reasons that indicate a benefit–risk method should be considered within a trial were identified: (1) when the success of the trial depends on more than one outcome; (2) when important outcomes within the trial are in competing directions (i.e. a health technology is better for one outcome, but worse for another); (3) to allow patient preferences to be included and directly influence trial results; (4) to provide transparency on subjective recommendations from a trial; (5) to provide consistency in the approach to presenting results from a trial; and (6) to synthesise multiple outcomes into a single metric. Further information was provided to support the use of benefit–risk methods in appropriate circumstances, including the following: methods identified from the review were collated into different groupings and described to aid the selection of a method; potential implementation of methods throughout the trial process were provided and discussed (with examples); and general considerations were described for those using benefit–risk methods. Finally, a checklist of five pieces of information that should be present when reporting benefit–risk methods was defined, with two additional items specifically for reporting the results. Conclusions These recommendations will assist research teams in selecting which trial design to use and deciding whether or not a benefit–risk method could be included to ensure research questions are answered appropriately. Additional information is provided to support consistent use and clear reporting of benefit–risk methods in the future. The recommendations can also be used by funding committees to confirm that appropriate considerations of the trial design have been made. Limitations This research was limited in scope and should be considered in conjunction with other trial design methodologies to assess appropriateness. In addition, further research is needed to provide concrete information about which benefit–risk methods are best to use in publicly funded trials, along with recommendations that are specific to each method. Study registration The rapid review is registered as PROSPERO CRD42019144882. Funding Funded by the Medical Research Council UK and the National Institute for Health and Care Research as part of the Medical Research Council–National Institute for Health and Care Research Methodology Research programme

Remote ischaemic conditioning for fatigue after stroke (RICFAST): a pilot randomised controlled trial.

Post stroke fatigue (PSF) affects 50 % of stroke survivors and can be disabling. Remote ischaemic conditioning (RIC) can preserve mitochondrial function, improve tissue perfusion and may mitigate PSF. This pilot randomised controlled trial evaluates the safety and feasibility of using RIC for PSF and evaluated measures of cellular bioenergetics. 24 people with debilitating PSF (7 item Fatigue Severity Score, FSS-7 > 4) were randomised (1:1) in this single-centre phase 2 study to RIC (blood pressure cuff inflation around the upper arm 200 mmHg for 5 min followed by 5 min of deflation), or sham (inflation pressure 20 mmHg), repeated 4 cycles, 3 times per week for 6 weeks. Primary outcomes were safety, acceptability and compliance. Secondary outcomes included FSS-7, 6 min walking test (6MWT), peak oxygen consumption (V̇O2peak), ventilatory anaerobic threshold (VAT), and muscle adenosine triphosphate (ATP) content measured using 31-phosphorous magnetic resonance spectroscopy of tibialis anterior. RIC was safe (no serious adverse events, adverse events mild) and adherence excellent (91 % sessions completed). Exploratory analysis revealed lower FSS-7 scores in the RIC group compared to sham at 6 weeks (between group difference FSS-7 -0.7, 95 %CI -2.0 to 0.6), 3 months (-1.0, 95 %CI -2.2 to 0.2) and 6 months (-0.9, 95 %CI -2.0 to 0.2). There were trends towards increased VAT, increased muscle ATP content and improved 6MWT in the RIC group. RIC is safe and acceptable for people with PSF and may result in clinically meaningful improvements in fatigue and muscle bioenergetics that require further investigation in larger studies

Remote ischaemic conditioning for fatigue after stroke (RICFAST): A pilot randomised controlled trial

Background Post stroke fatigue (PSF) affects 50 % of stroke survivors, and can be disabling. Remote ischaemic conditioning (RIC), can preserve mitochondrial function, improve tissue perfusion and may mitigate PSF. This pilot randomised controlled trial evaluates the safety and feasibility of using RIC for PSF and evaluated measures of cellular bioenergetics. Methods 24 people with debilitating PSF (7 item Fatigue Severity Score, FSS-7 > 4) were randomised (1:1) in this single-centre phase 2 study to RIC (blood pressure cuff inflation around the upper arm 200 mmHg for 5 min followed by 5 min of deflation), or sham (inflation pressure 20 mmHg), repeated 4 cycles, 3 times per week for 6 weeks. Primary outcomes were safety, acceptability, and compliance. Secondary outcomes included FSS-7, 6 min walking test (6MWT), peak oxygen consumption (V̇O2peak), ventilatory anaerobic threshold (VAT), and muscle adenosine triphosphate (ATP) content measured using 31-phosphorous magnetic resonance spectroscopy of tibialis anterior. Results RIC was safe (no serious adverse events, adverse events mild) and adherence excellent (91 % sessions completed). Exploratory analysis revealed lower FSS-7 scores in the RIC group compared to sham at 6 weeks (between group difference FSS-7 -0.7, 95 %CI -2.0 to 0.6), 3 months (-1.0, 95 %CI -2.2 to 0.2) and 6 months (-0.9, 95 %CI -2.0 to 0.2). There were trends towards increased VAT, increased muscle ATP content and improved 6MWT in the RIC group. Discussion RIC is safe and acceptable for people with PSF and may result in clinically meaningful improvements in fatigue and muscle bioenergetics that require further investigation in larger studies

Remote ischaemic conditioning for fatigue after stroke (RICFAST): A pilot randomised controlled trial

Background: Post stroke fatigue (PSF) affects 50 % of stroke survivors, and can be disabling. Remote ischaemic conditioning (RIC), can preserve mitochondrial function, improve tissue perfusion and may mitigate PSF. This pilot randomised controlled trial evaluates the safety and feasibility of using RIC for PSF and evaluated measures of cellular bioenergetics. Methods: 24 people with debilitating PSF (7 item Fatigue Severity Score, FSS-7 > 4) were randomised (1:1) in this single-centre phase 2 study to RIC (blood pressure cuff inflation around the upper arm 200 mmHg for 5 min followed by 5 min of deflation), or sham (inflation pressure 20 mmHg), repeated 4 cycles, 3 times per week for 6 weeks. Primary outcomes were safety, acceptability, and compliance. Secondary outcomes included FSS-7, 6 min walking test (6MWT), peak oxygen consumption (V˙O 2 peak), ventilatory anaerobic threshold (VAT), and muscle adenosine triphosphate (ATP) content measured using 31-phosphorous magnetic resonance spectroscopy of tibialis anterior. Results: RIC was safe (no serious adverse events, adverse events mild) and adherence excellent (91 % sessions completed). Exploratory analysis revealed lower FSS-7 scores in the RIC group compared to sham at 6 weeks (between group difference FSS-7-0.7, 95 %CI-2.0 to 0.6), 3 months (-1.0, 95 %CI-2.2 to 0.2) and 6 months (-0.9, 95 %CI-2.0 to 0.2). There were trends towards increased VAT, increased muscle ATP content and improved 6MWT in the RIC group. Discussion: RIC is safe and acceptable for people with PSF and may result in clinically meaningful improvements in fatigue and muscle bioenergetics that require further investigation in larger studies. Introduction Stroke is a leading cause of adult death and disability affecting over 12 million new people each year worldwide, 1 imparting global economic costs of over US$700 billion. 2 Increasing stroke incidence and effective treatments improving mortality result in larger numbers of people living with longer term complications after stroke. Post-stroke fatigue (PSF) is a multi-dimensional motor-perceptive, emotional and cognitive experience characterised by exhaustion persisting even after rest. 3 It affects over 50 % of stroke survivors at some point in their recovery , 4 impairs concentration and engagement in rehabilitation, is associated with greater risk of death and dependency, 5 and poore

MARGIN

Methods for Adjusting the Irrelevance Margin in Non-Inferiority Trial

A review of sample sizes for UK pilot and feasibility studies on the ISRCTN registry from 2013 to 2020

Abstract Background Pilot and feasibility studies provide information to be used when planning a full trial. A sufficient sample size within the pilot/feasibility study is required so this information can be extracted with suitable precision. This work builds upon previous reviews of pilot and feasibility studies to evaluate whether the target sample size aligns with recent recommendations and whether these targets are being reached. Methods A review of the ISRCTN registry was completed using the keywords “pilot” and “feasibility”. The inclusion criteria were UK-based randomised interventional trials that started between 2013 (end of the previous review) and 2020. Target sample size, actual sample size and key design characteristics were extracted. Descriptive statistics were used to present sample sizes overall and by key characteristics. Results In total, 761 studies were included in the review of which 448 (59%) were labelled feasibility studies, 244 (32%) pilot studies and 69 (9%) described as both pilot and feasibility studies. Over all included pilot and feasibility studies (n = 761), the median target sample size was 30 (IQR 20–50). This was consistent when split by those labelled as a pilot or feasibility study. Slightly larger sample sizes (median = 33, IQR 20–50) were shown for those labelled both pilot and feasibility (n = 69). Studies with a continuous outcome (n = 592) had a median target sample size of 30 (IQR 20–43) whereas, in line with recommendations, this was larger for those with binary outcomes (median = 50, IQR 25–81, n = 97). There was no descriptive difference in the target sample size based on funder type. In studies where the achieved sample size was available (n = 301), 173 (57%) did not reach their sample size target; however, the median difference between the target and actual sample sizes was small at just minus four participants (IQR −25–0). Conclusions Target sample sizes for pilot and feasibility studies have remained constant since the last review in 2013. Most studies in the review satisfy the earlier and more lenient recommendations however do not satisfy the most recent largest recommendation. Additionally, most studies did not reach their target sample size meaning the information collected may not be sufficient to estimate the required parameters for future definitive randomised controlled trials