Update on the transfusion in gastrointestinal bleeding (TRIGGER) trial: statistical analysis plan for a cluster-randomised feasibility trial

BACKGROUND: Previous research has suggested an association between more liberal red blood cell (RBC) transfusion and greater risk of further bleeding and mortality following acute upper gastrointestinal bleeding (AUGIB). METHODS AND DESIGN: The Transfusion in Gastrointestinal Bleeding (TRIGGER) trial is a pragmatic cluster-randomised feasibility trial which aims to evaluate the feasibility of implementing a restrictive vs. liberal RBC transfusion policy for adult patients admitted to hospital with AUGIB in the UK. This trial will help to inform the design and methodology of a phase III trial. The protocol for TRIGGER has been published in Transfusion Medicine Reviews. Recruitment began in September 2012 and was completed in March 2013. This update presents the statistical analysis plan, detailing how analysis of the TRIGGER trial will be performed. It is hoped that prospective publication of the full statistical analysis plan will increase transparency and give readers a clear overview of how TRIGGER will be analysed. TRIAL REGISTRATION: ISRCTN85757829

Pre-specification of statistical analysis approaches in published clinical trial protocols was inadequate

OBJECTIVES: Results from randomized trials can depend on the statistical analysis approach used. It is important to prespecify the analysis approach in the trial protocol to avoid selective reporting of analyses based on those which provide the most favourable results. We undertook a review of published trial protocols to assess how often the statistical analysis of the primary outcome was adequately prespecified. METHODS: We searched protocols of randomized trials indexed in PubMed in November 2016. We identified whether the following aspects of the statistical analysis approach for the primary outcome were adequately prespecified: (1) analysis population; (2) analysis model; (3) use of covariates; and (4) method of handling missing data. RESULTS: e identified 99 eligible protocols. Very few protocols adequately prespecified the analysis population (8/99, 8%), analysis model (27/99, 27%), covariates (40/99, 40%), or approach to handling missing data (10/99, 10%). Most protocols did not adequately predefine any of these four aspects of their statistical analysis approach (39%) or predefined only one aspect (36%). No protocols adequately predefined all four aspects of the analysis. CONCLUSION: The statistical analysis approach is rarely prespecified in published trial protocols. This may allow selective reporting of results based on different analyses

Access to unpublished protocols and statistical analysis plans of randomised trials

Background: Access to protocols and statistical analysis plans (SAPs) increases the transparency of randomised trial by allowing readers to identify and interpret unplanned changes to study methods, however they are often not made publicly available. We sought to determine how often study investigators would share unavailable documents upon request. Methods: We used trials from two previously identified cohorts (cohort 1: 101 trials published in high impact factor journals between January and April of 2018; cohort 2: 100 trials published in June 2018 in journals indexed in PubMed) to determine whether study investigators would share unavailable protocols/SAPs upon request. We emailed corresponding authors of trials with no publicly available protocol or SAP up to four times. Results: Overall, 96 of 201 trials (48%) across the two cohorts had no publicly available protocol or SAP (11/101 high-impact cohort, 85/100 PubMed cohort). In total, 8/96 authors (8%) shared some trial documentation (protocol only [n = 5]; protocol and SAP [n = 1]; excerpt from protocol [n = 1]; research ethics application form [n = 1]). We received protocols for 6/96 trials (6%), and a SAP for 1/96 trial (1%). Seventy-three authors (76%) did not respond, 7 authors responded (7%) but declined to share a protocol or SAP, and eight email addresses were invalid (8%). A total of 329 emails were sent (an average of 41 emails for every trial which sent documentation). After emailing authors, the total number of trials with an available protocol increased by only 3%, from 52% in to 55%. Conclusions: Most study investigators did not share their unpublished protocols or SAPs upon direct request. Alternative strategies are needed to increase transparency of randomised trials and ensure access to protocols and SAPs

Adjusting for multiple prognostic factors in the analysis of randomised trials

Background: When multiple prognostic factors are adjusted for in the analysis of a randomised trial, it is unclear (1) whether it is necessary to account for each of the strata, formed by all combinations of the prognostic factors (stratified analysis), when randomisation has been balanced within each stratum (stratified randomisation), or whether adjusting for the main effects alone will suffice, and (2) the best method of adjustment in terms of type I error rate and power, irrespective of the randomisation method. Methods: We used simulation to (1) determine if a stratified analysis is necessary after stratified randomisation, and (2) to compare different methods of adjustment in terms of power and type I error rate. We considered the following methods of analysis: adjusting for covariates in a regression model, adjusting for each stratum using either fixed or random effects, and Mantel-Haenszel or a stratified Cox model depending on outcome. Results: Stratified analysis is required after stratified randomisation to maintain correct type I error rates when (a) there are strong interactions between prognostic factors, and (b) there are approximately equal number of patients in each stratum. However, simulations based on real trial data found that type I error rates were unaffected by the method of analysis (stratified vs unstratified), indicating these conditions were not met in real datasets. Comparison of different analysis methods found that with small sample sizes and a binary or time-to-event outcome, most analysis methods lead to either inflated type I error rates or a reduction in power; the lone exception was a stratified analysis using random effects for strata, which gave nominal type I error rates and adequate power. Conclusions: It is unlikely that a stratified analysis is necessary after stratified randomisation except in extreme scenarios. Therefore, the method of analysis (accounting for the strata, or adjusting only for the covariates) will not generally need to depend on the method of randomisation used. Most methods of analysis work well with large sample sizes, however treating strata as random effects should be the analysis method of choice with binary or time-to-event outcomes and a small sample size

The risks and rewards of covariate adjustment in randomized trials: an assessment of 12 outcomes from 8 studies

Adjustment for prognostic covariates can lead to increased power in the analysis of randomized trials. However, adjusted analyses are not often performed in practice

Cluster over individual randomization: are study design choices appropriately justified? Review of a random sample of trials

Taljaard, M., Goldstein, C. E., Giraudeau, B., Nicholls, S. G., Carroll, K., Hey, S. P., … Weijer, C. (2020). Cluster over individual randomization: are study design choices appropriately justified? Review of a random sample of trials. Clinical Trials. Copyright © The Author(s), 2020. DOI: https://doi.org/10.1177/174077451989679

Reducing bias in open-label trials where blinded outcome assessment is not feasible: strategies from two randomised trials

Background Blinded outcome assessment is recommended in open-label trials to reduce bias, however it is not always feasible. It is therefore important to find other means of reducing bias in these scenarios. Methods We describe two randomised trials where blinded outcome assessment was not possible, and discuss the strategies used to reduce the possibility of bias. Results TRIGGER was an open-label cluster randomised trial whose primary outcome was further bleeding. Because of the cluster randomisation, all researchers in a hospital were aware of treatment allocation and so could not perform a blinded assessment. A blinded adjudication committee was also not feasible as it was impossible to compile relevant information to send to the committee in a blinded manner. Therefore, the definition of further bleeding was modified to exclude subjective aspects (such as whether symptoms like vomiting blood were severe enough to indicate the outcome had been met), leaving only objective aspects (the presence versus absence of active bleeding in the upper gastrointestinal tract confirmed by an internal examination). TAPPS was an open-label trial whose primary outcome was whether the patient was referred for a pleural drainage procedure. Allowing a blinded assessor to decide whether to refer the patient for a procedure was not feasible as many clinicians may be reluctant to enrol patients into the trial if they cannot be involved in their care during follow-up. Assessment by an adjudication committee was not possible, as the outcome either occurred or did not. Therefore, the decision pathway for procedure referral was modified. If a chest x-ray indicated that more than a third of the pleural space filled with fluid, the patient could be referred for a procedure; otherwise, the unblinded clinician was required to reach a consensus on referral with a blinded assessor. This process allowed the unblinded clinician to be involved in the patient’s care, while reducing the potential for bias. Conclusions When blinded outcome assessment is not possible, it may be useful to modify the outcome definition or method of assessment to reduce the risk of bias

Increased risk of type I errors in cluster randomised trials with small or medium numbers of clusters: a review, reanalysis,and simulation study

Background: Cluster randomised trials (CRTs) are commonly analysed using mixed-effects models or generalised estimating equations (GEEs). However, these analyses do not always perform well with the small number of clusters typical of most CRTs. They can lead to increased risk of a type I error (finding a statistically significant treatment effect when it does not exist) if appropriate corrections are not used. Methods: We conducted a small simulation study to evaluate the impact of using small-sample corrections for mixed-effects models or GEEs in CRTs with a small number of clusters. We then reanalysed data from TRIGGER, a CRT with six clusters, to determine the effect of using an inappropriate analysis method in practice. Finally, we reviewed 100 CRTs previously identified by a search on PubMed in order to assess whether trials were using appropriate methods of analysis. Trials were classified as at risk of an increased type I error rate if they did not report using an analysis method which accounted for clustering, or if they had fewer than 40 clusters and performed an individual-level analysis without reporting the use of an appropriate small-sample correction. Results: Our simulation study found that using mixed-effects models or GEEs without an appropriate correction led to inflated type I error rates, even for as many as 70 clusters. Conversely, using small-sample corrections provided correct type I error rates across all scenarios. Reanalysis of the TRIGGER trial found that inappropriate methods of analysis gave much smaller P values (P ≤ 0.01) than appropriate methods (P = 0.04–0.15). In our review, of the 99 trials that reported the number of clusters, 64 (65 %) were at risk of an increased type I error rate; 14 trials did not report using an analysis method which accounted for clustering, and 50 trials with fewer than 40 clusters performed an individual-level analysis without reporting the use of an appropriate correction. Conclusions: CRTs with a small or medium number of clusters are at risk of an inflated type I error rate unless appropriate analysis methods are used. Investigators should consider using small-sample corrections with mixed-effects models or GEEs to ensure valid results. Abbreviations: CRT, Cluster randomised trial; CI, Confidence interval; GEE, Generalised estimating equations; TRIGGER, Trial in Gastrointestinal Transfusio

Accounting for genomic pre-selection in national BLUP evaluations in dairy cattle

<p>Abstract</p> <p>Background</p> <p>In future Best Linear Unbiased Prediction (BLUP) evaluations of dairy cattle, genomic selection of young sires will cause evaluation biases and loss of accuracy once the selected ones get progeny.</p> <p>Methods</p> <p>To avoid such bias in the estimation of breeding values, we propose to include information on all genotyped bulls, including the culled ones, in BLUP evaluations. Estimated breeding values based on genomic information were converted into genomic pseudo-performances and then analyzed simultaneously with actual performances. Using simulations based on actual data from the French Holstein population, bias and accuracy of BLUP evaluations were computed for young sires undergoing progeny testing or genomic pre-selection. For bulls pre-selected based on their genomic profile, three different types of information can be included in the BLUP evaluations: (1) data from pre-selected genotyped candidate bulls with actual performances on their daughters, (2) data from bulls with both actual and genomic pseudo-performances, or (3) data from all the genotyped candidates with genomic pseudo-performances. The effects of different levels of heritability, genomic pre-selection intensity and accuracy of genomic evaluation were considered.</p> <p>Results</p> <p>Including information from all the genotyped candidates, i.e. genomic pseudo-performances for both selected and culled candidates, removed bias from genetic evaluation and increased accuracy. This approach was effective regardless of the magnitude of the initial bias and as long as the accuracy of the genomic evaluations was sufficiently high.</p> <p>Conclusions</p> <p>The proposed method can be easily and quickly implemented in BLUP evaluations at the national level, although some improvement is necessary to more accurately propagate genomic information from genotyped to non-genotyped animals. In addition, it is a convenient method to combine direct genomic, phenotypic and pedigree-based information in a multiple-step procedure.</p

Standardisation of intestinal ultrasound scoring in clinical trials for luminal Crohn's disease

Background: Intestinal ultrasound (IUS) is a valuable tool for assessment of Crohn’s disease (CD). However, there is no widely accepted luminal disease activity index. / Aims: To identify appropriate IUS protocols, indices, items, and scoring methods for measurement of luminal CD activity and integration of IUS in CD clinical trials. / Methods: An expert international panel of adult and paediatric gastroenterologists (n = 15) and radiologists (n = 3) rated the appropriateness of 120 statements derived from literature review and expert opinion (scale of 1-9) using modified RAND/UCLA methodology. Median panel scores of 1 to ≤3.5, >3.5 to <6.5 and ≥6.5 to 9 were considered inappropriate, uncertain and appropriate ratings respectively. The statement list and survey results were discussed prior to voting. / Results: A total of 91 statements were rated appropriate with agreement after two rounds of voting. Items considered appropriate measures of disease activity were bowel wall thickness (BWT), vascularity, stratification and mesenteric inflammatory fat. There was uncertainty if any of the existing IUS disease activity indices were appropriate for use in CD clinical trials. Appropriate trial applications for IUS included patient recruitment qualification when diseased segments cannot be adequately assessed by ileocolonoscopy and screening for exclusionary complications. At outcome assessment, remission endpoints including BWT and vascularity, with or without mesenteric inflammatory fat, were considered appropriate. Components of an ideal IUS disease activity index were identified based upon panel discussions. / Conclusions: The panel identified appropriate component items and applications of IUS for CD clinical trials. Empiric evidence, and development and validation of an IUS disease activity index are needed