Risk of selection bias in randomised trials

Background: Selection bias occurs when recruiters selectively enrol patients into the trial based on what the next treatment allocation is likely to be. This can occur even if appropriate allocation concealment is used if recruiters can guess the next treatment assignment with some degree of accuracy. This typically occurs in unblinded trials when restricted randomisation is implemented to force the number of patients in each arm or within each centre to be the same. Several methods to reduce the risk of selection bias have been suggested; however, it is unclear how often these techniques are used in practice. Methods: We performed a review of published trials which were not blinded to assess whether they utilised methods for reducing the risk of selection bias. We assessed the following techniques: (a) blinding of recruiters; (b) use of simple randomisation; (c) avoidance of stratification by site when restricted randomisation is used; (d) avoidance of permuted blocks if stratification by site is used; and (e) incorporation of prognostic covariates into the randomisation procedure when restricted randomisation is used. We included parallel group, individually randomised phase III trials published in four general medical journals (BMJ, Journal of the American Medical Association, The Lancet, and New England Journal of Medicine) in 2010. Results: We identified 152 eligible trials. Most trials (98%) provided no information on whether recruiters were blind to previous treatment allocations. Only 3% of trials used simple randomisation; 63% used some form of restricted randomisation, and 35% did not state the method of randomisation. Overall, 44% of trials were stratified by site of recruitment; 27% were not, and 29% did not report this information. Most trials that did stratify by site of recruitment used permuted blocks (58%), and only 15% reported using random block sizes. Many trials that used restricted randomisation also included prognostic covariates in the randomisation procedure (56%). Conclusions: The risk of selection bias could not be ascertained for most trials due to poor reporting. Many trials which did provide details on the randomisation procedure were at risk of selection bias due to a poorly chosen randomisation methods. Techniques to reduce the risk of selection bias should be more widely implemented

Effects of anticoagulation in patients with device-detected atrial fibrillation and multiple stroke risk factors: a win ratio analysis of the NOAH-AFNET 6 trial

Aims Patients with device-detected atrial fibrillation (DDAF) have a lower stroke risk than those with ECG-diagnosed AF, requiring careful evaluation of oral anticoagulation benefits vs. its inherent bleeding risk. Methods and results An unmatched win ratio analysis was performed of the NOAH-AFNET 6 trial dataset, using components of the primary efficacy and safety outcomes of the trial. The primary analysis used this hierarchical order: (1) all-cause death, (2) stroke, (3) systemic or pulmonary embolism/myocardial infarction, and (4) major bleeding. Two additional analyses replaced all-cause death with cardiovascular death or included patient-reported outcomes. Win odds were calculated to account for undecided comparisons. Among 2534 patients 77 ± 7 years old, 947 (37%) women, median CHA2DS2-VA score 3 [interquartile range (IQR), 3–4], median follow-up 21 months (IQR, 10–38) 1 605 280 win ratio pairs were analyzed. The win ratio comparing edoxaban to no anticoagulation was 0.87 (95% CI: 0.68–1.10; P = 0.23). Most comparisons resulted in no clear winner (undecided pairs 84.9%). In the remaining comparisons, edoxaban won in 46% of the cases, placebo in 54%. Death and major bleeding were the most common events. The win odds was 0.98 (95% CI: 0.94–1.01; P = 0.23). Conclusions This hypothesis-generating win ratio analysis, integrating death, thrombotic events, and major bleeds with and without quality of life, did not find an advantage of anticoagulation with edoxaban over no anticoagulation in patients with DDAF. The most common events were death and major bleeding

Retrospective analyses versus RCTs: comparing like with like?

Ralf Baron,1 Lieven Nils Kennes,2 Christian Elling31Division of Neurological Pain Research and Therapy, Department of Neurology, University Hospital of Schleswig-Holstein, Kiel Campus, Kiel, 2Department of Economics and Business Administration, University of Applied Sciences Stralsund, Stralsund, 3Grünenthal GmbH, Medical Affairs Europe and North America, Aachen, GermanyIn their recent retrospective analysis assessing oxycodone/naloxone (OXN) vs. tapentadol (TAP) treatment for chronic low-back pain with a neuropathic component, Ueberall and Mueller-Schwefe1 compare their results to the findings of an earlier phase 3b/4 study.2 In our opinion, a proper comparison to the prospective, randomized, controlled, open-label study by Baron and colleagues is scientifically not appropriate. Although Ueberall and Mueller-Schwefe use the terms “prospective,” “randomly,” and “blinded” and refer to the PROBE design (prospective, randomized, open-label, blinded endpoint),3 their database study is retrospective, nonrandomized, and nonblinded with the treatment choice left to the discretion of the physicians. In this context, the use of the term “intention-to-treat (ITT) population” is inappropriate because ITT is unambiguously defined as including all randomized subjects and thus inseparable from true randomization (ICH E9).4View original paper by Ueberall and Mueller-Schwefe

Implementing unequal randomization in clinical trials with heterogeneous treatment costs

Equal randomization has been a popular choice in clinical trial practice. However, in trials with heterogeneous variances and/or variable treatment costs, as well as in the settings where maximization of every trial participant’s benefit is an important design consideration, optimal allocation proportions may be unequal across study treatment arms. In this paper, we investigate optimal allocation designs minimizing study cost under statistical efficiency constraints for parallel group clinical trials comparing several investigational treatments against the control. We show theoretically that equal allocation designs may be suboptimal, and unequal allocation designs can provide higher statistical power for the same budget, or result in a smaller cost for the same level of power. We also show how the optimal allocation can be implemented in practice by means of restricted randomization procedures, and how to perform statistical inference following these procedures, using invoked population-based or randomization-based approaches. Our results provide further support to some previous findings in the literature that unequal randomization designs can be cost-efficient and can be successfully implemented in practice. We conclude that the choice of the target allocation, the randomization procedure and the statistical methodology for data analysis are essential components to ensure valid, powerful, and robust clinical trial results.</p

The impact of selection bias in randomized multi-arm parallel group clinical trials

The impact of selection bias on the results of clinical trials has been analyzed extensively for trials of two treatments, yet its impact in multi-arm trials is still unknown. In this paper, we investigate selection bias in multi-arm trials by its impact on the type I error probability. We propose two models for selection bias, so-called biasing policies, that both extend the classic guessing strategy by Blackwell and Hodges. We derive the distribution of the F-test statistic under the misspecified outcome model and provide a formula for the type I error probability under selection bias. We apply the presented approach to quantify the influence of selection bias in multi-arm trials with increasing number of treatment groups using a permuted block design for different assumptions and different biasing strategies. Our results confirm previous findings that smaller block sizes lead to a higher proportion of sequences with inflated type I error probability. Astonishingly, our results also show that the proportion of sequences with inflated type I error probability remains constant when the number of treatment groups is increased. Realizing that the impact of selection bias cannot be completely eliminated, we propose a bias adjusted statistical model and show that the power of the statistical test is only slightly deflated for larger block sizes.</div