Use of external evidence for design and Bayesian analysis of clinical trials:a qualitative study of trialists’ views

Abstract Background Evidence from previous studies is often used relatively informally in the design of clinical trials: for example, a systematic review to indicate whether a gap in the current evidence base justifies a new trial. External evidence can be used more formally in both trial design and analysis, by explicitly incorporating a synthesis of it in a Bayesian framework. However, it is unclear how common this is in practice or the extent to which it is considered controversial. In this qualitative study, we explored attitudes towards, and experiences of, trialists in incorporating synthesised external evidence through the Bayesian design or analysis of a trial. Methods Semi-structured interviews were conducted with 16 trialists: 13 statisticians and three clinicians. Participants were recruited across several universities and trials units in the United Kingdom using snowball and purposeful sampling. Data were analysed using thematic analysis and techniques of constant comparison. Results Trialists used existing evidence in many ways in trial design, for example, to justify a gap in the evidence base and inform parameters in sample size calculations. However, no one in our sample reported using such evidence in a Bayesian framework. Participants tended to equate Bayesian analysis with the incorporation of prior information on the intervention effect and were less aware of the potential to incorporate data on other parameters. When introduced to the concepts, many trialists felt they could be making more use of existing data to inform the design and analysis of a trial in particular scenarios. For example, some felt existing data could be used more formally to inform background adverse event rates, rather than relying on clinical opinion as to whether there are potential safety concerns. However, several barriers to implementing these methods in practice were identified, including concerns about the relevance of external data, acceptability of Bayesian methods, lack of confidence in Bayesian methods and software, and practical issues, such as difficulties accessing relevant data. Conclusions Despite trialists recognising that more formal use of external evidence could be advantageous over current approaches in some areas and useful as sensitivity analyses, there are still barriers to such use in practice

Empirical evidence of study design biases in randomized trials:Systematic review of meta-epidemiological studies

To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials.Systematic review of meta-epidemiological studies.We retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic.We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear.Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed

Scalloped channels enhance tear mixing under hydrogel contact lenses.

PurposeTear exchange under a soft contact lens is directly related to the amount of lateral and transverse lens motion. Hydrodynamic modeling suggests that channels placed on the back surface of a soft lens will reduce fluid resistance and increase transverse lens movement. This study measured the effect of posterior lens surface scalloped channels on tear exchange.MethodsTear exchange in the postlens tear film (PoLTF) was estimated using a fluorometer to measure the exponential depletion of high-MW fluorescein under the lens expressed as the time to deplete 95% of dye (T95). A total of 32 subjects wore two pairs of identical lenses except that the experimental lens had 12 scalloped channels placed radially in the midperiphery of the posterior lens surface, whereas lenses without channels served as controls.ResultsThe mean +/- standard error T95 values for the channel lenses was 28 +/- 2 minutes compared with 32 +/- 2 minutes for the control lenses (p = 0.107). There was a marginally significant difference in T95 between two lens groups in Asian eyes (p = 0.054).ConclusionPlacing scallop-shaped channels on high-H2O content soft lenses improved the postlens tear pumping in Asian eyes

How Often Do Safety Signals Occur by Chance in First-in-Human Trials?

Clinicians working on first-in-human clinical studies need to be able to judge whether safety signals observed on an investigational drug were more likely to have occurred by chance or to have been caused by the drug. We retrospectively reviewed 84 Novartis studies including 1234 healthy volunteers receiving placebo, to determine the expected incidence of changes in commonly measured laboratory parameters and vital signs, in the absence of any active agent. We calculated the frequency of random incidence of safety signals, focusing on the liver, cardiovascular system, kidney and pancreas. Using the liver enzyme alanine aminotransferase (ALT) as an example, we illustrate how a predictive model can be used to determine the probability of a given subject to experience an elevation of ALT above the upper limit of the normal range under placebo, conditional on the characteristics of this subject and the study

Exploring the impact of selection bias in observational studies of COVID-19: a simulation study

BACKGROUND: Non-random selection of analytic subsamples could introduce selection bias in observational studies. We explored the potential presence and impact of selection in studies of SARS-CoV-2 infection and COVID-19 prognosis. METHODS: We tested the association of a broad range of characteristics with selection into COVID-19 analytic subsamples in the Avon Longitudinal Study of Parents and Children (ALSPAC) and UK Biobank (UKB). We then conducted empirical analyses and simulations to explore the potential presence, direction and magnitude of bias due to this selection (relative to our defined UK-based adult target populations) when estimating the association of body mass index (BMI) with SARS-CoV-2 infection and death-with-COVID-19. RESULTS: In both cohorts, a broad range of characteristics was related to selection, sometimes in opposite directions (e.g. more-educated people were more likely to have data on SARS-CoV-2 infection in ALSPAC, but less likely in UKB). Higher BMI was associated with higher odds of SARS-CoV-2 infection and death-with-COVID-19. We found non-negligible bias in many simulated scenarios. CONCLUSIONS: Analyses using COVID-19 self-reported or national registry data may be biased due to selection. The magnitude and direction of this bias depend on the outcome definition, the true effect of the risk factor and the assumed selection mechanism; these are likely to differ between studies with different target populations. Bias due to sample selection is a key concern in COVID-19 research based on national registry data, especially as countries end free mass testing. The framework we have used can be applied by other researchers assessing the extent to which their results may be biased for their research question of interest