Empirical evidence of study design biases in randomized trials:Systematic review of meta-epidemiological studies

To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials.Systematic review of meta-epidemiological studies.We retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic.We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear.Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed

Influence and management of conflicts of interest in randomised clinical trials : qualitative interview study

OBJECTIVE: To characterise and analyse the experiences of trial researchers of if and how conflicts of interest had unduly influenced clinical trials they had worked on, what management strategies they had used to minimise any potential influence, and their experiences and views on conflicts of interest more generally. DESIGN: Qualitative interview study. PARTICIPANTS: Trial researchers who had participated in at least 10 clinical trials with methodological or statistical expertise. Researchers differed by geographical location, educational background, and experience with different types of funders. Interviewees were identified by searches on Web of Science and snowball sampling. 52 trial researchers were approached by email; 20 agreed to be interviewed. SETTING: Interviews conducted by telephone, recorded, transcribed verbatim, imported to NVivo 12, and analysed by systematic text condensation. Semistructured interviews focused on financial and non-financial conflicts of interest. RESULTS: The interviewees had participated in a median of 37.5 trials and were mainly male physicians who had experience with commercial and non-commercial trial funders. Two predefined themes (influence of conflicts of interest and management strategies) and two additional themes (definition and reporting of conflicts of interest) emerged. Examples of perceived influence of conflicts of interest were: choice of inferior comparator, manipulation of the randomisation process, prematurely stopping the trials, fabrication of data, blocking access to data, and spin (eg, overly favourable interpretation of the results). Examples of strategies to manage conflicts of interest were: disclosure procedures, exclusion of the funder from design and analysis, independent committees, contracts ensuring complete access to the data, and no restriction by the funder on analysis and reporting. Interviewees used different definitions or thresholds for what they considered to be conflicts of interest, and they described different criteria for when to report them. Some interviewees considered non-commercial financial conflicts of interest (eg, funding of trials by governmental health agencies with a political agenda) to be equally or more important than commercial financial conflicts of interest (eg, funding by drug and device companies), but more challenging to report and manage. CONCLUSION: This study described how trial researchers perceive conflicts of interest unduly influencing clinical trials they had worked on, and the management strategies they used to prevent these influences. The results indicated considerable variability in researchers' understanding of what conflicts of interest are and when they should be reported

Three-Armed Trials Including Placebo and No-Treatment Groups May Be Subject to Publication Bias: Systematic Review

Background: It has been argued that placebos may not have important clinical impacts in general. However, there is increasing evidence of a publication bias among trials published in journals. Therefore, we explored the potential for publication bias in randomized trials with active treatment, placebo, and no-treatment groups. Methods: Three-armed randomized trials of acupuncture, acupoint stimulation, and transcutaneous electrical stimulation were obtained from electronic databases. Effect sizes between treatment and placebo groups were calculated for treatment effect, and effect sizes between placebo and no-treatment groups were calculated for placebo effect. All data were then analyzed for publication bias. Results: For the treatment effect, small trials with fewer than 100 patients per arm showed more benefits than large trials with at least 100 patients per arm in acupuncture and acupoint stimulation. For the placebo effect, no differences were found between large and small trials. Further analyses showed that the treatment effect in acupuncture and acupoint stimulation may be subject to publication bias because study design and any known factors of heterogeneity were not associated with the small study effects. In the simulation, the magnitude of the placebo effect was smaller than that calculated after considering publication bias. Conclusions: Randomized three-armed trials, which are necessary for estimating the placebo effect, may be subject t

Systematic review finds that appraisal tools for medical research studies address conflicts of interest superficially

OBJECTIVE: To identify and summarise 1) appraisal tools and other guides which address conflicts of interest in medical research studies; and 2) top journals with policies on managing conflicts of interest in journal papers. STUDY DESIGN AND SETTING: Systematic review. We searched bibliographic databases, other sources and websites of 30 top medical journals. Two authors selected documents and extracted data. RESULTS: We included 27 appraisal tools. None were designed specifically for addressing conflicts of interest and they included only 1-2 short items on conflicts of interest. We also included eight other types of guides. Of 27 appraisal tools, 23 addressed study funding and 19 authors' conflicts of interest. Nine tools addressed availability of conflicts of interest information, 13 reported conflicts of interest, and five influence from conflicts of interest. Twelve of 30 top journals had conflicts of interest managing policies (beyond disclosure). One journal restricted non-research papers (e.g. editorials) to authors without financial conflicts of interest and ten only restricted under certain circumstances. CONCLUSION: Appraisal tools that address conflicts of interest typically do so superficially and rarely address how conflicts of interest may influence studies. Less than half of top medical journals have explicit policies on managing conflicts of interest

A structured approach to information retrieval improved identification of funding and researchers’ conflicts of interest in trials included in Cochrane reviews

Objectives: To compare the contemporary Cochrane review approach for retrieving information on trial funding and researchers’ conflicts of interest with a structured approach for information retrieval. Study Design and Setting: Methodological study of 100 Cochrane reviews from August to December 2020 and one randomly selected trial from each review. Reporting of trial funding and researchers’ conflicts of interest in reviews was compared with information identified using a structured retrieval process, and time to retrieve information was noted. We also formulated a guide to systematic reviewers for efficient information retrieval. Results: Sixty-eight of 100 Cochrane reviews reported trial funding and 24 reported trial researchers’ conflicts of interest. A simple structured approach, searching only trial publications (including conflicts of interest disclosure forms), identified funding for 16 additional trials and conflicts of interest information for 39 additional trials. A comprehensive structured approach, searching multiple information sources, identified funding for two additional trials and conflicts of interest for 14 additional trials. The median time to retrieve information was 10 minutes per trial (interquartile range: 7–15) for the simple approach and 20 minutes (11–43) for the comprehensive approach. Conclusion: A structured information retrieval approach improves identification of funding and researchers’ conflicts of interest in trials included in Cochrane reviews

Use of placebo interventions among Swiss primary care providers

Background: Placebo interventions can have meaningful effects for patients. However, little is known about the circumstances of their use in clinical practice. We aimed to investigate to what extent and in which way Swiss primary care providers use placebo interventions. Furthermore we explored their ideas about the ethical and legal issues involved. Methods: 599 questionnaires were sent to general practitioners (GPs) and paediatricians in private practice in the Canton of Zurich in Switzerland. To allow for subgroup analysis GPs in urban, suburban, and rural areas as well as paediatricians were selected in an even ratio. Results: 233 questionnaires were completed (response rate 47%). 28% of participants reported that they never used placebo interventions. More participants used impure placebos therapeutically than pure placebos (57% versus 17%, McNemar's chi2 = 78, p<0.001). There is not one clear main reason for placebo prescription. Placebo use was communicated to patients mostly as being "a drug or a therapy" (64%). The most frequently chosen ethical premise was that they "can be used as long as the physician and the patient work together in partnership" (60% for pure and 75% for impure placebos, McNemar's chi2 = 12, p<0.001). A considerable number of participants (11-38%) were indecisive about statements regarding the ethical and legal legitimacy of using placebos. Conclusions: The data obtained from Swiss primary care providers reflect a broad variety of views about placebo interventions as well as a widespread uncertainty regarding their legitimacy. Primary care providers seem to preferentially use impure as compared to pure placebos in their daily practice. An intense debate is required on appropriate standards regarding the clinical use of placebo interventions among medical professionals

Agreement was moderate between data-based and opinion-based assessments of biases affecting randomised trials within meta-analyses

BACKGROUND: Randomised trials included in meta-analyses are often affected by bias caused by methodological flaws or limitations, but the degree of bias is unknown. Two proposed methods adjust trial results for bias using: (1) empirical evidence from published meta-epidemiological studies; or (2) expert opinion. METHODS: We investigated agreement between data-based and opinion-based approaches to assessing bias in each of four domains: sequence generation, allocation concealment, blinding and incomplete outcome data. From each sampled meta-analysis, a pair of trials with the highest and lowest empirical model-based bias estimates was selected. Independent assessors were asked which trial within each pair was judged more biased on the basis of detailed trial design summaries. RESULTS: Assessors judged trials to be equally biased in 68% of pairs evaluated. When assessors judged one trial as more biased, the proportion of judgements agreeing with the model-based ranking was highest for allocation concealment (79%) and blinding (79%) and lower for sequence generation (59%) and incomplete outcome data (56%). CONCLUSIONS: Most trial pairs found to be discrepant empirically were judged to be equally biased by assessors. We found moderate agreement between opinion and data-based evidence in pairs where assessors ranked one trial as more biased