Ten tips for successful assessment of risk of bias in randomized trials using the RoB 2 tool: Early lessons from Cochrane

Introduction: RoB 2 is a tool used by systematic reviewers to assess risk of bias in randomized trials. Over a period of 19 months working as editors for Cochrane, we saw many instances where users of RoB 2 frequently applied the tool in ways the developers had not intended, despite availability of detailed guidance, webinars and FAQs. Methods: In this paper we highlight the ten main issues that we observed, with the aims of optimising the application of the RoB 2 tool, avoiding some of the frequent misapplications of the tool. Results: Issues noted included failure to state an effect of interest, applying the tool to an entire study rather than to a specific numerical result, omitting key signaling questions and relying on outdated views of causes of bias. Conclusion: Such omissions and misapplications can lead to overly harsh or lenient assessments of bias with potential to change the confidence we have in an evidence base of randomized trials. We recommend that teams planning to use RoB 2 include at least one member familiar with the RoB 2 detailed guidance and that they use the free resources, such as webinars and FAQs, from the developers of RoB 2 and Cochrane. Our ten tips should be useful to non‐Cochrane systematic reviewers as well as to peer reviewers and editors in Cochrane and other journals

Risk factors in adolescent suicides

Samoubojstvo je treći vodeći uzrok smrti među adolescentima. Brojni su uzroci koji dovode do povećanja suicidalnog rizika. Dominiraju mentalne bolesti jer je 90 % onih koji su počinili samoubojstvo imalo psihijatrijsku dijagnozu. Odnosi unutar obitelji te odnosi s okolinom također imaju važnu ulogu. Nepripadanje dominantnoj kulturi može povećati suicidalni rizik, pogotovo u onim sredinama koje su zatvorene i netolerantne. Kao najčešći motivi za samoubojstvo mladih spominju se: školski neuspjeh, pad razreda, isključenje iz škole, socijalna nestabilnost, nesigurnost, patološka ljubomora, konflikti s roditeljima i profesorima. Upravo razumijevanje rizičnih čimbenika koji mogu dovesti do samoubojstva ključno je u prevenciji ovakvog ponašanja.Suicide is the third leading cause of death among adolescents. Many causes canincrease suicidal risk. 90 % of those who committed suicide had psychiatric diagnosis. Relationships within the family and with the social surroundings also play an important role. Not being a part of the dominant culture may increase suicidal risk, especially in those societies which are closed and intolerant. The most common motives for youth suicide are school failure, drop in grades, expulsion from school, social instability, insecurity, pathological jealousy, conflict with parents and teachers. Understanding risk factors which can lead to suicide is crucial in preventing this behavior

Impregnated central venous catheters for prevention of bloodstream infection in children (the CATCH trial): a randomised controlled trial.

BACKGROUND: Impregnated central venous catheters are recommended for adults to reduce bloodstream infections but not for children because there is not enough evidence to prove they are effective. We aimed to assess the effectiveness of any type of impregnation (antibiotic or heparin) compared with standard central venous catheters to prevent bloodstream infections in children needing intensive care. METHODS: We did a randomised controlled trial of children admitted to 14 English paediatric intensive care units. Children younger than 16 years were eligible if they were admitted or being prepared for admission to a participating paediatric intensive care unit and were expected to need a central venous catheter for 3 or more days. Children were randomly assigned (1:1:1) to receive a central venous catheter impregnated with antibiotics, a central venous catheter impregnated with heparin, or a standard central venous catheter with computer generated randomisation in blocks of three and six, stratified by method of consent, site, and envelope storage location within the site. The clinician responsible for inserting the central venous catheter was not masked to allocation, but allocation was concealed from patients, their parents, and the paediatric intensive care unit personnel responsible for their care. The primary outcome was time to first bloodstream infection between 48 h after randomisation and 48 h after central venous catheter removal with impregnated (antibiotic or heparin) versus standard central venous catheters, assessed in the intention-to-treat population. Safety analyses compared central venous catheter-related adverse events in the subset of children for whom central venous catheter insertion was attempted (per-protocol population). This trial is registered with ISRCTN number, ISRCTN34884569. FINDINGS: Between Nov 25, 2010, and Nov 30, 2012, 1485 children were recruited to this study. We randomly assigned 502 children to receive standard central venous catheters, 486 to receive antibiotic-impregnated catheters, and 497 to receive heparin-impregnated catheters. Bloodstream infection occurred in 18 (4%) of those in the standard catheters group, 7 (1%) in the antibiotic-impregnated group, and 17 (3%) assigned to heparin-impregnated catheters. Primary analyses showed no effect of impregnated (antibiotic or heparin) catheters compared with standard central venous catheters (hazard ratio [HR] for time to first bloodstream infection 0.71, 95% CI 0.37-1.34). Secondary analyses showed that antibiotic central venous catheters were better than standard central venous catheters (HR 0.43, 0.20-0.96) and heparin central venous catheters (HR 0.42, 0.19-0.93), but heparin did not differ from standard central venous catheters (HR 1.04, 0.53-2.03). Clinically important and statistically significant absolute risk differences were identified only for antibiotic-impregnated catheters versus standard catheters (-2.15%, 95% CI -4.09 to -0.20; number needed to treat [NNT] 47, 95% CI 25-500) and antibiotic-impregnated catheters versus heparin-impregnated catheters (-1.98%, -3.90 to -0.06, NNT 51, 26-1667). Nine children (2%) in the standard central venous catheter group, 14 (3%) in the antibiotic-impregnated group, and 8 (2%) in the heparin-impregnated group had catheter-related adverse events. 45 (8%) in the standard group, 35 (8%) antibiotic-impregnated group, and 29 (6%) in the heparin-impregnated group died during the study. INTERPRETATION: Antibiotic-impregnated central venous catheters significantly reduced the risk of bloodstream infections compared with standard and heparin central venous catheters. Widespread use of antibiotic-impregnated central venous catheters could help prevent bloodstream infections in paediatric intensive care units. FUNDING: National Institute for Health Research, UK

Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. METHODOLOGY/PRINCIPAL FINDINGS: We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials

How Much Participant Outcome Data Is Missing from Sight:Findings from a Cohort of Trials Submitted to a German Research Ethics Committee

Study publication bias and outcome reporting bias have been recognised as two threats to the validity of systematic reviews. The purpose of this research was to estimate the proportion of missing participant outcome data from randomised controlled trials (RCTs) due to lack of publication of whole studies and due to outcome data missing within study publications.Data were extracted from protocols of clinical research projects submitted to the research ethics committee of the University of Freiburg (Germany) between 2000 and 2002 and associated fully published articles. The total amount of published and unpublished outcome data from all trial participants was calculated for each trial and the overall proportion of missing data from both unpublished and published trials computed. Full and partially reported outcome data was also taken into consideration. The impact of funding source on missingness was also considered at the trial level. From 308 parallel group trials in the study cohort, 167 were published and 141 were unpublished. Overall, 260,563 participants contributed to a total of 2,618,116 participant outcome data across all trials. About half (47%) of the participant outcome data from the 308 trials was reported in full but at least 81% were partially reported. Of the 19% of participant data that were missing, 4% was attributable to missing data from published trials and 15% from unpublished trials. Commercially funded trials had a higher probability of publication (relative risk 1.20, 95% confidence interval 0.86, 1.67; p = 0.27) but were less likely to fully report all outcomes than non-commercially funded trials (relative risk 0.64, 95% confidence interval 0.30, 1.38; p = 0.26).Missing participant outcome data from both published and unpublished trials is frequent. Clinical trial registration including outcome information not only identifies that clinical trials exist but the systematic examination and monitoring of trial information within a registry can help detect selective reporting of entire studies and of outcome data within studies and possibly prevent it

Outcome, participant and funding characteristics of trials that were published and unpublished.

<p>Outcome, participant and funding characteristics of trials that were published and unpublished.</p

Proportion of reported and missing participant outcome data.

<p>Proportion of reported and missing participant outcome data.</p

Exercise for depression.

BACKGROUND: Depression is a common and important cause of morbidity and mortality worldwide. Depression is commonly treated with antidepressants and/or psychological therapy, but some people may prefer alternative approaches such as exercise. There are a number of theoretical reasons why exercise may improve depression. This is an update of an earlier review first published in 2009. OBJECTIVES: To determine the effectiveness of exercise in the treatment of depression in adults compared with no treatment or a comparator intervention. SEARCH METHODS: We searched the Cochrane Depression, Anxiety and Neurosis Review Group’s Controlled Trials Register (CCDANCTR) to 13 July 2012. This register includes relevant randomised controlled trials from the following bibliographic databases: The Cochrane Library (all years); MEDLINE (1950 to date); EMBASE (1974 to date) and PsycINFO (1967 to date). We also searched www.controlled‐trials.com, ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform. No date or language restrictions were applied to the search. We conducted an additional search of the CCDANCTR up to 1st March 2013 and any potentially eligible trials not already included are listed as 'awaiting classification.' SELECTION CRITERIA: Randomised controlled trials in which exercise (defined according to American College of Sports Medicine criteria) was compared to standard treatment, no treatment or a placebo treatment, pharmacological treatment, psychological treatment or other active treatment in adults (aged 18 and over) with depression, as defined by trial authors. We included cluster trials and those that randomised individuals. We excluded trials of postnatal depression. DATA COLLECTION AND ANALYSIS: Two review authors extracted data on primary and secondary outcomes at the end of the trial and end of follow‐up (if available). We calculated effect sizes for each trial using Hedges' g method and a standardised mean difference (SMD) for the overall pooled effect, using a random‐effects model risk ratio for dichotomous data. Where trials used a number of different tools to assess depression, we included the main outcome measure only in the meta‐analysis. Where trials provided several 'doses' of exercise, we used data from the biggest 'dose' of exercise, and performed sensitivity analyses using the lower 'dose'. We performed subgroup analyses to explore the influence of method of diagnosis of depression (diagnostic interview or cut‐off point on scale), intensity of exercise and the number of sessions of exercise on effect sizes. Two authors performed the 'Risk of bias' assessments. Our sensitivity analyses explored the influence of study quality on outcome. MAIN RESULTS: Thirty‐nine trials (2326 participants) fulfilled our inclusion criteria, of which 37 provided data for meta‐analyses. There were multiple sources of bias in many of the trials; randomisation was adequately concealed in 14 studies, 15 used intention‐to‐treat analyses and 12 used blinded outcome assessors. For the 35 trials (1356 participants) comparing exercise with no treatment or a control intervention, the pooled SMD for the primary outcome of depression at the end of treatment was ‐0.62 (95% confidence interval (CI) ‐0.81 to ‐0.42), indicating a moderate clinical effect. There was moderate heterogeneity (I² = 63%). When we included only the six trials (464 participants) with adequate allocation concealment, intention‐to‐treat analysis and blinded outcome assessment, the pooled SMD for this outcome was not statistically significant (‐0.18, 95% CI ‐0.47 to 0.11). Pooled data from the eight trials (377 participants) providing long‐term follow‐up data on mood found a small effect in favour of exercise (SMD ‐0.33, 95% CI ‐0.63 to ‐0.03). Twenty‐nine trials reported acceptability of treatment, three trials reported quality of life, none reported cost, and six reported adverse events. For acceptability of treatment (assessed by number of drop‐outs during the intervention), the risk ratio was 1.00 (95% CI 0.97 to 1.04). Seven trials compared exercise with psychological therapy (189 participants), and found no significant difference (SMD ‐0.03, 95% CI ‐0.32 to 0.26). Four trials (n = 300) compared exercise with pharmacological treatment and found no significant difference (SMD ‐0.11, ‐0.34, 0.12). One trial (n = 18) reported that exercise was more effective than bright light therapy (MD ‐6.40, 95% CI ‐10.20 to ‐2.60). For each trial that was included, two authors independently assessed for sources of bias in accordance with the Cochrane Collaboration 'Risk of bias' tool. In exercise trials, there are inherent difficulties in blinding both those receiving the intervention and those delivering the intervention. Many trials used participant self‐report rating scales as a method for post‐intervention analysis, which also has the potential to bias findings. AUTHORS' CONCLUSIONS: Exercise is moderately more effective than a control intervention for reducing symptoms of depression, but analysis of methodologically robust trials only shows a smaller effect in favour of exercise. When compared to psychological or pharmacological therapies, exercise appears to be no more effective, though this conclusion is based on a few small trials