Evidence-based Urology:Understanding Heterogeneity in Systematic Reviews

Variability in the results of randomized trials presents challenges to the interpretation and application of the evidence to patient care. Understanding how systematic reviews deal with this problem of "heterogeneity" will help clinicians in applying results in their patient management. This manuscript offers a review of heterogeneity from the clinical urological perspective.  PATIENT SUMMARY: Systematic reviews of the literature are necessary to accurately summarize the available evidence to inform clinical decisions. In this mini-review, we explain how to understand and deal with the differences between studies-which we call heterogeneity-included in these types of reviews

Premature Discontinuation of Pediatric Randomized Controlled Trials : A Retrospective Cohort Study

Objectives To determine the proportion of pediatric randomized controlled trials (RCTs) that are prematurely discontinued, examine the reasons for discontinuation, and compare the risk for recruitment failure in pediatric and adult RCTs. Study design A retrospective cohort study of RCTs approved by 1 of 6 Research Ethics Committees (RECs) in Switzerland, Germany, and Canada between 2000 and 2003. We recorded trial characteristics, trial discontinuation, and reasons for discontinuation from protocols, corresponding publications, REC files, and a survey of trialists. Results We included 894 RCTs, of which 86 enrolled children and 808 enrolled adults. Forty percent of the pediatric RCTs and 29% of the adult RCTs were discontinued. Slow recruitment accounted for 56% of pediatric RCT discontinuations and 43% of adult RCT discontinuations. Multivariable logistic regression analyses suggested that pediatric RCT was not an independent risk factor for recruitment failure after adjustment for other potential risk factors (aOR, 1.22; 95% CI, 0.57-2.63). Independent risk factors were acute care setting (aOR, 4.00; 95% CI, 1.72-9.31), nonindustry sponsorship (aOR, 4.45; 95% CI, 2.59-7.65), and smaller planned sample size (aOR, 1.05; 95% CI 1.01-1.09, in decrements of 100 participants). Conclusion Forty percent of pediatric RCTs were discontinued prematurely, owing predominately to slow recruitment. Enrollment of children was not an independent risk factor for recruitment failure.Peer reviewe

Effect of remdesivir post hospitalization for COVID-19 infection from the randomized SOLIDARITY Finland trial

We report the first long-term follow-up of a randomized trial (NCT04978259) addressing the effects of remdesivir on recovery (primary outcome) and other patient-important outcomes one year after hospitalization resulting from COVID-19. Of the 208 patients recruited from 11 Finnish hospitals, 198 survived, of whom 181 (92%) completed follow-up. At one year, self-reported recovery occurred in 85% in remdesivir and 86% in standard of care (SoC) (RR 0.94, 95% CI 0.47-1.90). We infer no convincing difference between remdesivir and SoC in quality of life or symptom outcomes (p > 0.05). Of the 21 potential long-COVID symptoms, patients reported moderate/major bother from fatigue (26%), joint pain (22%), and problems with memory (19%) and attention/concentration (18%). In conclusion, after a one-year follow-up of hospitalized patients, one in six reported they had not recovered well from COVID-19. Our results provide no convincing evidence of remdesivir benefit, but wide confidence intervals included possible benefit and harm.Peer reviewe

Systematic reviews of observational studies of Risk of Thrombosis and Bleeding in General and Gynecologic Surgery (ROTBIGGS): introduction and methodology

Background: Venous thromboembolism (VTE) and bleeding are serious and potentially fatal complications of surgical procedures. Pharmacological thromboprophylaxis decreases the risk of VTE but increases the risk of major post-operative bleeding. The decision to use pharmacologic prophylaxis therefore represents a trade-off that critically depends on the incidence of VTE and bleeding in the absence of prophylaxis. These baseline risks vary widely between procedures, but their magnitude is uncertain. Systematic reviews addressing baseline risks are scarce, needed, and require innovations in methodology. Indeed, systematic summaries of these baseline risk estimates exist neither in general nor gynecologic surgery. We will fill this knowledge gap by performing a series of systematic reviews and meta-analyses of the procedure-specific and patient risk factor stratified risk estimates in general and gynecologic surgeries.Methods: We will perform comprehensive literature searches for observational studies in general and gynecologic surgery reporting symptomatic VTE or bleeding estimates. Pairs of methodologically trained reviewers will independently assess the studies for eligibility, evaluate the risk of bias by using an instrument developed for this review, and extract data. We will perform meta-analyses and modeling studies to adjust the reported risk estimates for the use of thromboprophylaxis and length of follow up. We will derive the estimates of risk from the median estimates of studies rated at the lowest risk of bias. The primary outcomes are the risk estimates of symptomatic VTE and major bleeding at 4 weeks post-operatively for each procedure stratified by patient risk factors. We will apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to rate evidence certainty.Discussion: This series of systematic reviews, modeling studies, and meta-analyses will inform clinicians and patients regarding the trade-off between VTE prevention and bleeding in general and gynecologic surgeries. Our work advances the standards in systematic reviews of surgical complications, including assessment of risk of bias, criteria for arriving at the best estimates of risk (including modeling of the timing of events and dealing with suboptimal data reporting), dealing with subgroups at higher and lower risk of bias, and use of the GRADE approach.Systematic review registration: PROSPERO CRD42021234119</p

Procedure-specific Risks of Thrombosis and Bleeding in Urological Non-cancer Surgery : Systematic Review and Meta-analysis

Context: Pharmacological thromboprophylaxis involves a trade-off between a reduction in venous thromboembolism (VTE) and increased bleeding. No guidance specific for procedure and patient factors exists in urology. Objective: To inform estimates of absolute risk of symptomatic VTE and bleeding requiring reoperation in urological non-cancer surgery. Evidence acquisition: We searched for contemporary observational studies and estimated the risk of symptomatic VTE or bleeding requiring reoperation in the 4 wk after urological surgery. We used the GRADE approach to assess the quality of the evidence. Evidence synthesis: The 37 eligible studies reported on 11 urological non-cancer procedures. The duration of prophylaxis varied widely both within and between procedures; for example, the median was 12.3 d (interquartile range [IQR] 3.1-55) for open recipient nephrectomy (kidney transplantation) studies and 1 d (IQR 0-1.3) for percutaneous nephrolithotomy, open prolapse surgery, and reconstructive pelvic surgery studies. Studies of open recipient nephrectomy reported the highest risks of VTE and bleeding (1.8-7.4% depending on patient characteristics and 2.4% for bleeding). The risk of VTE was low for 8/11 procedures (0.2-0.7% for patients with low/medium risk; 0.8-1.4% for high risk) and the risk of bleeding was low for 6/7 procedures ( Conclusions: Although inferences are limited owing to low-quality evidence, our results suggest that extended prophylaxis is warranted for some procedures (eg, kidney transplantation procedures in high-risk patients) but not others (transurethral resection of the prostate and reconstructive female pelvic surgery in low-risk patients). Patient summary: The best evidence suggests that the benefits of blood-thinning drugs to prevent clots after surgery outweigh the risks of bleeding in some procedures ( such as kidney transplantation procedures in patients at high risk of clots) but not others ( such as prostate surgery in patients at low risk of clots). (C) 2017 European Association of Urology. Published by Elsevier B.V.Peer reviewe

Procedure-specific Risks of Thrombosis and Bleeding in Urological Cancer Surgery : Systematic Review and Meta-analysis

Context: Pharmacological thromboprophylaxis involves balancing a lower risk of venous thromboembolism (VTE) against a higher risk of bleeding, a trade-off that critically depends on the risks of VTE and bleeding in the absence of prophylaxis (baseline risk). Objective: To provide estimates of the baseline risk of symptomatic VTE and bleeding requiring reoperation in urological cancer surgery. Evidence acquisition: We identified contemporary observational studies reporting symptomatic VTE or bleeding after urological procedures. We used studies with the lowest risk of bias and accounted for use of thromboprophylaxis and length of follow-up to derive best estimates of the baseline risks within 4wk of surgery. We used the GRADE approach to assess the quality of the evidence. Evidence synthesis: We included 71 studies reporting on 14 urological cancer procedures. The quality of the evidence was generally moderate for prostatectomy and cystectomy, and low or very low for other procedures. The duration of thromboprophylaxis was highly variable. The risk of VTE in cystectomies was high (2.6-11.6% across risk groups) whereas the risk of bleeding was low (0.3%). The risk of VTE in prostatectomies varied by procedure, from 0.2-0.9% in robotic prostatectomy without pelvic lymph node dissection (PLND) to 3.9-15.7% in open prostatectomy with extended PLND. The risk of bleeding was 0.1-1.0%. The risk of VTE following renal procedures was 0.7-2.9% for low-risk patients and 2.6-11.6% for high-risk patients; the risk of bleeding was 0.1-2.0%. Conclusions: Extended thromboprophylaxis is warranted in some procedures (eg, open and robotic cystectomy) but not others (eg, robotic prostatectomy without PLND in low-risk patients). For "close call'' procedures, decisions will depend on values and preferences with regard to VTE and bleeding. Patient summary: Clinicians often give blood thinners to patients to prevent blood clots after surgery for urological cancer. Unfortunately, blood thinners also increase bleeding. This study provides information on the risk of clots and bleeding that is crucial in deciding for or against giving blood thinners. (C) 2017 European Association of Urology. Published by Elsevier B.V.Peer reviewe

Do clinicians understand the size of treatment effects? A randomized survey across 8 countries.

BACKGROUND Meta-analyses of continuous outcomes typically provide enough information for decision-makers to evaluate the extent to which chance can explain apparent differences between interventions. The interpretation of the magnitude of these differences - from trivial to large - can, however, be challenging. We investigated clinicians' understanding and perceptions of usefulness of 6 statistical formats for presenting continuous outcomes from meta-analyses (standardized mean difference, minimal important difference units, mean difference in natural units, ratio of means, relative risk and risk difference). METHODS We invited 610 staff and trainees in internal medicine and family medicine programs in 8 countries to participate. Paper-based, self-administered questionnaires presented summary estimates of hypothetical interventions versus placebo for chronic pain. The estimates showed either a small or a large effect for each of the 6 statistical formats for presenting continuous outcomes. Questions addressed participants' understanding of the magnitude of treatment effects and their perception of the usefulness of the presentation format. We randomly assigned participants 1 of 4 versions of the questionnaire, each with a different effect size (large or small) and presentation order for the 6 formats (1 to 6, or 6 to 1). RESULTS Overall, 531 (87.0%) of the clinicians responded. Respondents best understood risk difference, followed by relative risk and ratio of means. Similarly, they perceived the dichotomous presentation of continuous outcomes (relative risk and risk difference) to be most useful. Presenting results as a standardized mean difference, the longest standing and most widely used approach, was poorly understood and perceived as least useful. INTERPRETATION None of the presentation formats were well understood or perceived as extremely useful. Clinicians best understood the dichotomous presentations of continuous outcomes and perceived them to be the most useful. Further initiatives to help clinicians better grasp the magnitude of the treatment effect are needed

Randomized controlled trials in de-implementation research:a systematic scoping review

Abstract Background: Healthcare costs are rising, and a substantial proportion of medical care is of little value. De-implementation of low-value practices is important for improving overall health outcomes and reducing costs. We aimed to identify and synthesize randomized controlled trials (RCTs) on de-implementation interventions and to provide guidance to improve future research. Methods: MEDLINE and Scopus up to May 24, 2021, for individual and cluster RCTs comparing de-implementation interventions to usual care, another intervention, or placebo. We applied independent duplicate assessment of eligibility, study characteristics, outcomes, intervention categories, implementation theories, and risk of bias. Results: Of the 227 eligible trials, 145 (64%) were cluster randomized trials (median 24 clusters; median follow-up time 305 days), and 82 (36%) were individually randomized trials (median follow-up time 274 days). Of the trials, 118 (52%) were published after 2010, 149 (66%) were conducted in a primary care setting, 163 (72%) aimed to reduce the use of drug treatment, 194 (85%) measured the total volume of care, and 64 (28%) low-value care use as outcomes. Of the trials, 48 (21%) described a theoretical basis for the intervention, and 40 (18%) had the study tailored by context-specific factors. Of the de-implementation interventions, 193 (85%) were targeted at physicians, 115 (51%) tested educational sessions, and 152 (67%) multicomponent interventions. Missing data led to high risk of bias in 137 (60%) trials, followed by baseline imbalances in 99 (44%), and deficiencies in allocation concealment in 56 (25%). Conclusions: De-implementation trials were mainly conducted in primary care and typically aimed to reduce low-value drug treatments. Limitations of current de-implementation research may have led to unreliable effect estimates and decreased clinical applicability of studied de-implementation strategies. We identified potential research gaps, including de-implementation in secondary and tertiary care settings, and interventions targeted at other than physicians. Future trials could be improved by favoring simpler intervention designs, better control of potential confounders, larger number of clusters in cluster trials, considering context-specific factors when planning the intervention (tailoring), and using a theoretical basis in intervention design

Reporting of costs and economic impacts in randomized trials of de-implementation interventions for low-value care:a systematic scoping review

Abstract Background: De-implementation of low-value care can increase health care sustainability. We evaluated the reporting of direct costs of de-implementation and subsequent change (increase or decrease) in health care costs in randomized trials of de-implementation research. Methods: We searched MEDLINE and Scopus databases without any language restrictions up to May 2021. We conducted study screening and data extraction independently and in duplicate. We extracted information related to study characteristics, types and characteristics of interventions, de-implementation costs, and impacts on health care costs. We assessed risk of bias using a modified Cochrane risk-of-bias tool. Results: We screened 10,733 articles, with 227 studies meeting the inclusion criteria, of which 50 included information on direct cost of de-implementation or impact of de-implementation on health care costs. Studies were mostly conducted in North America (36%) or Europe (32%) and in the primary care context (70%). The most common practice of interest was reduction in the use of antibiotics or other medications (74%). Most studies used education strategies (meetings, materials) (64%). Studies used either a single strategy (52%) or were multifaceted (48%). Of the 227 eligible studies, 18 (8%) reported on direct costs of the used de-implementation strategy; of which, 13 reported total costs, and 12 reported per unit costs (7 reported both). The costs of de-implementation strategies varied considerably. Of the 227 eligible studies, 43 (19%) reported on impact of de-implementation on health care costs. Health care costs decreased in 27 studies (63%), increased in 2 (5%), and were unchanged in 14 (33%). Conclusion: De-implementation randomized controlled trials typically did not report direct costs of the de-implementation strategies (92%) or the impacts of de-implementation on health care costs (81%). Lack of cost information may limit the value of de-implementation trials to decision-makers. Trial registration: OSF (Open Science Framework): https://osf.io/ueq32