Evaluating the role of quality assessment of primary studies in systematic reviews of cancer practice guidelines

BACKGROUND: The purpose of this study was to evaluate the role of study quality assessment of primary studies in cancer practice guidelines. METHODS: Reliable and valid study quality assessment scales were sought and applied to published reports of trials included in systematic reviews of cancer guidelines. Sensitivity analyses were performed to evaluate the relationship between quality scores and pooled odds ratios (OR) for mortality and need for blood transfusion. RESULTS: Results found that that whether trials were classified as high or low quality depended on the scale used to assess them. Although the results of the sensitivity analyses found some variation in the ORs observed, the confidence intervals (CIs) of the pooled effects from each of the analyses of high quality trials overlapped with the CI of the pooled odds of all trials. Quality score was not predictive of pooled ORs studied here. CONCLUSIONS: Had sensitivity analyses based on study quality been conducted prospectively, it is highly unlikely that different conclusions would have been found or that different clinical recommendations would have emerged in the guidelines

Clinicians' evaluations of, endorsements of, and intentions to use practice guidelines change over time: a retrospective analysis from an organized guideline program

<p>Abstract</p> <p>Purpose</p> <p>Clinical practice guidelines (CPGs) can improve clinical care but uptake and application are inconsistent. Objectives were: to examine temporal trends in clinicians' evaluations of, endorsements of, and intentions to use cancer CPGs developed by an established CPG program; and to evaluate how predictor variables (clinician characteristics, beliefs, and attitudes) are associated with these trends.</p> <p>Design and methods</p> <p>Between 1999 and 2005, 756 clinicians evaluated 84 Cancer Care Ontario CPGs, yielding 4,091 surveys that targeted four CPG quality domains (rigour, applicability, acceptability, and comparative value), clinicians' endorsement levels, and clinicians' intentions to use CPGs in practice.</p> <p>Results</p> <p>Time: In contrast to the applicability and intention to use in practice scores, there were small but statistically significant annual net gains in ratings for rigour, acceptability, comparative value, and CPG endorsement measures (p < 0.05 for all rating categories). Predictors: In 17 comparisons, ratings were significantly higher among clinicians having the most favourable beliefs and most positive attitudes and lowest for those having the least favourable beliefs and most negative attitudes (p < 0.05). Interactions Time × Predictors: Over time, differences in outcomes among clinicians decreased due to positive net gains in scores by clinicians whose beliefs and attitudes were least favorable.</p> <p>Conclusion</p> <p>Individual differences among clinicians largely explain variances in outcomes measured. Continued engagement of clinicians least receptive to CPGs may be worthwhile because they are the ones showing most significant gains in CPG quality ratings, endorsement ratings, and intentions to use in practice ratings.</p

E-learning interventions are comparable to user's manual in a randomized trial of training strategies for the AGREE II

<p>Abstract</p> <p>Background</p> <p>Practice guidelines (PGs) are systematically developed statements intended to assist in patient and practitioner decisions. The AGREE II is the revised tool for PG development, reporting, and evaluation, comprised of 23 items, two global rating scores, and a new User's Manual. In this study, we sought to develop, execute, and evaluate the impact of two internet interventions designed to accelerate the capacity of stakeholders to use the AGREE II.</p> <p>Methods</p> <p>Participants were randomized to one of three training conditions. 'Tutorial'--participants proceeded through the online tutorial with a virtual coach and reviewed a PDF copy of the AGREE II. 'Tutorial + Practice Exercise'--in addition to the Tutorial, participants also appraised a 'practice' PG. For the practice PG appraisal, participants received feedback on how their scores compared to expert norms and formative feedback if scores fell outside the predefined range. <it>'</it>AGREE II User's Manual PDF (control condition)'<it>--</it>participants reviewed a PDF copy of the AGREE II only. All participants evaluated a test PG using the AGREE II. Outcomes of interest were learners' performance, satisfaction, self-efficacy, mental effort, time-on-task, and perceptions of AGREE II.</p> <p>Results</p> <p>No differences emerged between training conditions on any of the outcome measures.</p> <p>Conclusions</p> <p>We believe these results can be explained by better than anticipated performance of the AGREE II PDF materials (control condition) or the participants' level of health methodology and PG experience rather than the failure of the online training interventions. Some data suggest the online tools may be useful for trainees new to this field; however, this requires further study.</p

A randomized trial to evaluate e-learning interventions designed to improve learner's performance, satisfaction, and self-efficacy with the AGREE II

<p>Abstract</p> <p>Background</p> <p>Practice guidelines (PGs) are systematically developed statements intended to assist in patient, practitioner, and policy decisions. The AGREE II is the revised and updated standard tool for guideline development, reporting and evaluation. It is comprised of 23 items and a user's Manual. The AGREE II is ready for use.</p> <p>Objectives</p> <p>To develop, execute, and evaluate the impact of two internet-based educational interventions designed to accelerate the capacity of stakeholders to use the AGREE II: a multimedia didactic tutorial with a virtual coach, and a higher intensity training program including both the didactic tutorial and an interactive practice exercise component.</p> <p>Methods</p> <p>Participants (clinicians, developers, and policy makers) will be randomly assigned to one of three conditions. <it>Condition one, didactic tutorial </it>-- participants will go through the on-line AGREE II tutorial supported by a virtual coach and review of the AGREE II prior to appraising the test PG. <it>Condition two, tutorial + practice </it>-- following the multimedia didactic tutorial with a virtual coach, participants will review the on-line AGREE II independently and use it to appraise a practice PG. Upon entering their AGREE II score for the practice PG, participants will be given immediate feedback on how their score compares to expert norms. If their score falls outside a predefined range, the participant will receive a series of hints to guide the appraisal process. Participants will receive an overall summary of their performance appraising the PG compared to expert norms. <it>Condition three, control arm </it>-- participants will receive a PDF copy of the AGREE II for review and to appraise the test PG on-line. All participants will then rate one of ten test PGs with the AGREE II. The outcomes of interest are learners' performance, satisfaction, self-efficacy, mental effort, and time-on-task; comparisons will be made across each of the test groups.</p> <p>Discussion</p> <p>Our research will test innovative educational interventions of various intensities and instructional design to promote the adoption of AGREE II and to identify those strategies that are most effective for training. The results will facilitate international capacity to apply the AGREE II accurately and with confidence and to enhance the overall guideline enterprise.</p

Clinical practice guideline on the optimal radiotherapeutic management of brain metastases

BACKGROUND: An evidence-based clinical practice guideline on the optimal radiotherapeutic management of single and multiple brain metastases was developed. METHODS: A systematic review and meta-analysis was performed. The Supportive Care Guidelines Group formulated clinical recommendations based on their interpretation of the evidence. External review of the report by Ontario practitioners was obtained through a mailed survey, and final approval was obtained from Cancer Care Ontario's Practice Guidelines Coordinating Committee (PGCC). RESULTS: One hundred and nine Ontario practitioners responded to the survey (return rate 44%). Ninety-six percent of respondents agreed with the interpretation of the evidence, and 92% agreed that the report should be approved. Minor revisions were made based on feedback from external reviewers and the PGCC. The PGCC approved the final practice guideline report. CONCLUSIONS: For adult patients with a clinical and radiographic diagnosis of brain metastases (single or multiple) we conclude that, • Surgical excision should be considered for patients with good performance status, minimal or no evidence of extracranial disease, and a surgically accessible single brain metastasis. • Postoperative whole brain radiotherapy (WBRT) should be considered to reduce the risk of tumour recurrence for patients who have undergone resection of a single brain metastasis. • Radiosurgery boost with WBRT may improve survival in select patients with unresectable single brain metastases. • The whole brain should be irradiated for multiple brain metastases. Standard dose-fractionation schedules are 3000 cGy in 10 fractions or 2000 cGy in 5 fractions. • Radiosensitizers are not recommended outside research studies. • In select patients, radiosurgery may be considered as boost therapy with WBRT to improve local tumour control. Radiosurgery boost may improve survival in select patients. • Chemotherapy as primary therapy or chemotherapy with WBRT remains experimental. • Supportive care is an option but there is a lack of Level 1 evidence as to which subsets of patients should be managed with supportive care alone. Qualifying statements addressing factors to consider when applying these recommendations are provided in the full report. The rigorous development, external review and approval process has resulted in a practice guideline that is strongly endorsed by Ontario practitioners

Radiotherapy fractionation for the palliation of uncomplicated painful bone metastases – an evidence-based practice guideline

BACKGROUND: This practice guideline was developed to provide recommendations to clinicians in Ontario on the preferred standard radiotherapy fractionation schedule for the treatment of painful bone metastases. METHODS: A systematic review and meta-analysis was performed and published elsewhere. The Supportive Care Guidelines Group, a multidisciplinary guideline development panel, formulated clinical recommendations based on their interpretation of the evidence. In addition to evidence from clinical trials, the panel also considered patient convenience and ease of administration of palliative radiotherapy. External review of the draft report by Ontario practitioners was obtained through a mailed survey, and final approval was obtained from the Practice Guidelines Coordinating Committee. RESULTS: Meta-analysis did not detect a significant difference in complete or overall pain relief between single treatment and multifraction palliative radiotherapy for bone metastases. Fifty-nine Ontario practitioners responded to the mailed survey (return rate 62%). Forty-two percent also returned written comments. Eighty-three percent of respondents agreed with the interpretation of the evidence and 75% agreed that the report should be approved as a practice guideline. Minor revisions were made based on feedback from the external reviewers and the Practice Guidelines Coordinating Committee. The Practice Guidelines Coordinating Committee approved the final practice guideline report. CONCLUSION: For adult patients with single or multiple radiographically confirmed bone metastases of any histology corresponding to painful areas in previously non-irradiated areas without pathologic fractures or spinal cord/cauda equine compression, we conclude that: • Where the treatment objective is pain relief, a single 8 Gy treatment, prescribed to the appropriate target volume, is recommended as the standard dose-fractionation schedule for the treatment of symptomatic and uncomplicated bone metastases. Several factors frequently considered in clinical practice when applying this evidence such as the effect of primary histology, anatomical site of treatment, risk of pathological fracture, soft tissue disease and cord compression, use of antiemetics, and the role of retreatment are discussed as qualifying statements. Our systematic review and meta-analysis provided high quality evidence for the key recommendation in this clinical practice guideline. Qualifying statements addressing factors that should be considered when applying this recommendation in clinical practice facilitate its clinical application. The rigorous development and approval process result in a final document that is strongly endorsed by practitioners as a practice guideline

Salvage for cervical recurrences of head and neck cancer with dissection and interstitial high dose rate brachytherapy

Salvage therapy in head and neck cancer (HNC) is a controversy issue and the literature is scarce regarding the use of interstitial high-dose rate brachytherapy (I-HDR) in HNC. We evaluated the long-term results of a treatment policy combining salvage surgery and I-HDR for cervical recurrences of HNC. Charts of 21 patients treated from 1994 to 2004 were reviewed. The crude local control rate for all patients was 52.4%. The 5- and 8-years overall (OS) and local relapse-free survival (LRFS) rates were 50%, 42.9%, 42.5% and 28.6%, respectively. The only predictive factor associated to LFRS and OS was negative margin status (p = 0.0007 and p = 0.0002). We conclude that complete surgery is mandatory for long term control and the doses given by brachytherapy are not high enough to compensate for microscopic residual disease after surgery