Development and Validation of a Tool to Assess the Quality of Clinical Practice Guideline Recommendations

Clinical practice guidelines (CPGs) may lack rigor and suitability to the setting in which they are to be applied. Methods to yield clinical practice guideline recommendations that are credible and implementable remain to be determined. To describe the development of AGREE-REX (Appraisal of Guidelines Research and Evaluation-Recommendations Excellence), a tool designed to evaluate the quality of clinical practice guideline recommendations. A cross-sectional study of 322 international stakeholders representing CPG developers, users, and researchers was conducted between December 2015 and March 2019. Advertisements to participate were distributed through professional organizations as well as through the AGREE Enterprise social media accounts and their registered users. Between 2015 and 2017, participants appraised 1 of 161 CPGs using the Draft AGREE-REX tool and completed the AGREE-REX Usability Survey. Usability and measurement properties of the tool were assessed with 7-point scales (1 indicating strong disagreement and 7 indicating strong agreement). Internal consistency of items was assessed with the Cronbach, and the Spearman-Brown reliability adjustment was used to calculate reliability for 2 to 5 raters. A total of 322 participants (202 female participants [62.7%]; 83 aged 40-49 years [25.8%]) rated the survey items (on a 7-point scale). All 11 items were rated as easy to understand (with a mean [SD] ranging from 5.2 [1.38] for the alignment of values item to 6.3 [0.87] for the evidence item) and easy to apply (with a mean [SD] ranging from 4.8 [1.49] for the alignment of values item to 6.1 [1.07] for the evidence item). Participants provided favorable feedback on the tool's instructions, which were considered clear (mean [SD], 5.8 [1.06]), helpful (mean [SD], 5.9 [1.00]), and complete (mean [SD], 5.8 [1.11]). Participants considered the tool easy to use (mean [SD], 5.4 [1.32]) and thought that it added value to the guideline enterprise (mean [SD], 5.9 [1.13]). Internal consistency of the items was high (Cronbach = 0.94). Positive correlations were found between the overall AGREE-REX score and the implementability score (r = 0.81) and the clinical credibility score (r = 0.76). This cross-sectional study found that the AGREE-REX tool can be useful in evaluating CPG recommendations, differentiating among them, and identifying those that are clinically credible and implementable for practicing health professionals and decision makers who use recommendations to inform clinical policy.

Assessment of the quality of recommendations from 161 clinical practice guidelines using the Appraisal of Guidelines for Research and Evaluation-Recommendations Excellence (AGREE-REX) instrument shows there is room for improvement

To assess the quality of recommendations from 161 clinical practice guidelines (CPGs) using AGREE-REX-D (Appraisal of Guidelines REsearch and Evaluation-Recommendations Excellence Draft). International CPG community. Three hundred twenty-two international CPG developers, users, and researchers. Participants were assigned to appraise one of 161 CPGs selected for the study using the AGREE-REX-D tool Main outcome measures: AGREE-REX-D scores of 161 CPGs (7-point scale, maximum 7). Recommendations from 161 CPGs were appraised by 322 participants using the AGREE-REX-D. CPGs were developed by 67 different organizations. The total overall average score of the CPG recommendations was 4.23 (standard deviation (SD) = 1.14). AGREE-REX-D items that scored the highest were (mean; SD): evidence (5.51; 1.14), clinical relevance (5.95; SD 0.8), and patients/population relevance (4.87; SD 1.33), while the lowest scores were observed for the policy values (3.44; SD 1.53), local applicability (3,56; SD 1.47), and resources, tools, and capacity (3.49; SD 1.44) items. CPGs developed by government-supported organizations and developed in the UK and Canada had significantly higher recommendation quality scores with the AGREE-REX-D tool (p < 0.05) than their comparators. We found that there is significant room for improvement of some CPGs such as the considerations of patient/population values, policy values, local applicability and resources, tools, and capacity. These findings may be considered a baseline upon which to measure future improvements in the quality of CPGs

Charting the elements of pedagogic frailty

Background: The concept of pedagogic frailty has been proposed as a unifying concept that may help to integrate institutional efforts to enhance teaching improvement within universities by helping to maintain a simultaneous focus on four key areas that are thought to impede development. Purpose: The variation in internal structure of the four dimensions of pedagogic frailty and the links that have been proposed to connect them are explored here through the analysis of interviews with academics working in a variety of disciplinary areas. Methods: The application of concept map-mediated interviews allows us to view the variable connections within and between these dimensions and the personal ways they are conceptualised by academics working across the heterogeneous university context. Results: The data show that academics conceptualise the discourse of teaching in various ways that have implications for the links that may be developed to integrate the elements within the model. Conclusions: Whilst the form and content of the maps representing dimensions of the pedagogic frailty model exhibit considerable variation, it is suggested that factors such as academic resilience and the explicit use of integrative concepts within disciplines may help to overcome some of the vulnerabilities that accompany pedagogic frailty. The data also raises questions about the links between factors that tend to be under individual control and those that tend towards institutional control

Biomarkers in anal cancer: from biological understanding to stratified treatment

Squamous cell carcinomas of the anus and anal canal represent a model of a cancer and perhaps the first where level 1 evidence supported primary chemoradiotherapy (CRT) in treating locoregional disease with curative intent. The majority of tumours are associated with infection with oncogenic subtypes of human papilloma virus and this plays a significant role in their sensitivity to treatment. However, not all tumours are cured with CRT and there remain opportunities to improve outcomes in terms of oncological control and also reducing late toxicities. Understanding the biology of ASCC promises to allow a more personalised approach to treatment, with the development and validation of a range of biomarkers and associated techniques that are the focus of this review

Engaging Cancer Patients in Clinical Practice Guideline Development: A Pilot Study

Background: Patient engagement is a key quality component of cancer guideline development; however, the optimal strategy for engaging patients in guideline development remains unclear. The feasibility and efficacy of two patient engagement models was tested by Cancer Care Ontario’s cancer guideline development program, the Program in Evidence-Based Care (PEBC). Methods: In model 1, patients participated in the guideline development process as active members of a working group. In model 2, patients formed a separate consultation group to review project plans and recommendations generated by multiple working groups. Training included online resources (model 1) and an in-person orientation (model 2). The PEBC’s standard patient engagement process acted as a control. The study was conducted for 1 year. Surveys measured the satisfaction of patients and members of the guideline working groups with the process and the outcome of each model. Results: Three guideline projects used model 1 to engage patients, six projects used model 2 to receive feedback, and one project was used as a control group (14 patients total). Most participants, whatever the model, reported satisfaction with their experience. Key challenges to implementation included patient recruitment and long wait times between meetings (model 1), and difficulty focusing on the discussion topic and poor meeting attendance on the part of patients (model 2). Conclusions: The pilot study demonstrated that, although both models are feasible and effective for the engagement of patients in cancer guideline development, modifications are required to optimize their continued interest. The PEBC will use the study results to inform the implementation of a patient engagement strategy for its program

For health or for profit? Understanding how private financing and for-profit delivery operate within Canadian healthcare (4H|4P): protocol for a multimethod knowledge mobilisation research project

Introduction Privatisation through the expansion of private payment and investor-owned corporate healthcare delivery in Canada raises potential conflicts with equity principles on which Medicare (Canadian public health insurance) is founded. Some cases of privatisation are widely recognised, while others are evolving and more hidden, and their extent differs across provinces and territories likely due in part to variability in policies governing private payment (out-of-pocket payments and private insurance) and delivery.Methods and analysis This pan-Canadian knowledge mobilisation project will collect, classify, analyse and interpret data about investor-owned privatisation of healthcare financing and delivery systems in Canada. Learnings from the project will be used to develop, test and refine a new conceptual framework that will describe public-private interfaces operating within Canada’s healthcare system. In Phase I, we will conduct an environmental scan to: (1) document core policies that underpin public-private interfaces; and (2) describe new or emerging forms of investor-owned privatisation (‘cases’). We will analyse data from the scan and use inductive content analysis with a pragmatic approach. In Phase II, we will convene a virtual policy workshop with subject matter experts to refine the findings from the environmental scan and, using an adapted James Lind Alliance Delphi process, prioritise health system sectors and/or services in need of in-depth research on the impacts of private financing and investor-owned delivery.Ethics and dissemination We have obtained approval from the research ethics boards at Simon Fraser University, University of British Columbia and University of Victoria through Research Ethics British Columbia (H23-00612). Participants will provide written informed consent. In addition to traditional academic publications, study results will be summarised in a policy report and a series of targeted policy briefs distributed to workshop participants and decision/policymaking organisations across Canada. The prioritised list of cases will form the basis for future research projects that will investigate the impacts of investor-owned privatisation