Validity and usability testing of a health systems guidance appraisal tool, the AGREE-HS

Health systems guidance (HSG) provides recommendations to address health systems challenges. No tools exist to inform HSG developers and users about the components of high quality HSG and to differentiate between HSG of varying quality. In response, we developed a tool to assist with the development, reporting and appraisal of HSG - the Appraisal of Guidelines for Research and Evaluation-Health Systems (AGREE-HS). This paper reports on the validity, usability and initial measurement properties of the AGREE-HS.; To establish face validity (Study 1), stakeholders completed a survey about the AGREE-HS and provided feedback on its content and structure. Revisions to the tool were made in response. To establish usability (Study 2), the revised tool was applied to 85 HSG documents and the appraisers provided feedback about their experiences via an online survey. An initial test of the revised tool's measurement properties, including internal consistency, inter-rater reliability and criterion validity, was conducted. Additional revisions to the tool were made in response.; In Study 1, the AGREE-HS Overview, User Manual, quality item content and structure, and overall assessment questions were rated favourably. Participants indicated that the AGREE-HS would be useful, feasible to use, and that they would apply it in their context. In Study 2, participants indicated that the quality items were easy to understand and apply, and the User Manual, usefulness and usability of the tool were rated favourably. Study 2 participants also indicated intentions to use the AGREE-HS.; The AGREE-HS comprises a User Manual, five quality items and two overall assessment questions. It is available at agreetrust.org

Additional file 1: of Understanding optimal approaches to patient and caregiver engagement in the development of cancer practice guidelines: a mixed methods study

Survey. This survey was completed by a group of eligible participants who were unable or preferred not to attend the workshop sessions. The survey asks respondents to comment on their knowledge of PGs, information and participation preferences, attitudes towards participating in PG development, and anticipated barriers and facilitators to patient and caregiver participation. (DOCX 67 kb

Additional file 2: of Understanding optimal approaches to patient and caregiver engagement in the development of cancer practice guidelines: a mixed methods study

Telephone interview guide. This form was used to guide telephone interviews with individuals who had had previous experience contributing to PG development as patients and/or caregivers. The interview form includes questions regarding recruitment tactics, training, methods of engaging patients and caregivers in PG development, barriers and facilitators to participation, as well as questions about their personal experience of contributing to this process. (DOCX 56 kb

Opioid agonist therapy during residential treatment of opioid use disorder:cohort study on access and outcomes

OBJECTIVE: To determine access to opioid agonist therapy (OAT) for those entering residential treatment for opioid use disorder; to report on treatment outcomes for those taking OAT and those not taking OAT; and to determine the association between OAT use and residential treatment completion.DESIGN: Retrospective cohort study.SETTING: Ontario.PARTICIPANTS: Patients with opioid use disorder admitted to publicly funded residential treatment programs in the province of Ontario between January 1, 2013, and December 31, 2016.MAIN OUTCOME MEASURES: Access to OAT during residential treatment using descriptive statistics. Treatment outcomes (ie, completed the program, voluntarily left early, involuntary discharged, and other) for the entire cohort and for the OAT and non-OAT groups using descriptive statistics. Association between OAT use at admission and treatment completion (a binary outcome) using bivariate and multivariate models.RESULTS: Among an identified cohort of 1910 patients with opioid use disorder, 52.8% entered programs that permitted access to OAT. Overall, 56.8% of patients completed treatment, 23.3% voluntarily left early (eg, were no-shows, dropped out), 17.0% were involuntarily discharged, and 2.9% were discharged early for other reasons. Those taking OAT were as likely to complete treatment as those not taking OAT (53.9% vs 57.5%, respectively; adjusted odds ratio of 1.07, 95% CI 0.77 to 1.38).CONCLUSION: This study demonstrates 2 large gaps in care for patients with opioid use disorder. First, these patients have poor access to OAT-the first-line treatment of opioid use disorder-while in publicly funded residential treatment programs; and second, many are involuntarily discharged from treatment. Additionally, this study indicates that patients taking OAT have similar likelihood of completing residential treatment as those not taking OAT do. Limitations of this study are that it is based on observational data for patients who self-selected before admission to use OAT or not, and it is likely not all confounders were accounted for.</p

Additional file 2 of Validity and usability testing of a health systems guidance appraisal tool, the AGREE-HS

AGREE-HS item information. (DOCX 23 kb

Additional file 1 of Validity and usability testing of a health systems guidance appraisal tool, the AGREE-HS

AGREE-HS Reporting Checklist. (DOCX 683 kb