What research impacts do Australian primary health care researchers expect and achieve?

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Background Funding for research is under pressure to be accountable in terms of benefits and translation of research findings into practice and policy. Primary health care research has considerable potential to improve health care in a wide range of settings, but little is known about the extent to which these impacts actually occur. This study examines the impact of individual primary health care research projects on policy and practice from the perspective of Chief Investigators (CIs). Methods The project used an online survey adapted from the Buxton and Hanney Payback Framework to collect information about the impacts that CIs expected and achieved from primary health care research projects funded by Australian national competitive grants. Results and Discussion Chief Investigators (CIs) provided information about seventeen completed projects. While no CI expected their project to have an impact in every domain of the framework used in the survey, 76% achieved at least half the impacts they expected. Sixteen projects had published and/or presented their work, 10 projects included 11 doctorate awards in their research capacity domain. All CIs expected their research to lead to further research opportunities with 11 achieving this. Ten CIs achieved their expectation of providing information for policy making but only four reported their research had influenced policy making. However 11 CIs achieved their expectation of providing information for organizational decision making and eight reported their research had influenced organizational decision making. Conclusion CIs reported that nationally funded primary health care research projects made an impact on knowledge production, staff development and further research, areas within the realm of influence of the research team and within the scope of awareness of the CIs. Some also made an impact on policy and organizational decision-making, and on localized clinical practice and service delivery. CIs reported few broader economic benefits from their research. Routine use of an instrument of this type would facilitate primary health care research funders' determination of the payback for funding of research in this sector

The feasibility of determining the impact of primary health care research projects using the Payback Framework

<p>Abstract</p> <p>Background</p> <p>Primary health care research is under pressure to be accountable to funders in terms of benefits for practice and policy. However, methods to assess the impact of primary health care research must be appropriate to use with the diverse topics, settings and approaches of this sector. This project explored the feasibility of using the Buxton and Hanney Payback Framework to determine the impact of a stratified random sample (n = 4) of competitively funded, primary health care research projects.</p> <p>Methods</p> <p>The project conducted telephone interviews based on the Payback Framework with leaders of the research teams and nominated users of their research, used bibliometric methods for assessing impact through publication outputs and obtained documentary evidence of impact where possible. The purpose was to determine the effectiveness of the data collection methods and the applicability of the Payback Framework, and any other issues which arose around the assessment of impact of primary health care research.</p> <p>Results and discussion</p> <p>The thirteen interviews were resource intensive to organise conduct and analyse but provided better information about impact than bibliometric analysis or documentary analysis. Bibliometric analysis of the papers published from the four projects was hampered by the inclusion of only one of the journals in major citation indexes. Document analysis provided more evidence of dissemination than of impact.</p> <p>The payback framework and logic model were a sound basis for assessing impact. Chief investigators and nominated users of research provided substantial information relevant to the impact categories closest to their spheres of influence and awareness, but less about the impact their research had on the wider health sector, population health or economic benefits. An additional category of impact emerged from the interviews, that of strengthening research networks which could enhance the impact of later work. The framework provided rich information about the pathways to impact, better understanding of which may enhance impact.</p> <p>Conclusion</p> <p>It is feasible to use the Buxton and Hanney Payback framework and logic model to determine the proximal impacts of primary health care research. Though resource intensive, telephone interviews of chief investigators and nominated users provided rich information.</p

The EXpert PANel Decision (EXPAND) method: a way to measure the impact of diverse quality improvement activities of clinical networks

Objectives and importance of study: Evaluating impacts of quality improvement activities across diverse clinical focus areas is challenging. However, evaluation is necessary to determine if the activities had an impact on quality of care and resulted in system-wide change. Clinical networks of health providers aim to provide a platform for accelerating quality improvement activities and adopting evidence based practices. However, most networks do not collect primary data that would enable evaluation of impact. We adapted an established expert panel approach to measure the impacts of efforts in 19 clinical networks to improve care and promote health system change, to determine whether these efforts achieved their purpose. Study type: A retrospective cross-sectional study of 19 clinical networks using multiple methods of data collection including the EXpert PANel Decision (EXPAND) method. Methods: Network impacts were identified through interviews with network managers (n = 19) and co-chairs (n = 32), and document review. The EXPAND method brought together five independent experts who provided initial individual ratings of overall network impact. After attendance at an in-person moderated meeting where aggregate scores were discussed, the experts provided a final rating. Median scores of postmeeting ratings were the final measures of network impact. Results: Among the 19 clinical networks, experts rated 47% (n = 9) as having a limited impact on improving quality of care, 37% (n = 7) as having a moderate impact and 16% (n = 3) as having a high impact. The experts rated 26% (n = 5) of clinical networks as having a limited impact on facilitating system-wide change, 37% (n = 7) as having a moderate impact and 37% (n = 7) as having a high impact. Conclusion: The EXPAND method enabled appraisal of diverse clinical networks in the absence of primary data that could directly evaluate network impacts. The EXPAND method can be applied to assess the impact of quality improvement initiatives across diverse clinical areas to inform healthcare planning, delivery and performance. Further research is needed to assess its reliability and validity