The development and application of audit criteria for assessing knowledge exchange plans in health research grant applications.

Background: Research funders expect evidence of end user engagement and impact plans in research proposals. Drawing upon existing frameworks, we developed audit criteria to help researchers and their institutions assess the knowledge exchange plans of health research proposals. Findings: Criteria clustered around five themes: problem definition; involvement of research users; public and patient engagement; dissemination and implementation; and planning, management and evaluation of knowledge exchange. We applied these to a sample of grant applications from one research institution in the United Kingdom to demonstrate feasibility. Conclusion: Our criteria may be useful as a tool for researcher self-assessment and for research institutions to assess the quality of knowledge exchange plans and identify areas for systematic improvement

Does the world need a scientific society for research on how to improve healthcare?

In this editorial, we reflect on the arguments for starting a scientific society focused on research on how to improve healthcare. This society would take an inclusive approach to what constitutes healthcare. For instance, it should include mental health healthcare, treatment for substance abuse, the work of allied health professions, and preventive healthcare. The society would be open to researchers from all traditions. Thus, we take an inclusive approach to what constitutes scientific research, as long as it uses rigorous methods, is focused on improving healthcare, and aims at knowledge that can be transferred across settings. The society would primarily target scientific researchers but would invite others with an interest in this area of research, regardless of their discipline, position, field of application, or group affiliation (e.g., improvement science, behavioral medicine, knowledge translation). A society would need fruitful collaboration with related societies and organizations, which may include having combined meetings. Special links may be developed with one or more journals. A website to provide information on relevant resources, events, and training opportunities is another key activity. It would also provide a voice for the field at funding agencies, political arenas, and similar institutions. An organizational structure and financial resources are required to develop and run these activities. Our aim is to start an international debate, to discover if we can establish a shared vision across academics and stakeholders engaged with creating scientific knowledge on how to improve healthcare. We invite readers to express their views in the online questionnaire accessed by following the URL link provided at the end of the editorial

Looking inside the black box : a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada

Background: Randomised controlled trials of implementation strategies tell us whether (or not) an intervention results in changes in professional behaviour but little about the causal mechanisms that produce any change. Theory-based process evaluations collect data on theoretical constructs alongside randomised trials to explore possible causal mechanisms and effect modifiers. This is similar to measuring intermediate endpoints in clinical trials to further understand the biological basis of any observed effects (for example, measuring lipid profiles alongside trials of lipid lowering drugs where the primary endpoint could be reduction in vascular related deaths). This study protocol describes a theory-based process evaluation alongside the Ontario Printed Educational Message (OPEM) trial. We hypothesize that the OPEM interventions are most likely to operate through changes in physicians' behavioural intentions due to improved attitudes or subjective norms with little or no change in perceived behavioural control. We will test this hypothesis using a well-validated social cognition model, the theory of planned behaviour (TPB) that incorporates these constructs. Methods/design: We will develop theory-based surveys using standard methods based upon the TPB for the second and third replications, and survey a subsample of Ontario family physicians from each arm of the trial two months before and six months after the dissemination of the index edition of informed, the evidence based newsletter used for the interventions. In the third replication, our study will converge with the "TRY-ME" protocol (a second study conducted alongside the OPEM trial), in which the content of educational messages was constructed using both standard methods and methods informed by psychological theory. We will modify Dillman's total design method to maximise response rates. Preliminary analyses will initially assess the internal reliability of the measures and use regression to explore the relationships between predictor and dependent variable (intention to advise diabetic patients to have annual retinopathy screening and to prescribe thiazide diuretics for first line treatment of uncomplicated hypertension). We will then compare groups using methods appropriate for comparing independent samples to determine whether there have been changes in the predicted constructs (attitudes, subjective norms, or intentions) across the study groups as hypothesised, and will assess the convergence between the process evaluation results and the main trial results.The OPEM trial and OPEM process evaluation are funded by the Canadian Institute of Health Research (CIHR). The OPEM process evaluation study was developed as part of the CIHR funded interdisciplinary capacity enhancement team KT-ICEBeRG. Gaston Godin, Jeremy Grimshaw and France Légaré hold Canada Research Chairs. Louise Lemyre holds an R.S. McLaughlin Research Chair

Understanding factors associated with the translation of cardiovascular research: A multinational case study approach

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.This article has been made available through the Brunel Open Access Publishing Fund.Background: Funders of health research increasingly seek to understand how best to allocate resources in order to achieve maximum value from their funding. We built an international consortium and developed a multinational case study approach to assess benefits arising from health research. We used that to facilitate analysis of factors in the production of research that might be associated with translating research findings into wider impacts, and the complexities involved. Methods: We built on the Payback Framework and expanded its application through conducting co-ordinated case studies on the payback from cardiovascular and stroke research in Australia, Canada and the United Kingdom. We selected a stratified random sample of projects from leading medical research funders. We devised a series of innovative steps to: minimize the effect of researcher bias; rate the level of impacts identified in the case studies; and interrogate case study narratives to identify factors that correlated with achieving high or low levels of impact. Results: Twenty-nine detailed case studies produced many and diverse impacts. Over the 15 to 20 years examined, basic biomedical research has a greater impact than clinical research in terms of academic impacts such as knowledge production and research capacity building. Clinical research has greater levels of wider impact on health policies, practice, and generating health gains. There was no correlation between knowledge production and wider impacts. We identified various factors associated with high impact. Interaction between researchers and practitioners and the public is associated with achieving high academic impact and translation into wider impacts, as is basic research conducted with a clinical focus. Strategic thinking by clinical researchers, in terms of thinking through pathways by which research could potentially be translated into practice, is associated with high wider impact. Finally, we identified the complexity of factors behind research translation that can arise in a single case. Conclusions: We can systematically assess research impacts and use the findings to promote translation. Research funders can justify funding research of diverse types, but they should not assume academic impacts are proxies for wider impacts. They should encourage researchers to consider pathways towards impact and engage potential research users in research processes. © 2014 Wooding et al.; licensee BioMed Central Ltd.RAND Europe and HERG, with subsequent funding from the NHFA, the HSFC and the CIHR. This research was also partially supported by the Policy Research Programme in the English Department of Health

Looking inside the black box: results of a theory-based process evaluation exploring the results of a randomized controlled trial of printed educational messages to increase primary care physicians' diabetic retinopathy referrals [Trial registration number ISRCTN72772651]

Background: Theory-based process evaluations conducted alongside randomized controlled trials provide the opportunity to investigate hypothesized mechanisms of action of interventions, helping to build a cumulative knowledge base and to inform the interpretation of individual trial outcomes. Our objective was to identify the underlying causal mechanisms in a cluster randomized trial of the effectiveness of printed educational materials (PEMs) to increase referral for diabetic retinopathy screening. We hypothesized that the PEMs would increase physicians’ intention to refer patients for retinal screening by strengthening their attitude and subjective norm, but not their perceived behavioral control. Methods: Design: A theory based process evaluation alongside the Ontario Printed Educational Material (OPEM) cluster randomized trial. Postal surveys based on the Theory of Planned Behavior were sent to a random sample of trial participants two months before and six months after they received the intervention. Setting: Family physicians in Ontario, Canada. Participants: 1,512 family physicians (252 per intervention group) from the OPEM trial were invited to participate, and 31.3% (473/1512) responded at time one and time two. The final sample comprised 437 family physicians fully completing questionnaires at both time points. Main outcome measures: Primary: behavioral intention related to referring patient for retinopathy screening; secondary: attitude, subjective norm, perceived behavioral control. Results: At baseline, family physicians reported positive intention, attitude, subjective norm, and perceived behavioral control to advise patients about retinopathy screening suggesting limited opportunities for improvement in these constructs. There were no significant differences on intention, attitude, subjective norm, and perceived behavioral control following the intervention. Respondents also reported additional physician- and patient-related factors perceived to influence whether patients received retinopathy screening. Conclusions: Lack of change in the primary and secondary theory-based outcomes provides an explanation for the lack of observed effect of the main OPEM trial. High baseline levels of intention to advise patients to attend retinopathy screening suggest that post-intentional and other factors may explain gaps in care. Process evaluations based on behavioral theory can provide replicable and generalizable insights to aid interpretation of randomized controlled trials of complex interventions to change health professional behavior

An exploration of how guideline developer capacity and guideline implementability influence implementation and adoption: study protocol

<p>Abstract</p> <p>Background</p> <p>Practice guidelines can improve health care delivery and outcomes but several issues challenge guideline adoption, including their intrinsic attributes, and whether and how they are implemented. It appears that guideline format may influence accessibility and ease of use, which may overcome attitudinal barriers of guideline adoption, and appear to be important to all stakeholders. Guideline content may facilitate various forms of decision making about guideline adoption relevant to different stakeholders. Knowledge and attitudes about, and incentives and capacity for implementation on the part of guideline sponsors may influence whether and how they develop guidelines containing these features, and undertake implementation. Examination of these issues may yield opportunities to improve guideline adoption.</p> <p>Methods</p> <p>The attributes hypothesized to facilitate adoption will be expanded by thematic analysis, and quantitative and qualitative summary of the content of international guidelines for two primary care (diabetes, hypertension) and institutional care (chronic ulcer, chronic heart failure) topics. Factors that influence whether and how guidelines are implemented will be explored by qualitative analysis of interviews with individuals affiliated with guideline sponsoring agencies.</p> <p>Discussion</p> <p>Previous research examined guideline implementation by measuring rates of compliance with recommendations or associated outcomes, but this produced little insight on how the products themselves, or their implementation, could be improved. This research will establish a theoretical basis upon which to conduct experimental studies to compare the cost-effectiveness of interventions that enhance guideline development and implementation capacity. Such studies could first examine short-term outcomes predictive of guideline utilization, such as recall, attitude toward, confidence in, and adoption intention. If successful, then long-term objective outcomes reflecting the adoption of processes and associated patient care outcomes could be evaluated.</p

Knowledge brokering: Exploring the process of transferring knowledge into action

There are many theories about knowledge transfer but there are few clear descriptions of knowledge transfer interventions or the processes they involve. This failure to characterise structure and process in proposed KT interventions is a major barrier to the design and implementation of evaluations of particular KT strategies. This study is designed to provide a detailed description of the processes involved in a knowledge transfer intervention and to develop and refine a useful model of the knowledge transfer process

Does dissemination extend beyond publication: a survey of a cross section of public funded research in the UK

Background: In the UK, most funding bodies now expect a commitment or effort on the part of grant holders to disseminate the findings of their research. The emphasis is on ensuring that publicly funded research is made available, can be used to support decision making, and ultimately improve the quality and delivery of healthcare provided. In this study, we aimed to describe the dissemination practices and impacts of applied and public health researchers working across the UK.Methods: We conducted a survey of 485 UK-based principal investigators of publicly funded applied and public health research. Participants were contacted by email and invited to complete an online questionnaire via an embedded URL. Gift vouchers were given to all participants who completed the questionnaire. Four reminder emails were sent out to non-respondents at one, two, three, and four weeks; a fifth postal reminder was also undertaken.Results: A total of 243/485 (50%) questionnaires were returned (232 completed, 11 declining to participate). Most researchers recognise the importance of and appear committed to research dissemination. However, most dissemination activity beyond the publishing of academic papers appears to be undertaken an ad hoc fashion. There is some evidence that access to dissemination advice and support may facilitate more policy interactions; though access to such resources is lacking at an institutional level, and advice from funders can be variable. Although a minority of respondents routinely record details about the impact of their research, when asked about impact in relation to specific research projects most were able to provide simple narrative descriptions.Conclusions: Researchers recognise the importance of and appear committed to disseminating the findings of their work. Although researchers are focussed on academic publication, a range of dissemination activities are being applied albeit in an ad hoc fashion. However, what constitutes effective dissemination (in terms of impact and return on investment) remains unclear. Researchers need greater and clearer guidance on how best to plan, resource, and facilitate their dissemination activities

A theory-based process evaluation alongside a randomised controlled trial of printed educational messages to increase primary care physicians' prescription of thiazide diuretics for hypertension [ISRCTN72772651]

Background Pragmatic trials of implementation interventions focus on evaluating whether an intervention changes professional behaviour under real-world conditions rather than investigating the mechanism through which change occurs. Theory-based process evaluations conducted alongside pragmatic randomised trials address this by assessing whether the intervention changes theoretical constructs proposed to mediate change. The Ontario Printed Educational Materials (PEM) cluster trial was designed to increase family physicians’ guideline-recommended prescription of thiazide diuretics. The trial found no intervention effect. Using the theory of planned behaviour (TPB), we hypothesised that changes in thiazide prescribing would be reflected in changes in intention, consistent with changes in attitude and subjective norm, with no change to their perceived behavioural control (PBC), and tested this alongside the RCT. Methods We developed and sent TPB postal questionnaires to a random sub-sample of family physicians in each trial arm 2 months before and 6 months after dissemination of the PEMs. We used analysis of covariance to test for group differences using a 2 × 3 factorial design. We content-analysed an open-ended question about perceived barriers to thiazide prescription. Using control group data, we tested whether baseline measures of TPB constructs predicted self-reported thiazide prescribing at follow-up. Results Four hundred twenty-six physicians completed pre- and post-intervention questionnaires. Baseline scores on measures of TPB constructs were high: intention mean = 5.9 out of 7 (SD = 1.4), attitude mean = 5.8 (SD = 1.1), subjective norm mean = 5.8 (SD = 1.1) and PBC mean = 6.2 (SD = 1.0). The arms did not significantly differ post-intervention on any of the theory-based constructs, suggesting a possible ceiling effect. Content analysis of perceived barriers suggested post-intentional barriers to prescribing thiazides most often focused on specific patient clinical characteristics and potential side effects. Baseline intention (β = 0.63, p < 0.01) but not PBC (β = 0.04, p = 0.78) predicted 42.6 % of the variance in self-reported behaviour at follow-up in the control group. Conclusions Congruent with the Ontario Printed Educational Messages trial results and aligned with the TPB, we saw no impact of the intervention on any TPB constructs. The theoretical basis of this evaluation suggests possible explanations for the failure of the PEM intervention to change professional behaviour, which can directly inform the design and content of future theory-based PEM interventions to change professional behaviour

Who needs what from a national health research system: Lessons from reforms to the English Department of Health's R&D system

This article has been made available through the Brunel Open Access Publishing Fund.Health research systems consist of diverse groups who have some role in health research, but the boundaries around such a system are not clear-cut. To explore what various stakeholders need we reviewed the literature including that on the history of English health R&D reforms, and we also applied some relevant conceptual frameworks. We first describe the needs and capabilities of the main groups of stakeholders in health research systems, and explain key features of policymaking systems within which these stakeholders operate in the UK. The five groups are policymakers (and health care managers), health professionals, patients and the general public, industry, and researchers. As individuals and as organisations they have a range of needs from the health research system, but should also develop specific capabilities in order to contribute effectively to the system and benefit from it. Second, we discuss key phases of reform in the development of the English health research system over four decades - especially that of the English Department of Health's R&D system - and identify how far legitimate demands of key stakeholder interests were addressed. Third, in drawing lessons we highlight points emerging from contemporary reports, but also attempt to identify issues through application of relevant conceptual frameworks. The main lessons are: the importance of comprehensively addressing the diverse needs of various interacting institutions and stakeholders; the desirability of developing facilitating mechanisms at interfaces between the health research system and its various stakeholders; and the importance of additional money in being able to expand the scope of the health research system whilst maintaining support for basic science. We conclude that the latest health R&D strategy in England builds on recent progress and tackles acknowledged weaknesses. The strategy goes a considerable way to identifying and more effectively meeting the needs of key groups such as medical academics, patients and industry, and has been remarkably successful in increasing the funding for health research. There are still areas that might benefit from further recognition and resourcing, but the lessons identified, and progress made by the reforms are relevant for the design and coordination of national health research systems beyond England.This article is available through the Brunel Open Access Publishing Fund