Using the theory of planned behaviour as a process evaluation tool in randomised trials of knowledge translation strategies : A case study from UK primary care

Peer reviewedPublisher PD

Looking inside the black box : a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada

Background: Randomised controlled trials of implementation strategies tell us whether (or not) an intervention results in changes in professional behaviour but little about the causal mechanisms that produce any change. Theory-based process evaluations collect data on theoretical constructs alongside randomised trials to explore possible causal mechanisms and effect modifiers. This is similar to measuring intermediate endpoints in clinical trials to further understand the biological basis of any observed effects (for example, measuring lipid profiles alongside trials of lipid lowering drugs where the primary endpoint could be reduction in vascular related deaths). This study protocol describes a theory-based process evaluation alongside the Ontario Printed Educational Message (OPEM) trial. We hypothesize that the OPEM interventions are most likely to operate through changes in physicians' behavioural intentions due to improved attitudes or subjective norms with little or no change in perceived behavioural control. We will test this hypothesis using a well-validated social cognition model, the theory of planned behaviour (TPB) that incorporates these constructs. Methods/design: We will develop theory-based surveys using standard methods based upon the TPB for the second and third replications, and survey a subsample of Ontario family physicians from each arm of the trial two months before and six months after the dissemination of the index edition of informed, the evidence based newsletter used for the interventions. In the third replication, our study will converge with the "TRY-ME" protocol (a second study conducted alongside the OPEM trial), in which the content of educational messages was constructed using both standard methods and methods informed by psychological theory. We will modify Dillman's total design method to maximise response rates. Preliminary analyses will initially assess the internal reliability of the measures and use regression to explore the relationships between predictor and dependent variable (intention to advise diabetic patients to have annual retinopathy screening and to prescribe thiazide diuretics for first line treatment of uncomplicated hypertension). We will then compare groups using methods appropriate for comparing independent samples to determine whether there have been changes in the predicted constructs (attitudes, subjective norms, or intentions) across the study groups as hypothesised, and will assess the convergence between the process evaluation results and the main trial results.The OPEM trial and OPEM process evaluation are funded by the Canadian Institute of Health Research (CIHR). The OPEM process evaluation study was developed as part of the CIHR funded interdisciplinary capacity enhancement team KT-ICEBeRG. Gaston Godin, Jeremy Grimshaw and France Légaré hold Canada Research Chairs. Louise Lemyre holds an R.S. McLaughlin Research Chair

Looking inside the black box: results of a theory-based process evaluation exploring the results of a randomized controlled trial of printed educational messages to increase primary care physicians' diabetic retinopathy referrals [Trial registration number ISRCTN72772651]

Background: Theory-based process evaluations conducted alongside randomized controlled trials provide the opportunity to investigate hypothesized mechanisms of action of interventions, helping to build a cumulative knowledge base and to inform the interpretation of individual trial outcomes. Our objective was to identify the underlying causal mechanisms in a cluster randomized trial of the effectiveness of printed educational materials (PEMs) to increase referral for diabetic retinopathy screening. We hypothesized that the PEMs would increase physicians’ intention to refer patients for retinal screening by strengthening their attitude and subjective norm, but not their perceived behavioral control. Methods: Design: A theory based process evaluation alongside the Ontario Printed Educational Material (OPEM) cluster randomized trial. Postal surveys based on the Theory of Planned Behavior were sent to a random sample of trial participants two months before and six months after they received the intervention. Setting: Family physicians in Ontario, Canada. Participants: 1,512 family physicians (252 per intervention group) from the OPEM trial were invited to participate, and 31.3% (473/1512) responded at time one and time two. The final sample comprised 437 family physicians fully completing questionnaires at both time points. Main outcome measures: Primary: behavioral intention related to referring patient for retinopathy screening; secondary: attitude, subjective norm, perceived behavioral control. Results: At baseline, family physicians reported positive intention, attitude, subjective norm, and perceived behavioral control to advise patients about retinopathy screening suggesting limited opportunities for improvement in these constructs. There were no significant differences on intention, attitude, subjective norm, and perceived behavioral control following the intervention. Respondents also reported additional physician- and patient-related factors perceived to influence whether patients received retinopathy screening. Conclusions: Lack of change in the primary and secondary theory-based outcomes provides an explanation for the lack of observed effect of the main OPEM trial. High baseline levels of intention to advise patients to attend retinopathy screening suggest that post-intentional and other factors may explain gaps in care. Process evaluations based on behavioral theory can provide replicable and generalizable insights to aid interpretation of randomized controlled trials of complex interventions to change health professional behavior

Explaining the effects of an intervention designed to promote evidence-based diabetes care : a theory-based process evaluation of a pragmatic cluster randomised controlled trial

Background The results of randomised controlled trials can be usefully illuminated by studies of the processes by which they achieve their effects. The Theory of Planned Behaviour (TPB) offers a framework for conducting such studies. This study used TPB to explore the observed effects in a pragmatic cluster randomised controlled trial of a structured recall and prompting intervention to increase evidence-based diabetes care that was conducted in three Primary Care Trusts in England. Methods All general practitioners and nurses in practices involved in the trial were sent a postal questionnaire at the end of the intervention period, based on the TPB (predictor variables: attitude; subjective norm; perceived behavioural control, or PBC). It focussed on three clinical behaviours recommended in diabetes care: measuring blood pressure; inspecting feet; and prescribing statins. Multivariate analyses of variance and multiple regression analyses were used to explore changes in cognitions and thereby better understand trial effects. Results Fifty-nine general medical practitioners and 53 practice nurses (intervention: n = 55, 41.98% of trial participants; control: n = 57, 38.26% of trial participants) completed the questionnaire. There were no differences between groups in mean scores for attitudes, subjective norms, PBC or intentions. Control group clinicians had 'normatively-driven' intentions (i.e., related to subjective norm scores), whereas intervention group clinicians had 'attitudinally-driven' intentions (i.e., related to attitude scores) for foot inspection and statin prescription. After controlling for effects of the three predictor variables, this group difference was significant for foot inspection behaviour (trial group × attitude interaction, beta = 0.72, p < 0.05; trial group × subjective norm interaction, beta = -0.65, p < 0.05). Conclusion Attitudinally-driven intentions are proposed to be more consistently translated into action than normatively-driven intentions. This proposition was supported by the findings, thus offering an interpretation of the trial effects. This analytic approach demonstrates the potential of the TPB to explain trial effects in terms of different relationships between variables rather than differences in mean scores. This study illustrates the use of theory-based process evaluation to uncover processes underlying change in implementation trials.European Union ReBEQI projec

Effective Feedback to Improve Primary Care Prescribing Safety (EFIPPS) a pragmatic three-arm cluster randomised trial:designing the intervention (ClinicalTrials.gov registration NCT01602705)

Peer reviewedPublisher PD

Theories of behaviour change synthesised into a set of theoretical groupings: Introducing a thematic series on the Theoretical Domains Framework

Behaviour change is key to increasing the uptake of evidence into healthcare practice. Designing behaviour-change interventions first requires problem analysis, ideally informed by theory. Yet the large number of partly overlapping theories of behaviour makes it difficult to select the most appropriate theory. The need for an overarching theoretical framework of behaviour change was addressed in research in which 128 explanatory constructs from 33 theories of behaviour were identified and grouped. The resulting Theoretical Domains Framework (TDF) appears to be a helpful basis for investigating implementation problems. Research groups in several countries have conducted TDF-based studies. It seems timely to bring together the experience of these teams in a thematic series to demonstrate further applications and to report key developments. This overview article describes the TDF, provides a brief critique of the framework, and introduces this thematic series. In a brief review to assess the extent of TDF-based research, we identified 133 papers that cite the framework. Of these, 17 used the TDF as the basis for empirical studies to explore health professionals’ behaviour. The identified papers provide evidence of the impact of the TDF on implementation research. Two major strengths of the framework are its theoretical coverage and its capacity to elicit beliefs that could signify key mediators of behaviour change. The TDF provides a useful conceptual basis for assessing implementation problems, designing interventions to enhance healthcare practice, and understanding behaviour-change processes. We discuss limitations and research challenges and introduce papers in this series

Testing a TheoRY-inspired MEssage ('TRY-ME'): a sub-trial within the Ontario Printed Educational Message (OPEM) trial

<p>Abstract</p> <p>Background</p> <p>A challenge for implementation researchers is to develop principles that could generate testable hypotheses that apply across a range of clinical contexts, thus leading to generalisability of findings. Such principles may be provided by systematically developed theories. The opportunity has arisen to test some of these theoretical principles in the Ontario Printed Educational Materials (OPEM) trial by conducting a sub-trial within the existing trial structure. OPEM is a large factorial cluster-randomised trial evaluating the effects of short directive and long discursive educational messages embedded into <b><it>informed</it></b>, an evidence-based newsletter produced in Canada by the Institute for Clinical Evaluative Sciences (ICES) and mailed to all primary care physicians in Ontario. The content of educational messages in the sub-trial will be constructed using both standard methods and methods inspired by psychological theory. The aim of this study is to test the effectiveness of the TheoRY-inspired MEssage ('TRY-ME') compared with the 'standard' message in changing prescribing behaviour.</p> <p>Methods</p> <p>The OPEM trial participants randomised to receive the short directive message attached to the outside of <b><it>informed </it></b>(an 'outsert') will be sub-randomised to receive either a standard message or a message informed by the theory of planned behaviour (TPB) using a two (long insert or no insert) by three (theory-based outsert or standard outsert or no outsert) design. The messages will relate to prescription of thiazide diuretics as first line drug treatment for hypertension (described in the accompanying protocol, "The Ontario Printed Educational Materials trial"). The short messages will be developed independently by two research teams.</p> <p>The primary outcome is prescription of thiazide diuretics, measured by routinely collected data available within ICES. The study is designed to answer the question, is there any difference in guideline adherence (i.e., thiazide prescription rates) between physicians in the six groups? A process evaluation survey instrument based on the TPB will be administered pre- and post-intervention (described in the accompanying protocol, "Looking inside the black box"). The second research question concerns processes that may underlie observed differences in prescribing behaviour. We expect that effects of the messages on prescribing behaviour will be mediated through changes in physicians' cognitions.</p> <p>Trial registration number</p> <p>Current controlled trial ISRCTN72772651</p

A survey of factors associated with the successful recognition of agonal breathing and cardiac arrest by 9-1-1 call takers: design and methodology

<p>Abstract</p> <p>Background</p> <p>Cardiac arrest victims most often collapse at home, where only a modest proportion receives life-saving bystander cardiopulmonary resuscitation. As many as 40% of all sudden cardiac arrest victims have agonal or abnormal breathing in the first minutes following cardiac arrest. 9-1-1 call takers may wrongly interpret agonal breathing as a sign of life, and not initiate telephone cardiopulmonary resuscitation instructions. Improving 9-1-1 call takers' ability to recognize agonal breathing as a sign of cardiac arrest could result in improved bystander cardiopulmonary resuscitation and survival rates for out-of-hospital cardiac arrest victims.</p> <p>Methods/Design</p> <p>The overall goal of this study is to design and conduct a survey of 9-1-1 call takers in the province of Ontario to better understand the factors associated with the successful identification of cardiac arrest (including patients with agonal breathing) over the phone, and subsequent administration of cardiopulmonary resuscitation instructions to callers. This study will be conducted in three phases using the Theory of Planned Behaviour. In Phase One, we will conduct semi-structured qualitative interviews with a purposeful selection of 9-1-1 call takers from Ontario, and identify common themes and belief categories. In Phase Two, we will use the qualitative interview results to design and pilot a quantitative survey. In Phase Three, a final version of the quantitative survey will be administered via an electronic medium to all registered call takers in the province of Ontario. We will perform qualitative thematic analysis (Phase One) and regression modelling (Phases Two and Three), to determine direct and indirect relationship of behavioural constructs with intentions to provide cardiopulmonary resuscitation instructions.</p> <p>Discussion</p> <p>The results of this study will provide valuable insight into the factors associated with the successful recognition of agonal breathing and cardiac arrest by 9-1-1 call takers. This will guide future interventional studies, which may include continuing education and protocol changes, in order to help increase the number of callers appropriately receiving cardiopulmonary resuscitation instructions, and save the lives of more cardiac arrest victims.</p> <p>Trial registration</p> <p>Clinicaltrials.gov NCT00848588</p

Reliability of a tool for measuring theory of planned behaviour constructs for use in evaluating research use in policymaking

<p>Abstract</p> <p>Background</p> <p>Although measures of knowledge translation and exchange (KTE) effectiveness based on the theory of planned behavior (TPB) have been used among patients and providers, no measure has been developed for use among health system policymakers and stakeholders. A tool that measures the intention to use research evidence in policymaking could assist researchers in evaluating the effectiveness of KTE strategies that aim to support evidence-informed health system decision-making. Therefore, we developed a 15-item tool to measure four TPB constructs (intention, attitude, subjective norm and perceived control) and assessed its face validity through key informant interviews.</p> <p>Methods</p> <p>We carried out a reliability study to assess the tool's internal consistency and test-retest reliability. Our study sample consisted of 62 policymakers and stakeholders that participated in deliberative dialogues. We assessed internal consistency using Cronbach's alpha and generalizability (G) coefficients, and we assessed test-retest reliability by calculating Pearson correlation coefficients (<it>r</it>) and G coefficients for each construct and the tool overall.</p> <p>Results</p> <p>The internal consistency of items within each construct was good with alpha ranging from 0.68 to alpha = 0.89. G-coefficients were lower for a single administration (G = 0.34 to G = 0.73) than for the average of two administrations (G = 0.79 to G = 0.89). Test-retest reliability coefficients for the constructs ranged from <it>r </it>= 0.26 to <it>r </it>= 0.77 and from G = 0.31 to G = 0.62 for a single administration, and from G = 0.47 to G = 0.86 for the average of two administrations. Test-retest reliability of the tool using G theory was moderate (G = 0.5) when we generalized across a single observation, but became strong (G = 0.9) when we averaged across both administrations.</p> <p>Conclusion</p> <p>This study provides preliminary evidence for the reliability of a tool that can be used to measure TPB constructs in relation to research use in policymaking. Our findings suggest that the tool should be administered on more than one occasion when the intervention promotes an initial 'spike' in enthusiasm for using research evidence (as it seemed to do in this case with deliberative dialogues). The findings from this study will be used to modify the tool and inform further psychometric testing following different KTE interventions.</p

A theory-based process evaluation alongside a randomised controlled trial of printed educational messages to increase primary care physicians' prescription of thiazide diuretics for hypertension [ISRCTN72772651]

Background Pragmatic trials of implementation interventions focus on evaluating whether an intervention changes professional behaviour under real-world conditions rather than investigating the mechanism through which change occurs. Theory-based process evaluations conducted alongside pragmatic randomised trials address this by assessing whether the intervention changes theoretical constructs proposed to mediate change. The Ontario Printed Educational Materials (PEM) cluster trial was designed to increase family physicians’ guideline-recommended prescription of thiazide diuretics. The trial found no intervention effect. Using the theory of planned behaviour (TPB), we hypothesised that changes in thiazide prescribing would be reflected in changes in intention, consistent with changes in attitude and subjective norm, with no change to their perceived behavioural control (PBC), and tested this alongside the RCT. Methods We developed and sent TPB postal questionnaires to a random sub-sample of family physicians in each trial arm 2 months before and 6 months after dissemination of the PEMs. We used analysis of covariance to test for group differences using a 2 × 3 factorial design. We content-analysed an open-ended question about perceived barriers to thiazide prescription. Using control group data, we tested whether baseline measures of TPB constructs predicted self-reported thiazide prescribing at follow-up. Results Four hundred twenty-six physicians completed pre- and post-intervention questionnaires. Baseline scores on measures of TPB constructs were high: intention mean = 5.9 out of 7 (SD = 1.4), attitude mean = 5.8 (SD = 1.1), subjective norm mean = 5.8 (SD = 1.1) and PBC mean = 6.2 (SD = 1.0). The arms did not significantly differ post-intervention on any of the theory-based constructs, suggesting a possible ceiling effect. Content analysis of perceived barriers suggested post-intentional barriers to prescribing thiazides most often focused on specific patient clinical characteristics and potential side effects. Baseline intention (β = 0.63, p < 0.01) but not PBC (β = 0.04, p = 0.78) predicted 42.6 % of the variance in self-reported behaviour at follow-up in the control group. Conclusions Congruent with the Ontario Printed Educational Messages trial results and aligned with the TPB, we saw no impact of the intervention on any TPB constructs. The theoretical basis of this evaluation suggests possible explanations for the failure of the PEM intervention to change professional behaviour, which can directly inform the design and content of future theory-based PEM interventions to change professional behaviour