Looking inside the black box : a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada

Background: Randomised controlled trials of implementation strategies tell us whether (or not) an intervention results in changes in professional behaviour but little about the causal mechanisms that produce any change. Theory-based process evaluations collect data on theoretical constructs alongside randomised trials to explore possible causal mechanisms and effect modifiers. This is similar to measuring intermediate endpoints in clinical trials to further understand the biological basis of any observed effects (for example, measuring lipid profiles alongside trials of lipid lowering drugs where the primary endpoint could be reduction in vascular related deaths). This study protocol describes a theory-based process evaluation alongside the Ontario Printed Educational Message (OPEM) trial. We hypothesize that the OPEM interventions are most likely to operate through changes in physicians' behavioural intentions due to improved attitudes or subjective norms with little or no change in perceived behavioural control. We will test this hypothesis using a well-validated social cognition model, the theory of planned behaviour (TPB) that incorporates these constructs. Methods/design: We will develop theory-based surveys using standard methods based upon the TPB for the second and third replications, and survey a subsample of Ontario family physicians from each arm of the trial two months before and six months after the dissemination of the index edition of informed, the evidence based newsletter used for the interventions. In the third replication, our study will converge with the "TRY-ME" protocol (a second study conducted alongside the OPEM trial), in which the content of educational messages was constructed using both standard methods and methods informed by psychological theory. We will modify Dillman's total design method to maximise response rates. Preliminary analyses will initially assess the internal reliability of the measures and use regression to explore the relationships between predictor and dependent variable (intention to advise diabetic patients to have annual retinopathy screening and to prescribe thiazide diuretics for first line treatment of uncomplicated hypertension). We will then compare groups using methods appropriate for comparing independent samples to determine whether there have been changes in the predicted constructs (attitudes, subjective norms, or intentions) across the study groups as hypothesised, and will assess the convergence between the process evaluation results and the main trial results.The OPEM trial and OPEM process evaluation are funded by the Canadian Institute of Health Research (CIHR). The OPEM process evaluation study was developed as part of the CIHR funded interdisciplinary capacity enhancement team KT-ICEBeRG. Gaston Godin, Jeremy Grimshaw and France Légaré hold Canada Research Chairs. Louise Lemyre holds an R.S. McLaughlin Research Chair

Breast cancer risk reduction:is it feasible to initiate a randomised controlled trial of a lifestyle intervention programme (ActWell) within a national breast screening programme?

BackgroundBreast cancer is the most commonly diagnosed cancer and the second cause of cancer deaths amongst women in the UK. The incidence of the disease is increasing and is highest in women from least deprived areas. It is estimated that around 42% of the disease in post-menopausal women could be prevented by increased physical activity and reductions in alcohol intake and body fatness. Breast cancer control endeavours focus on national screening programmes but these do not include communications or interventions for risk reductionThis study aimed to assess the feasibility of delivery, indicative effects and acceptability of a lifestyle intervention programme initiated within the NHS Scottish Breast Screening Programme (NHSSBSP).MethodsA 1:1 randomised controlled trial (RCT) of the 3 month ActWell programme (focussing on body weight, physical activity and alcohol) versus usual care conducted in two NHSSBSP sites between June 2013 and January 2014. Feasibility assessments included recruitment, retention, and fidelity to protocol. Indicative outcomes were measured at baseline and 3 month follow-up (body weight, waist circumference, eating and alcohol habits and physical activity. At study end, a questionnaire assessed participant satisfaction and qualitative interviews elicited women¿s, coaches and radiographers¿ experiences. Statistical analysis used Chi squared tests for comparisons in proportions and paired t tests for comparisons of means. Linear regression analyses were performed, adjusted for baseline values, with group allocation as a fixed effectResultsA pre-set recruitment target of 80 women was achieved within 12 weeks and 65 (81%) participants (29 intervention, 36 control) completed 3 month assessments. Mean age was 58¿±¿5.6 years, mean BMI was 29.2¿±¿7.0 kg/m2 and many (44%) reported a family history of breast cancer.The primary analysis (baseline body weight adjusted) showed a significant between group difference favouring the intervention group of 2.04 kg (95%CI ¿3.24 kg to ¿0.85 kg). Significant, favourable between group differences were also detected for BMI, waist circumference, physical activity and sitting time. Women rated the programme highly and 70% said they would recommend it to others.ConclusionsRecruitment, retention, indicative results and participant acceptability support the development of a definitive RCT to measure long term effects.Trial registrationThe trial was registered with Current Controlled Trials (ISRCTN56223933)

Survey of knowledge and perception on the access to evidence-based practice and clinical practice change among maternal and infant health practitioners in South East Asia

Background Evidence-based practice (EBP) can provide appropriate care for women and their babies; however implementation of EBP requires health professionals to have access to knowledge, the ability to interpret health care information and then strategies to apply care. The aim of this survey was to assess current knowledge of evidence-based practice, information seeking practices, perceptions and potential enablers and barriers to clinical practice change among maternal and infant health practitioners in South East Asia. Methods Questionnaires about IT access for health information and evidence-based practice were administered during August to December 2005 to health care professionals working at the nine hospitals participating in the South East Asia Optimising Reproductive and Child Health in Developing countries (SEA-ORCHID) project in Indonesia, Malaysia, Thailand and The Philippines. Results The survey was completed by 660 staff from six health professional groups. Overall, easy IT access for health care information was available to 46% of participants. However, over a fifth reported no IT access was available and over half of nurses and midwives never used IT health information. Evidence-based practice had been heard of by 58% but the majority did not understand the concept. The most frequent sites accessed were Google and PubMed. The Cochrane Library had been heard of by 47% of whom 51% had access although the majority did not use it or used it less than monthly. Only 27% had heard of the WHO Reproductive Health Library and 35% had been involved in a clinical practice change and were able to identify enablers and barriers to change. Only a third of participants had been actively involved in practice change with wide variation between the countries. Willingness to participate in professional development workshops on evidence-based practice was high. Conclusion This survey has identified the need to improve IT access to health care information and health professionals' knowledge of evidence-based health care to assist in employing evidence base practice effectively.Ruth Martis, Jacqueline J Ho and Caroline A Crowther for The SEA-ORCHID Study Grou

A theory-based process evaluation alongside a randomised controlled trial of printed educational messages to increase primary care physicians' prescription of thiazide diuretics for hypertension [ISRCTN72772651]

Background Pragmatic trials of implementation interventions focus on evaluating whether an intervention changes professional behaviour under real-world conditions rather than investigating the mechanism through which change occurs. Theory-based process evaluations conducted alongside pragmatic randomised trials address this by assessing whether the intervention changes theoretical constructs proposed to mediate change. The Ontario Printed Educational Materials (PEM) cluster trial was designed to increase family physicians’ guideline-recommended prescription of thiazide diuretics. The trial found no intervention effect. Using the theory of planned behaviour (TPB), we hypothesised that changes in thiazide prescribing would be reflected in changes in intention, consistent with changes in attitude and subjective norm, with no change to their perceived behavioural control (PBC), and tested this alongside the RCT. Methods We developed and sent TPB postal questionnaires to a random sub-sample of family physicians in each trial arm 2 months before and 6 months after dissemination of the PEMs. We used analysis of covariance to test for group differences using a 2 × 3 factorial design. We content-analysed an open-ended question about perceived barriers to thiazide prescription. Using control group data, we tested whether baseline measures of TPB constructs predicted self-reported thiazide prescribing at follow-up. Results Four hundred twenty-six physicians completed pre- and post-intervention questionnaires. Baseline scores on measures of TPB constructs were high: intention mean = 5.9 out of 7 (SD = 1.4), attitude mean = 5.8 (SD = 1.1), subjective norm mean = 5.8 (SD = 1.1) and PBC mean = 6.2 (SD = 1.0). The arms did not significantly differ post-intervention on any of the theory-based constructs, suggesting a possible ceiling effect. Content analysis of perceived barriers suggested post-intentional barriers to prescribing thiazides most often focused on specific patient clinical characteristics and potential side effects. Baseline intention (β = 0.63, p < 0.01) but not PBC (β = 0.04, p = 0.78) predicted 42.6 % of the variance in self-reported behaviour at follow-up in the control group. Conclusions Congruent with the Ontario Printed Educational Messages trial results and aligned with the TPB, we saw no impact of the intervention on any TPB constructs. The theoretical basis of this evaluation suggests possible explanations for the failure of the PEM intervention to change professional behaviour, which can directly inform the design and content of future theory-based PEM interventions to change professional behaviour

Looking inside the black box: results of a theory-based process evaluation exploring the results of a randomized controlled trial of printed educational messages to increase primary care physicians' diabetic retinopathy referrals [Trial registration number ISRCTN72772651]

Background: Theory-based process evaluations conducted alongside randomized controlled trials provide the opportunity to investigate hypothesized mechanisms of action of interventions, helping to build a cumulative knowledge base and to inform the interpretation of individual trial outcomes. Our objective was to identify the underlying causal mechanisms in a cluster randomized trial of the effectiveness of printed educational materials (PEMs) to increase referral for diabetic retinopathy screening. We hypothesized that the PEMs would increase physicians’ intention to refer patients for retinal screening by strengthening their attitude and subjective norm, but not their perceived behavioral control. Methods: Design: A theory based process evaluation alongside the Ontario Printed Educational Material (OPEM) cluster randomized trial. Postal surveys based on the Theory of Planned Behavior were sent to a random sample of trial participants two months before and six months after they received the intervention. Setting: Family physicians in Ontario, Canada. Participants: 1,512 family physicians (252 per intervention group) from the OPEM trial were invited to participate, and 31.3% (473/1512) responded at time one and time two. The final sample comprised 437 family physicians fully completing questionnaires at both time points. Main outcome measures: Primary: behavioral intention related to referring patient for retinopathy screening; secondary: attitude, subjective norm, perceived behavioral control. Results: At baseline, family physicians reported positive intention, attitude, subjective norm, and perceived behavioral control to advise patients about retinopathy screening suggesting limited opportunities for improvement in these constructs. There were no significant differences on intention, attitude, subjective norm, and perceived behavioral control following the intervention. Respondents also reported additional physician- and patient-related factors perceived to influence whether patients received retinopathy screening. Conclusions: Lack of change in the primary and secondary theory-based outcomes provides an explanation for the lack of observed effect of the main OPEM trial. High baseline levels of intention to advise patients to attend retinopathy screening suggest that post-intentional and other factors may explain gaps in care. Process evaluations based on behavioral theory can provide replicable and generalizable insights to aid interpretation of randomized controlled trials of complex interventions to change health professional behavior

An exploration of the influences on under-representation of male pre-registration nursing students

Background:- Worldwide, men are under-represented in the nursing profession. In Scotland less than 10% of pre-registration nursing students are male. Reasons for this imbalance need to be understood. Objectives:- To explore the views of male pre-registration nursing students, nursing lecturers and school teachers about this imbalance. Design:- Mixed methods study using focus groups and online survey. Settings:- Focus groups in four locations across Scotland. Online survey sent to teachers across Scotland. Participants and methods:- Eight focus groups with 33 male nursing students; four focus groups with 21 university and college nursing lecturers; 46 school teachers returned the online survey. Results:- Although nursing was considered a worthwhile career with job stability and many opportunities, it was also viewed as not being a career for men. Assumptions about the profession and femininity were challenging for men and use of the term ‘male nurse’ was felt to be anomalous. In some circumstances the provision of intimate care to particular patient groups caused difficulty. Positive encouragement from others, a positive role model or knowledge of nursing from significant others could be helpful. However concerns about low earning potential and negative media publicity about the NHS could be a disincentive. Being mature and having resilience were important to cope with being a male nursing student in a mainly female workplace. Some more ‘technical’ specialties were felt to be more attractive to men. Conclusions:- Nursing is viewed as a worthwhile career choice for men, but the gendered assumptions about the feminine nature of nursing can be a deterrent

Audit and feedback and clinical practice guideline adherence: Making feedback actionable

BACKGROUND: As a strategy for improving clinical practice guideline (CPG) adherence, audit and feedback (A&F) has been found to be variably effective, yet A&F research has not investigated the impact of feedback characteristics on its effectiveness. This paper explores how high performing facilities (HPF) and low performing facilities (LPF) differ in the way they use clinical audit data for feedback purposes. METHOD: Descriptive, qualitative, cross-sectional study of a purposeful sample of six Veterans Affairs Medical Centers (VAMCs) with high and low adherence to six CPGs, as measured by external chart review audits. One-hundred and two employees involved with outpatient CPG implementation across the six facilities participated in one-hour semi-structured interviews where they discussed strategies, facilitators and barriers to implementing CPGs. Interviews were analyzed using techniques from the grounded theory method. RESULTS: High performers provided timely, individualized, non-punitive feedback to providers, whereas low performers were more variable in their timeliness and non-punitiveness and relied on more standardized, facility-level reports. The concept of actionable feedback emerged as the core category from the data, around which timeliness, individualization, non-punitiveness, and customizability can be hierarchically ordered. CONCLUSION: Facilities with a successful record of guideline adherence tend to deliver more timely, individualized and non-punitive feedback to providers about their adherence than facilities with a poor record of guideline adherence. Consistent with findings from organizational research, feedback intervention characteristics may influence the feedback's effectiveness at changing desired behaviors

Testing a TheoRY-inspired MEssage ('TRY-ME'): a sub-trial within the Ontario Printed Educational Message (OPEM) trial

<p>Abstract</p> <p>Background</p> <p>A challenge for implementation researchers is to develop principles that could generate testable hypotheses that apply across a range of clinical contexts, thus leading to generalisability of findings. Such principles may be provided by systematically developed theories. The opportunity has arisen to test some of these theoretical principles in the Ontario Printed Educational Materials (OPEM) trial by conducting a sub-trial within the existing trial structure. OPEM is a large factorial cluster-randomised trial evaluating the effects of short directive and long discursive educational messages embedded into <b><it>informed</it></b>, an evidence-based newsletter produced in Canada by the Institute for Clinical Evaluative Sciences (ICES) and mailed to all primary care physicians in Ontario. The content of educational messages in the sub-trial will be constructed using both standard methods and methods inspired by psychological theory. The aim of this study is to test the effectiveness of the TheoRY-inspired MEssage ('TRY-ME') compared with the 'standard' message in changing prescribing behaviour.</p> <p>Methods</p> <p>The OPEM trial participants randomised to receive the short directive message attached to the outside of <b><it>informed </it></b>(an 'outsert') will be sub-randomised to receive either a standard message or a message informed by the theory of planned behaviour (TPB) using a two (long insert or no insert) by three (theory-based outsert or standard outsert or no outsert) design. The messages will relate to prescription of thiazide diuretics as first line drug treatment for hypertension (described in the accompanying protocol, "The Ontario Printed Educational Materials trial"). The short messages will be developed independently by two research teams.</p> <p>The primary outcome is prescription of thiazide diuretics, measured by routinely collected data available within ICES. The study is designed to answer the question, is there any difference in guideline adherence (i.e., thiazide prescription rates) between physicians in the six groups? A process evaluation survey instrument based on the TPB will be administered pre- and post-intervention (described in the accompanying protocol, "Looking inside the black box"). The second research question concerns processes that may underlie observed differences in prescribing behaviour. We expect that effects of the messages on prescribing behaviour will be mediated through changes in physicians' cognitions.</p> <p>Trial registration number</p> <p>Current controlled trial ISRCTN72772651</p

The effect of using an interactive booklet on childhood respiratory tract infections in consultations: Study protocol for a cluster randomised controlled trial in primary care

Background: Respiratory tract infections in children result in more primary care consultations than any other acute condition, and are the most common reason for prescribing antibiotics (which are largely unnecessary). About a fifth of children consult again for the same illness episode. Providing parents with written information on respiratory tract infections may result in a reduction in re-consultation rates and antibiotic prescribing for these illnesses. Asking clinicians to provide and discuss the information during the consultation may enhance effectiveness. This paper outlines the protocol for a study designed to evaluate the use of a booklet on respiratory tract infections in children within primary care consultations. Methods/Design: This will be a cluster randomised controlled trial. General practices will be randomised to provide parents consulting because their child has an acute respiratory tract infection with either an interactive booklet, or usual care. The booklet provides information on the expected duration of their child's illness, the likely benefits of various treatment options, signs and symptoms that should prompt re-consultation, and symptomatic treatment advice. It has been designed for use within the consultation and aims to enhance communication through the use of specific prompts. Clinicians randomised to using the interactive booklet will receive online training in its use. Outcomes will be assessed via a telephone interview with the parent two weeks after first consulting. The primary outcome will be the proportion of children who re-consult for the same illness episode. Secondary outcomes include: antibiotic use, parental satisfaction and enablement, and illness costs. Consultation rates for respiratory tract infections for the subsequent year will be assessed by a review of practice notes. Discussion: Previous studies in adults and children have shown that educational interventions can result in reductions in re-consultation rates and use of antibiotics for respiratory tract infections. This will be the first study to determine whether providing parents with a booklet on respiratory tract infections in children, and discussing it with them during the consultation, reduces re-consultations and antibiotic use for the same illness without reducing satisfaction with care. Trial registration: Current Controlled Trials ISRCTN46104365 </p