Using formative evaluation in an implementation project to increase vaccination rates in high-risk veterans: QUERI Series

<p>Abstract</p> <p>Background</p> <p>Implementation of research into practice in health care systems is a challenging and often unsuccessful endeavor. The United States Department of Veterans Affairs (VA) Quality Enhancement Research Initiative (QUERI) research teams include formative evaluations (FE) in their action-oriented VA implementation projects to identify critical information about the processes of implementation that can guide adjustments to project activities, in order to better meet project goals. This article describes the development and use of FE in an action-oriented implementation research project.</p> <p>Methods</p> <p>This two-year action-oriented implementation research project was conducted at 23 VA Spinal Cord Injury (SCI) Centers, and targeted patients, staff and the system of care, such as administration and information technology. Data for FE were collected by electronic and paper surveys, semi-structured and open-ended interviews, notes during conference calls, and exchange of e-mail messages. Specific questions were developed for each intervention (designed to improve vaccination rates for influenza in veterans with spinal cord injury and disorder); informants were selected for their knowledge of interventions and their use in SCI Centers.</p> <p>Results</p> <p>Data from FE were compiled separately for each intervention to describe barriers to progress and guide adjustments to implementation activities. These data addressed the processes of implementing the interventions, problem-solving activities and the status of interventions at SCI Centers.</p> <p>Conclusion</p> <p>Formative evaluations provided the project team with a broad view of the processes of implementing multi-targeted interventions as well as the evolving status of the related best practice. Using FE was useful, although the challenges of conducting FE for non-field researchers should be addressed. Work is needed to develop methods for conducting FE across multiple sites, as well as acknowledging variations in local contexts that affect implementation of interventions.</p

Neurobehavioral consequences of chronic intrauterine opioid exposure in infants and preschool children: a systematic review and meta-analysis

&lt;b&gt;Background&lt;/b&gt;&lt;p&gt;&lt;/p&gt; It is assumed within the accumulated literature that children born of pregnant opioid dependent mothers have impaired neurobehavioral function as a consequence of chronic intrauterine opioid use.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Methods&lt;/b&gt;&lt;p&gt;&lt;/p&gt; Quantitative and systematic review of the literature on the consequences of chronic maternal opioid use during pregnancy on neurobehavioral function of children was conducted using the Meta-analysis of Observational Studies in Epidemiology (MOOSE) and the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. We searched Cinahl, EMBASE, PsychINFO and MEDLINE between the periods of January 1995 to January 2012.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Results&lt;/b&gt;&lt;p&gt;&lt;/p&gt; There were only 5 studies out of the 200 identified that quantitatively reported on neurobehavioral function of children after maternal opioid use during pregnancy. All 5 were case control studies with the number of exposed subjects within the studies ranging from 33–143 and 45–85 for the controls. This meta-analysis showed no significant impairments, at a non-conservative significance level of p &#60; 0.05, for cognitive, psychomotor or observed behavioural outcomes for chronic intra-uterine exposed infants and pre-school children compared to non-exposed infants and children. However, all domains suggested a trend to poor outcomes in infants/children of opioid using mothers. The magnitude of all possible effects was small according to Cohen’s benchmark criteria.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Conclusions&lt;/b&gt;&lt;p&gt;&lt;/p&gt; Chronic intra-uterine opioid exposed infants and pre-school children experienced no significant impairment in neurobehavioral outcomes when compared to non-exposed peers, although in all domains there was a trend to poorer outcomes. The findings of this review are limited by the small number of studies analysed, the heterogenous populations and small numbers within the individual studies. Longitudinal studies are needed to determine if any neuropsychological impairments appear after the age of 5 years and to help investigate further the role of environmental risk factors on the effect of ‘core’ phenotypes

An organizational framework and strategic implementation for system-level change to enhance research-based practice: QUERI Series

<p>Abstract</p> <p>Background</p> <p>The continuing gap between available evidence and current practice in health care reinforces the need for more effective solutions, in particular related to organizational context. Considerable advances have been made within the U.S. Veterans Health Administration (VA) in systematically implementing evidence into practice. These advances have been achieved through a system-level program focused on collaboration and partnerships among policy makers, clinicians, and researchers.</p> <p>The Quality Enhancement Research Initiative (QUERI) was created to generate research-driven initiatives that directly enhance health care quality within the VA and, simultaneously, contribute to the field of implementation science. This paradigm-shifting effort provided a natural laboratory for exploring organizational change processes. This article describes the underlying change framework and implementation strategy used to operationalize QUERI.</p> <p>Strategic approach to organizational change</p> <p>QUERI used an evidence-based organizational framework focused on three contextual elements: 1) cultural norms and values, in this case related to the role of health services researchers in evidence-based quality improvement; 2) capacity, in this case among researchers and key partners to engage in implementation research; 3) and supportive infrastructures to reinforce expectations for change and to sustain new behaviors as part of the norm. As part of a QUERI Series in <it>Implementation Science</it>, this article describes the framework's application in an innovative integration of health services research, policy, and clinical care delivery.</p> <p>Conclusion</p> <p>QUERI's experience and success provide a case study in organizational change. It demonstrates that progress requires a strategic, systems-based effort. QUERI's evidence-based initiative involved a deliberate cultural shift, requiring ongoing commitment in multiple forms and at multiple levels. VA's commitment to QUERI came in the form of visionary leadership, targeted allocation of resources, infrastructure refinements, innovative peer review and study methods, and direct involvement of key stakeholders. Stakeholders included both those providing and managing clinical care, as well as those producing relevant evidence within the health care system. The organizational framework and related implementation interventions used to achieve contextual change resulted in engaged investigators and enhanced uptake of research knowledge. QUERI's approach and progress provide working hypotheses for others pursuing similar system-wide efforts to routinely achieve evidence-based care.</p

EQUIP: Implementing chronic care principles and applying formative evaluation methods to improve care for schizophrenia: QUERI Series

<p>Abstract</p> <p>Background</p> <p>This paper presents a case study that demonstrates the evolution of a project entitled "Enhancing QUality-of-care In Psychosis" (EQUIP) that began approximately when the U.S. Department of Veterans Affairs' Quality Enhancement Research Initiative (QUERI), and implementation science were emerging. EQUIP developed methods and tools to implement chronic illness care principles in the treatment of schizophrenia, and evaluated this implementation using a small-scale controlled trial. The next iteration of the project, EQUIP-2, was further informed by implementation science and the use of QUERI tools.</p> <p>Methods</p> <p>This paper reports the background, development, results and implications of EQUIP, and also describes ongoing work in the second phase of the project (EQUIP-2). The EQUIP intervention uses implementation strategies and tools to increase the adoption and implementation of chronic illness care principles. In EQUIP-2, these strategies and tools are conceptually grounded in a stages-of-change model, and include clinical and delivery system interventions and adoption/implementation tools. Formative evaluation occurs in conjunction with the intervention, and includes developmental, progress-focused, implementation-focused, and interpretive evaluation.</p> <p>Results</p> <p>Evaluation of EQUIP provided an understanding of quality gaps <it>and </it>how to address related problems in schizophrenia. EQUIP showed that solutions to quality problems in schizophrenia differ by treatment domain and are exacerbated by a lack of awareness of evidence-based practices. EQUIP also showed that improving care requires creating resources for physicians to help them easily implement practice changes, plus intensive education as well as product champions who help physicians use these resources. Organizational changes, such as the addition of care managers and informatics systems, were shown to help physicians with identifying problems, making referrals, and monitoring follow-up. In EQUIP-2, which is currently in progress, these initial findings were used to develop a more comprehensive approach to implementing and evaluating the chronic illness care model.</p> <p>Discussion</p> <p>In QUERI, small-scale projects contribute to the development and enhancement of hands-on, action-oriented service-directed projects that are grounded in current implementation science. This project supports the concept that QUERI tools can be useful in implementing complex care models oriented toward evidence-based improvement of clinical care.</p

Measuring persistence of implementation: QUERI Series

As more quality improvement programs are implemented to achieve gains in performance, the need to evaluate their lasting effects has become increasingly evident. However, such long-term follow-up evaluations are scarce in healthcare implementation science, being largely relegated to the "need for further research" section of most project write-ups. This article explores the variety of conceptualizations of implementation sustainability, as well as behavioral and organizational factors that influence the maintenance of gains. It highlights the finer points of design considerations and draws on our own experiences with measuring sustainability, framed within the rich theoretical and empirical contributions of others. In addition, recommendations are made for designing sustainability analyses

Improving prescribing of antihypertensive and cholesterol-lowering drugs: a method for identifying and addressing barriers to change

BACKGROUND: We describe a simple approach we used to identify barriers and tailor an intervention to improve pharmacological management of hypertension and hypercholesterolaemia. We also report the results of a post hoc exercise and survey we carried out to evaluate our approach for identifying barriers and tailoring interventions. METHODS: We used structured reflection, searched for other relevant trials, surveyed general practitioners and talked with physicians during pilot testing of the intervention. The post hoc exercise was carried out as focus groups of international researchers in the field of quality improvement in health care. The post hoc survey was done by telephone interviews with physicians allocated to the experimental group of a randomised trial of our multifaceted intervention. RESULTS: A wide range of barriers was identified and several interventions were suggested through structured reflection. The survey led to some adjustments. Studying other trials and pilot testing did not lead to changes in the design of the intervention. Neither the post hoc focus groups nor the post hoc survey revealed important barriers or interventions that we had not considered or included in our tailored intervention. CONCLUSIONS: A simple approach to identifying barriers to change appears to have been adequate and efficient. However, we do not know for certain what we would have gained by using more comprehensive methods and we do not know whether the resulting intervention would have been more effective if we had used other methods. The effectiveness of our multifaceted intervention is under evaluation in a randomised controlled trial

'To take care of the patients': Qualitative analysis of Veterans Health Administration personnel experiences with a clinical informatics system

<p>Abstract</p> <p>Background</p> <p>The Veterans Health Administration (VA) has invested significant resources in designing and implementing a comprehensive electronic health record (EHR) that supports clinical priorities. EHRs in general have been difficult to implement, with unclear cost-effectiveness. We describe VA clinical personnel interactions with and evaluations of the EHR.</p> <p>Methods</p> <p>As part of an evaluation of a quality improvement initiative, we interviewed 72 VA clinicians and managers using a semi-structured interview format. We conducted a qualitative analysis of interview transcripts, examining themes relating to participants' interactions with and evaluations of the VA EHR.</p> <p>Results</p> <p>Participants described their perceptions of the positive and negative effects of the EHR on their clinical workflow. Although they appreciated the speed and ease of documentation that the EHR afforded, they were concerned about the time cost of using the technology and the technology's potential for detracting from interpersonal interactions.</p> <p>Conclusions</p> <p>VA personnel value EHRs' contributions to supporting communication, education, and documentation. However, participants are concerned about EHRs' potential interference with other important aspects of healthcare, such as time for clinical care and interpersonal communication with patients and colleagues. We propose that initial implementation of an EHR is one step in an iterative process of ongoing quality improvement.</p

Improving eye care for veterans with diabetes: An example of using the QUERI steps to move from evidence to implementation: QUERI Series

<p>Abstract</p> <p>Background</p> <p>Despite being a critical part of improving healthcare quality, little is known about how best to move important research findings into clinical practice. To address this issue, the Department of Veterans Affairs (VA) developed the Quality Enhancement Research Initiative (QUERI), which provides a framework, a supportive structure, and resources to promote the more rapid implementation of evidence into practice.</p> <p>Methods</p> <p>This paper uses a practical example to demonstrate the use of the six-step QUERI process, which was developed as part of QUERI and provides a systematic approach for moving along the research to practice pipeline. Specifically, we describe a series of projects using the six-step framework to illustrate how this process guided work by the Diabetes Mellitus QUERI (DM-QUERI) Center to assess and improve eye care for veterans with diabetes.</p> <p>Results</p> <p>Within a relatively short time, DM-QUERI identified a high-priority issue, developed evidence to support a change in the diabetes eye screening performance measure, and identified a gap in quality of care. A prototype scheduling system to address gaps in screening and follow-up also was tested as part of an implementation project. We did not succeed in developing a fully functional pro-active scheduling system. This work did, however, provide important information to help us further understand patients' risk status, gaps in follow-up at participating eye clinics, specific considerations for additional implementation work in the area of proactive scheduling, and contributed to a change in the prevailing diabetes eye care performance measure.</p> <p>Conclusion</p> <p>Work by DM-QUERI to promote changes in the delivery of eye care services for veterans with diabetes demonstrates the value of the QUERI process in facilitating the more rapid implementation of evidence into practice. However, our experience with using the QUERI process also highlights certain challenges, including those related to the hybrid nature of the research-operations partnership as a mechanism for promoting rapid, system-wide implementation of important research findings. In addition, this paper suggests a number of important considerations for future implementation work, both in the area of pro-active scheduling interventions, as well as for implementation science in general.</p

The implementation of a translational study involving a primary care based behavioral program to improve blood pressure control: The HTN-IMPROVE study protocol (01295)

<p>Abstract</p> <p>Background</p> <p>Despite the impact of hypertension and widely accepted target values for blood pressure (BP), interventions to improve BP control have had limited success.</p> <p>Objectives</p> <p>We describe the design of a 'translational' study that examines the implementation, impact, sustainability, and cost of an evidence-based nurse-delivered tailored behavioral self-management intervention to improve BP control as it moves from a research context to healthcare delivery. The study addresses four specific aims: assess the implementation of an evidence-based behavioral self-management intervention to improve BP levels; evaluate the clinical impact of the intervention as it is implemented; assess organizational factors associated with the sustainability of the intervention; and assess the cost of implementing and sustaining the intervention.</p> <p>Methods</p> <p>The project involves three geographically diverse VA intervention facilities and nine control sites. We first conduct an evaluation of barriers and facilitators for implementing the intervention at intervention sites. We examine the impact of the intervention by comparing 12-month pre/post changes in BP control between patients in intervention sites versus patients in the matched control sites. Next, we examine the sustainability of the intervention and organizational factors facilitating or hindering the sustained implementation. Finally, we examine the costs of intervention implementation. Key outcomes are acceptability and costs of the program, as well as changes in BP. Outcomes will be assessed using mixed methods (<it>e.g</it>., qualitative analyses--pattern matching; quantitative methods--linear mixed models).</p> <p>Discussion</p> <p>The study results will provide information about the challenges and costs to implement and sustain the intervention, and what clinical impact can be expected.</p

Implementing and evaluating a regional strategy to improve testing rates in VA patients at risk for HIV, utilizing the QUERI process as a guiding framework: QUERI Series

<p>Abstract</p> <p>Background</p> <p>We describe how we used the framework of the U.S. Department of Veterans Affairs (VA) Quality Enhancement Research Initiative (QUERI) to develop a program to improve rates of diagnostic testing for the Human Immunodeficiency Virus (HIV). This venture was prompted by the observation by the CDC that 25% of HIV-infected patients do not know their diagnosis – a point of substantial importance to the VA, which is the largest provider of HIV care in the United States.</p> <p>Methods</p> <p>Following the QUERI steps (or process), we evaluated: 1) whether undiagnosed HIV infection is a high-risk, high-volume clinical issue within the VA, 2) whether there are evidence-based recommendations for HIV testing, 3) whether there are gaps in the performance of VA HIV testing, and 4) the barriers and facilitators to improving current practice in the VA.</p> <p>Based on our findings, we developed and initiated a QUERI step 4/phase 1 pilot project using the precepts of the Chronic Care Model. Our improvement strategy relies upon electronic clinical reminders to provide <it>decision support</it>; audit/feedback as a <it>clinical information system</it>, and appropriate changes in <it>delivery system design</it>. These activities are complemented by academic detailing and social marketing interventions to achieve <it>provider activation</it>.</p> <p>Results</p> <p>Our preliminary formative evaluation indicates the need to ensure leadership and team buy-in, address facility-specific barriers, refine the reminder, and address factors that contribute to inter-clinic variances in HIV testing rates. Preliminary unadjusted data from the first seven months of our program show 3–5 fold increases in the proportion of at-risk patients who are offered HIV testing at the VA sites (stations) where the pilot project has been undertaken; no change was seen at control stations.</p> <p>Discussion</p> <p>This project demonstrates the early success of the application of the QUERI process to the development of a program to improve HIV testing rates. Preliminary unadjusted results show that the coordinated use of audit/feedback, provider activation, and organizational change can increase HIV testing rates for at-risk patients. We are refining our program prior to extending our work to a small-scale, multi-site evaluation (QUERI step 4/phase 2). We also plan to evaluate the durability/sustainability of the intervention effect, the costs of HIV testing, and the number of newly identified HIV-infected patients. Ultimately, we will evaluate this program in other geographically dispersed stations (QUERI step 4/phases 3 and 4).</p