Refining Expert Recommendations for Implementing Change (ERIC) strategy surveys using cognitive interviews with frontline providers

BACKGROUND: The Expert Recommendations for Implementing Change (ERIC) compilation includes 73 defined implementation strategies clustered into nine content areas. This taxonomy has been used to track implementation strategies over time using surveys. This study aimed to improve the ERIC survey using cognitive interviews with non-implementation scientist clinicians. METHODS: Starting in 2015, we developed and fielded annual ERIC surveys to evaluate liver care in the Veterans Health Administration (VA). We invited providers who had completed at least three surveys to participate in cognitive interviews (October 2020 to October 2021). Before the interviews, participants reviewed the complete 73-item ERIC survey and marked which strategies were unclear due to wording, conceptual confusion, or overlap with other strategies. They then engaged in semi-structured cognitive interviews to describe the experience of completing the survey and elaborate on which strategies required further clarification. RESULTS: Twelve VA providers completed surveys followed by cognitive interviews. The Engage Consumer and Support Clinicians clusters were rated most highly in terms of conceptual and wording clarity. In contrast, the Financial cluster had the most wording and conceptual confusion. The Adapt and Tailor to Context cluster strategies were considered to have the most redundancy. Providers outlined ways in which the strategies could be clearer in terms of wording (32%), conceptual clarity (51%), and clarifying the distinction between strategies (51%). CONCLUSIONS: Cognitive interviews with ERIC survey participants allowed us to identify and address issues with strategy wording, combine conceptually indistinct strategies, and disaggregate multi-barreled strategies. Improvements made to the ERIC survey based on these findings will ultimately assist VA and other institutions in designing, evaluating, and replicating quality improvement efforts

Getting to implementation: Adaptation of an implementation playbook

IntroductionImplementation strategies supporting the translation of evidence into practice need to be tailored and adapted for maximum effectiveness, yet the field of adapting implementation strategies remains nascent. We aimed to adapt “Getting To Outcomes”® (GTO), a 10-step implementation playbook designed to help community-based organizations plan and evaluate behavioral health programs, into “Getting To Implementation” (GTI) to support the selection, tailoring, and use of implementation strategies in health care settings.MethodsOur embedded evaluation team partnered with operations, external facilitators, and site implementers to employ participatory methods to co-design and adapt GTO for Veterans Health Administration (VA) outpatient cirrhosis care improvement. The Framework for Reporting Adaptations and Modifications to Evidenced-based Implementation Strategies (FRAME-IS) guided documentation and analysis of changes made pre- and post-implementation of GTI at 12 VA medical centers. Data from multiple sources (interviews, observation, content analysis, and fidelity tracking) were triangulated and analyzed using rapid techniques over a 3-year period.ResultsAdaptations during pre-implementation were planned, proactive, and focused on context and content to improve acceptability, appropriateness, and feasibility of the GTI playbook. Modifications during and after implementation were unplanned and reactive, concentrating on adoption, fidelity, and sustainability. All changes were collaboratively developed, fidelity consistent at the level of the facilitator and/or implementer.ConclusionGTO was initially adapted to GTI to support health care teams' selection and use of implementation strategies for improving guideline-concordant medical care. GTI required ongoing modification, particularly in steps regarding team building, context assessment, strategy selection, and sustainability due to difficulties with step clarity and progression. This work also highlights the challenges in pragmatic approaches to collecting and synthesizing implementation, fidelity, and adaptation data.Trial registrationThis study was registered on ClinicalTrials.gov (Identifier: NCT04178096)

Comparing the CFIR-ERIC matching tool recommendations to real-world strategy effectiveness data: a mixed-methods study in the Veterans Health Administration.

BACKGROUND: Practical and feasible methods for matching implementation strategies to diagnosed barriers of evidence-based interventions in real-world contexts are lacking. This evaluation compared actual implementation strategies applied with those recommended by an expert opinion-based tool to improve guideline-concordant cirrhosis care in a Veterans Health Administration national learning collaborative effort. METHODS: This convergent parallel mixed-methods study aimed to (1) identify pre-implementation Consolidated Framework for Implementation Research (CFIR) barriers to cirrhosis care through focus groups with frontline providers, (2) generate 20 recommended strategies using focus group identified barriers entered into the CFIR-Expert Recommendations for Implementing Change (ERIC) Implementation Strategy Matching Tool, (3) survey providers over two consecutive years on the actual use of 73 ERIC strategies and determine strategy effectiveness, (4) compare actual versus recommended strategy use, and (5) compare actual versus expected barriers by reverse applying the CFIR-ERIC Matching Tool. RESULTS: Eighteen semi-structured focus groups were conducted with 197 providers representing 95 VA sites to identify barriers to quality improvement, including cirrhosis care complexity, clarity of national goals, and local leadership support. The CFIR-ERIC Matching Tool recommended strategies such as assessing for readiness and needs, promoting adaptability, building local groups, preparing champions, and working with opinion leaders and early adopters. Subsequent strategy surveys found that sites used the top 20 recommended strategies no more frequently than other strategies. However, 14 (70%) of the top recommended strategies were significantly positively associated with cirrhosis care compared to 48% of actual strategies. Reverse CFIR-ERIC matching found that the strategies most used in the first year corresponded to the following barriers: opinion leaders, access to knowledge and information, and resources. The strategies most frequently employed in the second year addressed barriers such as champions, cosmopolitanism, readiness for implementation, relative priority, and patient needs and resources. Strategies used in both years were those that addressed adaptability, trialability, and compatibility. CONCLUSIONS: This study is among the first to empirically evaluate the relationship between CFIR-ERIC Matching Tool recommended strategies and actual strategy selection and effectiveness in the real world. We found closer connections between recommended strategies and strategy effectiveness compared to strategy frequency, suggesting validity of barrier identification, and application of the expert-informed tool

Characterizing patient-reported outcomes in veterans with cirrhosis.

Background and aimsThe Veterans Health Administration (VA) cares for over 80,000 Veterans with cirrhosis annually. Given the importance of understanding patient reported outcomes in this complex population, we aimed to assess the associations between attitudes towards care, disease knowledge, and health related quality of life (HRQoL) in a national sample.MethodsIn this cross-sectional study, we mailed paper surveys to a random sample of Veterans with cirrhosis, oversampling those with decompensated disease. Surveys included the Veterans RAND 12-Item Health Survey (measuring HRQoL) and questions about demographics, characteristics of care, satisfaction with care ("attitudes towards care"), and symptoms of cirrhosis. Those who reported being "unsure" about whether they had decompensation events were defined as "unsure about cirrhosis symptoms" ("disease knowledge"). We used multivariable regression models to assess the factors associated with HRQoL.ResultsOf 1374 surveys, 551 (40%) completed surveys were included for analysis. Most Veterans (63%) were "satisfied" or "very satisfied" with VA liver care. Patients often self-reported being unsure about whether they had experienced hepatic decompensation events (34%). Overall average physical (PCS) and mental (MCS) component scores of HRQoL were 30±11 and 41±12. In multivariable regression models, hepatic decompensation (PCS:β = -3.8, MCS:β = -2.2), medical comorbidities (β = --2.0, β = -1.7), and being unsure about cirrhosis symptoms (β = -1.9, β = -3.3) were associated with worse HRQoL, while age (β = 0.1, β = 0.2) and satisfaction with care (β = 0.6; β = 1.6) were associated with significantly better HRQoL.ConclusionsHepatic decompensation, lower satisfaction with care, and being unsure about cirrhosis symptoms were associated with reduced QOL scores in this national cohort

Refining Expert Recommendations for Implementing Change (ERIC) strategy surveys using cognitive interviews with frontline providers

BackgroundThe Expert Recommendations for Implementing Change (ERIC) compilation includes 73 defined implementation strategies clustered into nine content areas. This taxonomy has been used to track implementation strategies over time using surveys. This study aimed to improve the ERIC survey using cognitive interviews with non-implementation scientist clinicians.MethodsStarting in 2015, we developed and fielded annual ERIC surveys to evaluate liver care in the Veterans Health Administration (VA). We invited providers who had completed at least three surveys to participate in cognitive interviews (October 2020 to October 2021). Before the interviews, participants reviewed the complete 73-item ERIC survey and marked which strategies were unclear due to wording, conceptual confusion, or overlap with other strategies. They then engaged in semi-structured cognitive interviews to describe the experience of completing the survey and elaborate on which strategies required further clarification.ResultsTwelve VA providers completed surveys followed by cognitive interviews. The "Engage Consumer" and "Support Clinicians" clusters were rated most highly in terms of conceptual and wording clarity. In contrast, the "Financial" cluster had the most wording and conceptual confusion. The "Adapt and Tailor to Context" cluster strategies were considered to have the most redundancy. Providers outlined ways in which the strategies could be clearer in terms of wording (32%), conceptual clarity (51%), and clarifying the distinction between strategies (51%).ConclusionsCognitive interviews with ERIC survey participants allowed us to identify and address issues with strategy wording, combine conceptually indistinct strategies, and disaggregate multi-barreled strategies. Improvements made to the ERIC survey based on these findings will ultimately assist VA and other institutions in designing, evaluating, and replicating quality improvement efforts

Comparing the CFIR-ERIC matching tool recommendations to real-world strategy effectiveness data: a mixed-methods study in the Veterans Health Administration

Abstract Background Practical and feasible methods for matching implementation strategies to diagnosed barriers of evidence-based interventions in real-world contexts are lacking. This evaluation compared actual implementation strategies applied with those recommended by an expert opinion-based tool to improve guideline-concordant cirrhosis care in a Veterans Health Administration national learning collaborative effort. Methods This convergent parallel mixed-methods study aimed to (1) identify pre-implementation Consolidated Framework for Implementation Research (CFIR) barriers to cirrhosis care through focus groups with frontline providers, (2) generate 20 recommended strategies using focus group identified barriers entered into the CFIR-Expert Recommendations for Implementing Change (ERIC) Implementation Strategy Matching Tool, (3) survey providers over two consecutive years on the actual use of 73 ERIC strategies and determine strategy effectiveness, (4) compare actual versus recommended strategy use, and (5) compare actual versus expected barriers by reverse applying the CFIR-ERIC Matching Tool. Results Eighteen semi-structured focus groups were conducted with 197 providers representing 95 VA sites to identify barriers to quality improvement, including cirrhosis care complexity, clarity of national goals, and local leadership support. The CFIR-ERIC Matching Tool recommended strategies such as assessing for readiness and needs, promoting adaptability, building local groups, preparing champions, and working with opinion leaders and early adopters. Subsequent strategy surveys found that sites used the top 20 “recommended” strategies no more frequently than other strategies. However, 14 (70%) of the top recommended strategies were significantly positively associated with cirrhosis care compared to 48% of actual strategies. Reverse CFIR-ERIC matching found that the strategies most used in the first year corresponded to the following barriers: opinion leaders, access to knowledge and information, and resources. The strategies most frequently employed in the second year addressed barriers such as champions, cosmopolitanism, readiness for implementation, relative priority, and patient needs and resources. Strategies used in both years were those that addressed adaptability, trialability, and compatibility. Conclusions This study is among the first to empirically evaluate the relationship between CFIR-ERIC Matching Tool recommended strategies and actual strategy selection and effectiveness in the real world. We found closer connections between recommended strategies and strategy effectiveness compared to strategy frequency, suggesting validity of barrier identification, and application of the expert-informed tool

Characterizing patient-reported outcomes in veterans with cirrhosis.

Background and aimsThe Veterans Health Administration (VA) cares for over 80,000 Veterans with cirrhosis annually. Given the importance of understanding patient reported outcomes in this complex population, we aimed to assess the associations between attitudes towards care, disease knowledge, and health related quality of life (HRQoL) in a national sample.MethodsIn this cross-sectional study, we mailed paper surveys to a random sample of Veterans with cirrhosis, oversampling those with decompensated disease. Surveys included the Veterans RAND 12-Item Health Survey (measuring HRQoL) and questions about demographics, characteristics of care, satisfaction with care ("attitudes towards care"), and symptoms of cirrhosis. Those who reported being "unsure" about whether they had decompensation events were defined as "unsure about cirrhosis symptoms" ("disease knowledge"). We used multivariable regression models to assess the factors associated with HRQoL.ResultsOf 1374 surveys, 551 (40%) completed surveys were included for analysis. Most Veterans (63%) were "satisfied" or "very satisfied" with VA liver care. Patients often self-reported being unsure about whether they had experienced hepatic decompensation events (34%). Overall average physical (PCS) and mental (MCS) component scores of HRQoL were 30±11 and 41±12. In multivariable regression models, hepatic decompensation (PCS:β = -3.8, MCS:β = -2.2), medical comorbidities (β = --2.0, β = -1.7), and being unsure about cirrhosis symptoms (β = -1.9, β = -3.3) were associated with worse HRQoL, while age (β = 0.1, β = 0.2) and satisfaction with care (β = 0.6; β = 1.6) were associated with significantly better HRQoL.ConclusionsHepatic decompensation, lower satisfaction with care, and being unsure about cirrhosis symptoms were associated with reduced QOL scores in this national cohort

Characterizing patient-reported outcomes in veterans with cirrhosis.

Background and aimsThe Veterans Health Administration (VA) cares for over 80,000 Veterans with cirrhosis annually. Given the importance of understanding patient reported outcomes in this complex population, we aimed to assess the associations between attitudes towards care, disease knowledge, and health related quality of life (HRQoL) in a national sample.MethodsIn this cross-sectional study, we mailed paper surveys to a random sample of Veterans with cirrhosis, oversampling those with decompensated disease. Surveys included the Veterans RAND 12-Item Health Survey (measuring HRQoL) and questions about demographics, characteristics of care, satisfaction with care ("attitudes towards care"), and symptoms of cirrhosis. Those who reported being "unsure" about whether they had decompensation events were defined as "unsure about cirrhosis symptoms" ("disease knowledge"). We used multivariable regression models to assess the factors associated with HRQoL.ResultsOf 1374 surveys, 551 (40%) completed surveys were included for analysis. Most Veterans (63%) were "satisfied" or "very satisfied" with VA liver care. Patients often self-reported being unsure about whether they had experienced hepatic decompensation events (34%). Overall average physical (PCS) and mental (MCS) component scores of HRQoL were 30±11 and 41±12. In multivariable regression models, hepatic decompensation (PCS:β = -3.8, MCS:β = -2.2), medical comorbidities (β = --2.0, β = -1.7), and being unsure about cirrhosis symptoms (β = -1.9, β = -3.3) were associated with worse HRQoL, while age (β = 0.1, β = 0.2) and satisfaction with care (β = 0.6; β = 1.6) were associated with significantly better HRQoL.ConclusionsHepatic decompensation, lower satisfaction with care, and being unsure about cirrhosis symptoms were associated with reduced QOL scores in this national cohort

Getting to implementation: Adaptation of an implementation playbook

IntroductionImplementation strategies supporting the translation of evidence into practice need to be tailored and adapted for maximum effectiveness, yet the field of adapting implementation strategies remains nascent. We aimed to adapt "Getting To Outcomes"® (GTO), a 10-step implementation playbook designed to help community-based organizations plan and evaluate behavioral health programs, into "Getting To Implementation" (GTI) to support the selection, tailoring, and use of implementation strategies in health care settings.MethodsOur embedded evaluation team partnered with operations, external facilitators, and site implementers to employ participatory methods to co-design and adapt GTO for Veterans Health Administration (VA) outpatient cirrhosis care improvement. The Framework for Reporting Adaptations and Modifications to Evidenced-based Implementation Strategies (FRAME-IS) guided documentation and analysis of changes made pre- and post-implementation of GTI at 12 VA medical centers. Data from multiple sources (interviews, observation, content analysis, and fidelity tracking) were triangulated and analyzed using rapid techniques over a 3-year period.ResultsAdaptations during pre-implementation were planned, proactive, and focused on context and content to improve acceptability, appropriateness, and feasibility of the GTI playbook. Modifications during and after implementation were unplanned and reactive, concentrating on adoption, fidelity, and sustainability. All changes were collaboratively developed, fidelity consistent at the level of the facilitator and/or implementer.ConclusionGTO was initially adapted to GTI to support health care teams' selection and use of implementation strategies for improving guideline-concordant medical care. GTI required ongoing modification, particularly in steps regarding team building, context assessment, strategy selection, and sustainability due to difficulties with step clarity and progression. This work also highlights the challenges in pragmatic approaches to collecting and synthesizing implementation, fidelity, and adaptation data.Trial registrationThis study was registered on ClinicalTrials.gov (Identifier: NCT04178096)

Core implementation strategies for improving cirrhosis care in the Veterans Health Administration.

BACKGROUND AND AIMS: The Veterans Health Administration (VHA) provides care for more than 80,000 veterans with cirrhosis. This longitudinal, multimethod evaluation of a cirrhosis care quality improvement program aimed to (1) identify implementation strategies associated with evidence-based, guideline-concordant cirrhosis care over time, and (2) use qualitative interviews to operationalize strategies for a manualized intervention. APPROACH AND RESULTS: VHA providers were surveyed annually about the use of 73 implementation strategies to improve cirrhosis care in fiscal years 2018 (FY18) and 2019 (FY19). Implementation strategies linked to guideline-concordant cirrhosis care were identified using bivariate statistics and comparative configurational methods. Semistructured interviews were conducted with 12 facilities in the highest quartile of cirrhosis care to specify the successful implementation strategies and their mechanisms of change. A total of 106 VHA facilities (82%) responded at least once over the 2-year period (FY18, n = 63; FY19, n = 100). Facilities reported using a median of 12 (interquartile range [IQR] 20) implementation strategies in FY18 and 10 (IQR 19) in FY19. Of the 73 strategies, 35 (48%) were positively correlated with provision of evidence-based cirrhosis care. Configurational analysis identified multiple strategy pathways directly linked to more guideline-concordant cirrhosis care. Across both methods, a subset of eight strategies was determined to be core to cirrhosis care improvement and specified using qualitative interviews. CONCLUSIONS: In a national cirrhosis care improvement initiative, a multimethod approach identified a core subset of successful implementation strategy combinations. This process of empirically identifying and specifying implementation strategies may be applicable to other implementation challenges in hepatology