Implementation outcome instruments for use in physical healthcare settings: a systematic review

Abstract

Background:Implementation research aims to facilitate the timely and routine implementation and sustainment ofevidence-based interventions and services. A glaring gap in this endeavour is the capability of researchers,healthcare practitioners and managers to quantitatively evaluate implementation efforts using psychometricallysound instruments. To encourage and support the use of precise and accurate implementation outcome measures,this systematic review aimed to identify and appraise studies that assess the measurement properties ofquantitative implementation outcome instruments used in physical healthcare settings. Method:The following data sources were searched from inception to March 2019, with no language restrictions:MEDLINE, EMBASE, PsycINFO, HMIC, CINAHL and the Cochrane library. Studies that evaluated the measurementproperties of implementation outcome instruments in physical healthcare settings were eligible for inclusion.Proctor et al.’s taxonomy of implementation outcomes was used to guide the inclusion of implementationoutcomes: acceptability, appropriateness, feasibility, adoption, penetration, implementation cost and sustainability.Methodological quality of the included studies was assessed using the COnsensus-based Standards for the selectionof health Measurement INstruments (COSMIN) checklist. Psychometric quality of the included instruments wasassessed using the Contemporary Psychometrics checklist (ConPsy). Usability was determined by number of itemsper instrument. Results:Fifty-eight publications reporting on the measurement properties of 55 implementation outcomeinstruments (65 scales) were identified. The majority of instruments assessed acceptability (n= 33), followed byappropriateness (n= 7), adoption (n= 4), feasibility (n= 4), penetration (n= 4) and sustainability (n=3)ofevidence-based practice. The methodological quality of individual scales was low, with few studies rated as‘excellent’for reliability (6/62) and validity (7/63), and both studies that assessed responsiveness rated as‘poor’(2/2).The psychometric quality of the scales was also low, with 12/65 scales scoring 7 or more out of 22, indicatinggreater psychometric strength. Six scales (6/65) rated as‘excellent’for usability.Conclusion:Investigators assessing implementation outcomes quantitatively should select instruments based ontheir methodological and psychometric quality to promote consistent and comparable implementation evaluations.Rather than developing ad hoc instruments, we encourage further psychometric testing of instruments withpromising methodological and psychometric evidence. Systematic review registration:PROSPERO 2017CRD4201706534