A Patient-Centered Framework for Evaluating Digital Maturity of Health Services: A Systematic Review

© Kelsey Flott, Ryan Callahan, Ara Darzi, Erik Mayer.Background: Digital maturity is the extent to which digital technologies are used as enablers to deliver a high-quality health service. Extensive literature exists about how to assess the components of digital maturity, but it has not been used to design a comprehensive framework for evaluation. Consequently, the measurement systems that do exist are limited to evaluating digital programs within one service or care setting, meaning that digital maturity evaluation is not accounting for the needs of patients across their care pathways. Objective: The objective of our study was to identify the best methods and metrics for evaluating digital maturity and to create a novel, evidence-based tool for evaluating digital maturity across patient care pathways. Methods: We systematically reviewed the literature to find the best methods and metrics for evaluating digital maturity. We searched the PubMed database for all papers relevant to digital maturity evaluation. Papers were selected if they provided insight into how to appraise digital systems within the health service and if they indicated the factors that constitute or facilitate digital maturity. Papers were analyzed to identify methodology for evaluating digital maturity and indicators of digitally mature systems. We then used the resulting information about methodology to design an evaluation framework. Following that, the indicators of digital maturity were extracted and grouped into increasing levels of maturity and operationalized as metrics within the evaluation framework. Results: We identified 28 papers as relevant to evaluating digital maturity, from which we derived 5 themes. The first theme concerned general evaluation methodology for constructing the framework (7 papers). The following 4 themes were the increasing levels of digital maturity: resources and ability (6 papers), usage (7 papers), interoperability (3 papers), and impact (5 papers). The framework includes metrics for each of these levels at each stage of the typical patient care pathway. Conclusions: The framework uses a patient-centric model that departs from traditional service-specific measurements and allows for novel insights into how digital programs benefit patients across the health system

Coordination of European transnational research in organic food and farming

Research in organic food and farming is a fairly new, but rapidly ex-panding, discipline on the European research scene. One of the problems faced by the authorities seeking to initiate research programmes in organic food and farming is that the present research effort in Europe is caracterised by small research communities, which are often scattered and fragmented both geographically and institutionally. Therefore a gathering of the dispersed expertise to a critical mass in order to increase the competitive quality and relevance of the research as well as the dessemination and use of research is needed

Utilizing RxNorm to Support Practical Computing Applications: Capturing Medication History in Live Electronic Health Records

RxNorm was utilized as the basis for direct-capture of medication history data in a live EHR system deployed in a large, multi-state outpatient behavioral healthcare provider in the United States serving over 75,000 distinct patients each year across 130 clinical locations. This tool incorporated auto-complete search functionality for medications and proper dosage identification assistance. The overarching goal was to understand if and how standardized terminologies like RxNorm can be used to support practical computing applications in live EHR systems. We describe the stages of implementation, approaches used to adapt RxNorm's data structure for the intended EHR application, and the challenges faced. We evaluate the implementation using a four-factor framework addressing flexibility, speed, data integrity, and medication coverage. RxNorm proved to be functional for the intended application, given appropriate adaptations to address high-speed input/output (I/O) requirements of a live EHR and the flexibility required for data entry in multiple potential clinical scenarios. Future research around search optimization for medication entry, user profiling, and linking RxNorm to drug classification schemes holds great potential for improving the user experience and utility of medication data in EHRs.Comment: Appendix (including SQL/DDL Code) available by author request. Keywords: RxNorm; Electronic Health Record; Medication History; Interoperability; Unified Medical Language System; Search Optimizatio

One size does not fit all: investigating doctors' stated preference heterogeneity for job incentives to inform policy in Thailand.

This study investigates heterogeneity in Thai doctors' job preferences at the beginning of their career, with a view to inform the design of effective policies to retain them in rural areas. A discrete choice experiment was designed and administered to 198 young doctors. We analysed the data using several specifications of a random parameter model to account for various sources of preference heterogeneity. By modelling preference heterogeneity, we showed how sensitivity to different incentives varied in different sections of the population. In particular, doctors from rural backgrounds were more sensitive than others to a 45% salary increase and having a post near their home province, but they were less sensitive to a reduction in the number of on-call nights. On the basis of the model results, the effects of two types of interventions were simulated: introducing various incentives and modifying the population structure. The results of the simulations provide multiple elements for consideration for policy-makers interested in designing effective interventions. They also underline the interest of modelling preference heterogeneity carefully

Raising the visibility of protected data: A pilot data catalog project

Sharing research data that is protected for legal, regulatory, or contractual reasons can be challenging and current mechanisms for doing so may act as barriers to researchers and discourage data sharing. Additionally, the infrastructure commonly used for open data repositories does not easily support responsible sharing of protected data. This chapter presents a case study of an academic university library’s work to configure the existing institutional data repository to function as a data catalog. By engaging in this project, university librarians strive to enhance visibility and access to protected datasets produced at the institution and cultivate a data sharing culture

A Guide to Evaluating Marine Spatial Plans

Marine spatial plans are being developed in over 40 countries around the world, to distribute human activities in marine areas more sustainably and achieve ecological, social, and economic objectives. Monitoring and evaluation are often considered only after a plan has been developed. This guide will help marine planners and managers, monitor and evaluate the success of marine plans in achieving real results and outcomes. This report emphasizes the importance of early integration of monitoring and evaluation in the planning process, the importance of measurable and specific objectives, clear management actions, relevant indicators and targets, and involvement of stakeholders throughout the planning process.

Evaluation of crime prevention initiatives

This third toolbox in the series published by the EUCPN Secretariat focuses on the main theme of the Irish Presidency, which is the evaluation of crime prevention initiatives. The theme is explored and elaborated in various ways through: a literature review; two workshops with international experts and practitioners during which the strengths and weaknesses of programme evaluation were discussed in detail; a screening of existing guidelines and manuals on evaluation; and finally, a call which was launched by the EUCPN Secretariat to the Member States to collect some practices on the evaluation of crime prevention initiatives

Communicating for Policy Change

Synthesizes information from a November 2006 GIH Issue Dialogue meeting that explored communications theories and techniques and their application to policy work

Lost in Translation: Piloting a Novel Framework to Assess the Challenges in Translating Scientific Uncertainty From Empirical Findings to WHO Policy Statements.

BACKGROUND:Calls for evidence-informed public health policy, with implicit promises of greater program effectiveness, have intensified recently. The methods to produce such policies are not self-evident, requiring a conciliation of values and norms between policy-makers and evidence producers. In particular, the translation of uncertainty from empirical research findings, particularly issues of statistical variability and generalizability, is a persistent challenge because of the incremental nature of research and the iterative cycle of advancing knowledge and implementation. This paper aims to assess how the concept of uncertainty is considered and acknowledged in World Health Organization (WHO) policy recommendations and guidelines. METHODS:We selected four WHO policy statements published between 2008-2013 regarding maternal and child nutrient supplementation, infant feeding, heat action plans, and malaria control to represent topics with a spectrum of available evidence bases. Each of these four statements was analyzed using a novel framework to assess the treatment of statistical variability and generalizability. RESULTS:WHO currently provides substantial guidance on addressing statistical variability through GRADE (Grading of Recommendations Assessment, Development, and Evaluation) ratings for precision and consistency in their guideline documents. Accordingly, our analysis showed that policy-informing questions were addressed by systematic reviews and representations of statistical variability (eg, with numeric confidence intervals). In contrast, the presentation of contextual or "background" evidence regarding etiology or disease burden showed little consideration for this variability. Moreover, generalizability or "indirectness" was uniformly neglected, with little explicit consideration of study settings or subgroups. CONCLUSION:In this paper, we found that non-uniform treatment of statistical variability and generalizability factors that may contribute to uncertainty regarding recommendations were neglected, including the state of evidence informing background questions (prevalence, mechanisms, or burden or distributions of health problems) and little assessment of generalizability, alternate interventions, and additional outcomes not captured by systematic review. These other factors often form a basis for providing policy recommendations, particularly in the absence of a strong evidence base for intervention effects. Consequently, they should also be subject to stringent and systematic evaluation criteria. We suggest that more effort is needed to systematically acknowledge (1) when evidence is missing, conflicting, or equivocal, (2) what normative considerations were also employed, and (3) how additional evidence may be accrued

Evaluation Design and Technical Assistance Opportunities: Early Findings From the Beacon Community Program Evaluation Teams

Examines study designs, evaluation approaches, outcome measures, data sources, challenges, and technical assistance needs among sites in a project under the American Recovery and Reinvestment Act to strengthen health information technology capabilities