Methodological reflections on the evaluation of the implementation and adoption of national electronic health record systems

Copyright @ 2012, International Journal of Integrated Care (IJIC). This work is licensed under a (http://creativecommons.org/licenses/by/3.0) Creative Commons Attribution 3.0 Unported License.Introduction/purpose of presentation: Far-reaching policy commitments to information technology-centered transformations of healthcare systems have now been made in many countries. There is as yet little empirical evidence to justify such decisions, hence the need for rigorous independent evaluation of current implementation efforts. Such evaluations however pose a number of important challenges. This presentation has been designed as a part of a Panel based on our experience of evaluating the National Health Service’s (NHS) implementation of electronic health records (EHR) systems in hospitals throughout England. We discuss the methodological challenges encountered in planning and undertaking an evaluation of a program of this scale and reflect on why and how we adapted our evaluation approach—both conceptually and methodologically—in response to these challenges. Study design/population studied: Critical reflections on a multi-disciplinary and multi-facet independent evaluation of a national program to implement electronic health record systems into 12 ‘early wave’ NHS hospitals in England. Findings: Our initial plan was to employ a mixed methods longitudinal ‘before-during-after’ study design. We however found this unsustainable in the light of fluxes in policy, contractual issues and over-optimistic schedules for EHR deployments. More importantly, this research design failed adequately to address the core of multi-faceted evolving EHRs as understood by key stakeholders and as worked out in their distinct work settings. Thus conventional outcomes-centric evaluations may not easily scale-up when evaluating transformational programs and may indeed prove misleading. New assumptions concerning the implementation process of EHR need to be developed that recognize the constantly changing milieu of policy, product, projects and professions that are inherent to such national implementations. The approaches we subsequently developed substitute the positivist view that EHR initiatives are self-evident and self-contained interventions, which are amenable to traditional quantitative evaluations, to one that focuses on how they are understood by various stakeholders and made to work in specific contexts. These assumptions recast the role of evaluation towards an approach that explores and interprets processes of socio-technical change that surround EHR implementation and adoption as seen by multiple stakeholders. Conclusions and policy implications: There is likely to be an increase in politically-driven national programs of reform of healthcare based on information and communication technologies. Programs on such a scale are inherently complex with extended temporalities and extensive and dynamic sets of stakeholders. They are, in short, different and pose new evaluation challenges that previously formulated evaluation methods for health information systems cannot easily address. This calls for methodological innovation amongst research teams and their supporting bodies. We argue that evaluation of such system-wide transformation programs are likely to demand both breadth and depth of experience within a multidisciplinary research team, constant questioning of what is and what can be evaluated and how, and a particular way of working that emphasizes continuous dialogue and reflexivity. Making this transition is essential to enable evaluations that can usefully inform policy-making. Health policy experts urgently need to reassess the evaluation strategies they employ as they come to address national policies for system-wide transformation based on new electronic health infrastructures

Care: an integrated framework to support continuous, adaptable, reflective evaluation of eGovernment systems: A research note

CARE: an Integrated Framework to Support Continuous, Adaptable, Reflective Evaluation of Egovernment SystemsThis is an eGISE network paper. It is motivated by a concern to develop a better approach to learning from the experience of an eGovernment project and applying that knowledge in future projects. The proposed project is based on previous work in the construction industry that developed COLA, a Cross Organisational Learning Approach. Developing a similar strategy for Knowledge Management is likely to be effective because the ‘silo’ culture of local government organisations has parallels with the segmented organisational structures within the construction industry.Engineering and Physical Sciences Research Council (EPSRC) UK (grant GR/T27020/01

Dental Professionals in Non-Dental Settings

This report focuses on nine oral health innovations seeking to increase access to preventive oral health care in nondental settings. Two additional reports in this series describe the remaining programs that provide care in dental settings and care to young children. The nine innovations described here integrate service delivery and workforce models in order to reduce or eliminate socioeconomic, geographic, and cultural barriers to care. Although the programs are diverse in their approaches as well as in the specific characteristics of the communities they serve, a common factor among them is the implementation of multiple strategies to increase the number of children from low-income families who access preventive care, and also to engage families and communities in investing in and prioritizing oral health. For low-income children and their families, the barriers that must be addressed to increase access to preventive oral health care are numerous. For example, even children covered by public insurance programs face a shortage of dentists that accept Medicaid and who specialize in pediatric dentistry. The effects of poverty intersect with other barriers such as living in remote geographic areas and having a community-wide history of poor access to dental care in populations such as recent immigrants. Overcoming these barriers requires creative strategies that address transportation barriers, establish welcoming environments for oral health care, and are linguistically and culturally relevant. Each of these nine programs is based on such strategies, including:-Expanding the dental workforce through training new types of providers or adding new providers to the workforce toincrease reach and community presence;-Implementing new strategies to increase the cost-effectiveness of care so that more oral health care services are available and accessible;-Providing training and technical assistance that increase opportunities for and competence in delivering oral health education and care to children;-Offering oral health care services in existing, familiar community venues such as schools, Head Start programs and senior centers;-Developing creative service delivery models that address transportation and cultural barriers as well as the fear and stigma associated with dental care that may arise in communities with historically poor access.The findings from the EAs of these programs are synthesized to highlight diverse and innovative strategies for overcoming barriers to access. These strategies have potential for rigorous evaluation and could emerge as best practices. If proven effective, these innovative program elements could then be disseminated and replicated to increase access for populations in need of preventive oral health care

CARE: An integrated framework to support continuous, adaptable, reflective evaluation of egovernment systems: A research note

CARE: an Integrated Framework to Support Continuous, Adaptable, Reflective Evaluation of Egovernment SystemsThis is an eGISE network paper. It is motivated by a concern to develop a better approach to learning from the experience of an eGovernment project and applying that knowledge in future projects. The proposed project is based on previous work in the construction industry that developed COLA, a Cross Organisational Learning Approach. Developing a similar strategy for Knowledge Management is likely to be effective because the ‘silo’ culture of local government organisations has parallels with the segmented organisational structures within the construction industry.Engineering and Physical Sciences Research Council, UK (grant GR/T27020/01

Investing in Broadband Evaluation: Guiding Policy & Innovation for the Future

How we address the broadband challenge has been called the most important infrastructure challenge of the new century by the National Broadband Plan. High-speed Internet can connect remote communities, help coordinate and streamline health care services, enable our children with unparalleled access to learning opportunities, and spark and support innovation in numerous fields. The challenge, however, is understanding what works and why across all of these important uses of broadband technology. Program evaluation can answer this need, especially if it is built into new programs and policies from the start

save to DISC: Documenting Innovation in Music Learning

The paper discusses an approach to determining the worth and value of innovation in music education and measuring it’s capacity for meaning and engagement. It also aims to identify new examples of innovation across a broad range of music learning contexts and establish a rigorous digital process for documenting, evaluating and distributing innovative cases and resources for present and future contexts. It discusses specifically a pilot project that seeks to document innovation in sound curriculum (DISC). save to DISC is an exploratory study in an Australasian CRC for Interaction Design (ACID) project that proposes to establish flexible and effective procedures for the sourcing, evaluating, refereeing, editing, producing, validating, storing, publishing, and distributing of a wide range of media and content types. It involves documenting innovative and successful practice in music education, creating and evaluating programs in difficult/challenging school contexts and commissioning and encouraging the production of resource materials for 21 st century contexts

The Elusive Talent Strategy: An Excellent Teacher for Every Student in Every School

Calls for improving the teacher applicant pool; preparing, recruiting and retaining the best; incentivizing hard-to-fill assignments; utilizing data to improve support systems and evaluations; and bolstering accountability. Includes lessons from abroad

Validation of Ultrahigh Dependability for Software-Based Systems

Modern society depends on computers for a number of critical tasks in which failure can have very high costs. As a consequence, high levels of dependability (reliability, safety, etc.) are required from such computers, including their software. Whenever a quantitative approach to risk is adopted, these requirements must be stated in quantitative terms, and a rigorous demonstration of their being attained is necessary. For software used in the most critical roles, such demonstrations are not usually supplied. The fact is that the dependability requirements often lie near the limit of the current state of the art, or beyond, in terms not only of the ability to satisfy them, but also, and more often, of the ability to demonstrate that they are satisfied in the individual operational products (validation). We discuss reasons why such demonstrations cannot usually be provided with the means available: reliability growth models, testing with stable reliability, structural dependability modelling, as well as more informal arguments based on good engineering practice. We state some rigorous arguments about the limits of what can be validated with each of such means. Combining evidence from these different sources would seem to raise the levels that can be validated; yet this improvement is not such as to solve the problem. It appears that engineering practice must take into account the fact that no solution exists, at present, for the validation of ultra-high dependability in systems relying on complex software

Strengthening High School Teaching and Learning in New Hampshire's Competency-Based System

For a century, most students have advanced from grade to grade based on the number of days they spend in class, but in New Hampshire, schools have moved away from "seat time" and toward "competency-based learning," which advances students when they have mastered course content. This report profiles how two high schools in New Hampshire made this shift and examines the changes that were necessary to make competency-based advancement an important part of New Hampshire's strategy for implementing the Common Core State Standards and ensuring that students graduate ready for college and a career

Information technology and performance management for build-to-order supply chains

En las siguientes líneas se plantea un artículo de reflexión que tiene en cuenta parte del marco teórico que sustenta la investigación titulada “Prácticas pedagógicas que promueven la competencia argumentativa escrita (CAE) en niños campesinos de los grados 4° y 5° del Centro Educativo Municipal La Caldera, Sede Principal de Pasto”, desarrollada en el año 2012. En él se contemplan los aportes de las ciencias del lenguaje y la comunicación, la teoría de la argumentación, la didáctica de la lengua escrita y los géneros discursivos, que dan cuenta de la necesidad de desarrollar la capacidad crítica en los estudiantes a través de la argumentación, lo cual implica transformar las prácticas pedagógicas para que se alejen de la transmisión de conocimientos y den paso a la comunicación, para que la palabra escrita sea apropiada de manera significativa