Methodological reflections on the evaluation of the implementation and adoption of national electronic health record systems

Copyright @ 2012, International Journal of Integrated Care (IJIC). This work is licensed under a (http://creativecommons.org/licenses/by/3.0) Creative Commons Attribution 3.0 Unported License.Introduction/purpose of presentation: Far-reaching policy commitments to information technology-centered transformations of healthcare systems have now been made in many countries. There is as yet little empirical evidence to justify such decisions, hence the need for rigorous independent evaluation of current implementation efforts. Such evaluations however pose a number of important challenges. This presentation has been designed as a part of a Panel based on our experience of evaluating the National Health Service’s (NHS) implementation of electronic health records (EHR) systems in hospitals throughout England. We discuss the methodological challenges encountered in planning and undertaking an evaluation of a program of this scale and reflect on why and how we adapted our evaluation approach—both conceptually and methodologically—in response to these challenges. Study design/population studied: Critical reflections on a multi-disciplinary and multi-facet independent evaluation of a national program to implement electronic health record systems into 12 ‘early wave’ NHS hospitals in England. Findings: Our initial plan was to employ a mixed methods longitudinal ‘before-during-after’ study design. We however found this unsustainable in the light of fluxes in policy, contractual issues and over-optimistic schedules for EHR deployments. More importantly, this research design failed adequately to address the core of multi-faceted evolving EHRs as understood by key stakeholders and as worked out in their distinct work settings. Thus conventional outcomes-centric evaluations may not easily scale-up when evaluating transformational programs and may indeed prove misleading. New assumptions concerning the implementation process of EHR need to be developed that recognize the constantly changing milieu of policy, product, projects and professions that are inherent to such national implementations. The approaches we subsequently developed substitute the positivist view that EHR initiatives are self-evident and self-contained interventions, which are amenable to traditional quantitative evaluations, to one that focuses on how they are understood by various stakeholders and made to work in specific contexts. These assumptions recast the role of evaluation towards an approach that explores and interprets processes of socio-technical change that surround EHR implementation and adoption as seen by multiple stakeholders. Conclusions and policy implications: There is likely to be an increase in politically-driven national programs of reform of healthcare based on information and communication technologies. Programs on such a scale are inherently complex with extended temporalities and extensive and dynamic sets of stakeholders. They are, in short, different and pose new evaluation challenges that previously formulated evaluation methods for health information systems cannot easily address. This calls for methodological innovation amongst research teams and their supporting bodies. We argue that evaluation of such system-wide transformation programs are likely to demand both breadth and depth of experience within a multidisciplinary research team, constant questioning of what is and what can be evaluated and how, and a particular way of working that emphasizes continuous dialogue and reflexivity. Making this transition is essential to enable evaluations that can usefully inform policy-making. Health policy experts urgently need to reassess the evaluation strategies they employ as they come to address national policies for system-wide transformation based on new electronic health infrastructures

Building a house on shifting sand: methodological considerations when evaluating the implementation and adoption of national electronic health record systems

Background: A commitment to Electronic Health Record (EHR) systems now constitutes a core part of many governments’ healthcare reform strategies. The resulting politically-initiated large-scale or national EHR endeavors are challenging because of their ambitious agendas of change, the scale of resources needed to make them work, the (relatively) short timescales set, and the large number of stakeholders involved, all of whom pursue somewhat different interests. These initiatives need to be evaluated to establish if they improve care and represent value for money. Methods: Critical reflections on these complexities in the light of experience of undertaking the first national, longitudinal, and sociotechnical evaluation of the implementation and adoption of England’s National Health Service’s Care Records Service (NHS CRS). Results/discussion: We advance two key arguments. First, national programs for EHR implementations are likely to take place in the shifting sands of evolving sociopolitical and sociotechnical and contexts, which are likely to shape them in significant ways. This poses challenges to conventional evaluation approaches which draw on a model of baseline operations → intervention → changed operations (outcome). Second, evaluation of such programs must account for this changing context by adapting to it. This requires careful and creative choice of ontological, epistemological and methodological assumptions. Summary: New and significant challenges are faced in evaluating national EHR implementation endeavors. Based on experiences from this national evaluation of the implementation and adoption of the NHS CRS in England, we argue for an approach to these evaluations which moves away from seeing EHR systems as Information and Communication Technologies (ICT) projects requiring an essentially outcome-centred assessment towards a more interpretive approach that reflects the situated and evolving nature of EHR seen within multiple specific settings and reflecting a constantly changing milieu of policies, strategies and software, with constant interactions across such boundaries

Conceptualising production, productivity and technology in pharmacy practice: a novel framework for policy, education and research.

CONTEXT AND BACKGROUND: People and health systems worldwide face serious challenges due to shifting disease demographics, rising population demands and weaknesses in healthcare provision, including capacity shortages and lack of impact of healthcare services. These multiple challenges, linked with the global push to achieve universal health coverage, have made apparent the importance of investing in workforce development to improve population health and economic well-being. In relation to medicines, health systems face challenges in terms of access to needed medicines, optimising medicines use and reducing risk. In 2017, the International Pharmaceutical Federation (FIP) published global policy on workforce development ('the Nanjing Statements') that describe an envisioned future for professional education and training. The documents make clear that expanding the pharmacy workforce benefits patients, and continually improving education and training produces better clinical outcomes. AIMS AND PURPOSE: The opportunities for harnessing new technologies in pharmacy practice have been relatively ignored. This paper presents a conceptual framework for analysing production methods, productivity and technology in pharmacy practice that differentiates between dispensing and pharmaceutical care services. We outline a framework that may be employed to study the relationship between pharmacy practice and productivity, shaped by educational and technological inputs. METHOD AND RESULTS: The analysis is performed from the point of view of health systems economics. In relation to pharmaceutical care (patient-oriented practice), pharmacists are service providers; however, their primary purpose is not to deliver consultations, but to maximise the quantum of health gain they secure. Our analysis demonstrates that 'technology shock' is clearly beneficial compared with orthodox notions of productivity or incremental gain implementations. Additionally, the whole process of providing professional services using 'pharmaceutical care technologies' is governed by local institutional frames, suggesting that activities may be structured differently in different places and countries. DISCUSSION AND CONCLUSION: Addressing problems with medication use with the development of a pharmaceutical workforce that is sufficient in quantity and competence is a long-term issue. As a result of this analysis, there emerges a challenge about the profession's relationship with existing and emerging technical innovations. Our novel framework is designed to facilitate policy, education and research by providing an analytical approach to service delivery. By using this approach, the profession could develop examples of good practice in both developed and developing countries worldwide

England’s Electronic Prescription Service: Infrastructure in an Institutional Setting

We describe the development of the Electronic Prescription Service (EPS), the solution for the electronic transmission of prescriptions adopted by the English NHS for primary care. The chapter is based on both an analysis of data collected as part of a nationally commissioned evaluation of EPS, and on reports of contemporary developments in the service. Drawing on the notion of an installed infrastructural base, we illustrate how EPS has been assembled within a rich institutional and organizational context including causal pasts, contemporary practices and policy visions. This process of assembly is traced using three perspectives; as the realization and negotiation of constraints found in the wider NHS context, as a response to inertia arising from limited resources and weak incentive structures, and as a purposive fidelity to the existing institutional cultures of the NHS. The chapter concludes by reflecting on the significance of this analysis for notions of an installed base

Video-calls to reduce loneliness and social isolation within care environments for older people: an implementation study using collaborative action research

Background  Older people in care may be lonely with insufficient contact if families are unable to visit. Face-to-face contact through video-calls may help reduce loneliness, but little is known about the processes of engaging people in care environments in using video-calls. We aimed to identify the barriers to and facilitators of implementing video-calls for older people in care environments.  Methods  A collaborative action research (CAR) approach was taken to implement a video-call intervention in care environments. We undertook five steps of recruitment, planning, implementation, reflection and re-evaluation, in seven care homes and one hospital in the UK. The video-call intervention ‘Skype on Wheels’ (SoW) comprised a wheeled device that could hold an iPad and handset, and used Skype to provide a free video-call service. Care staff were collaborators who implemented the intervention within the care-setting by agreeing the intervention, recruiting older people and their family, and setting up video-calls. Field notes and reflective diaries on observations and conversations with staff, older people and family were maintained over 15 months, and analysed using thematic analysis.  Results  Four care homes implemented the intervention. Eight older people with their respective social contacts made use of video-calls. Older people were able to use SoW with assistance from staff, and enjoyed the use of video-calls to stay better connected with family. However five barriers towards implementation included staff turnover, risk averseness, the SoW design, lack of family commitment and staff attitudes regarding technology.  Conclusions  The SoW intervention, or something similar, could aid older people to stay better connected with their families in care environments, but if implemented as part of a rigorous evaluation, then co-production of the intervention at each recruitment site may be needed to overcome barriers and maximise engagement

Recognising the limits of virtual organizations

No description supplie

Defining the virtual organization

No description supplie

Chapter 15 – Recognizing the limits of virtual organizations

No description supplie

Sociotechnical changing in healthcare organisations: the introduction of Electronic Patient Record (EPR) in an English hospital

No description supplie

Evaluation as a multi-ontological endeavour: A case from the English National Programme for IT in health care

This paper analyzes how researchers’ different ontological and epistemological assumptions shape the process and outcomes of evaluation research. Focusing on the critical realism (CR) and social constructionism (SC) philosophical approaches, it outlines the rationale for multi-ontological evaluation and develops principles for conducting it. The paper draws from experience of evaluating a national implementation program of electronic health records in hospitals, one of the projects of the English National Programme for IT. It argues that an evaluation based on SC and one based on CR are significantly different in how they use knowledge gained in the field, and in the kind of evidence and recommendations that they can offer policy makers. The CR philosophy applied to evaluation research provides foundations from which judgments and abstractions can be presented in the form expected by the policy makers and managers who commission evaluations. In line with its ontological standing, social constructionism cannot simply or directly abstract and generalize across contexts, though it can offer other types of valuable evaluative insight. We show that, despite their differences, these two philosophical positions can, when taken together, produce jointly useful knowledge. This paper argues for the use of multi-ontological evaluation approaches and provides guidelines for undertaking such endeavors by emphasizing the need for mutual respect, dialogue, negotiation, and reflection