Driving digital health transformation in hospitals:a formative qualitative evaluation of the English Global Digital Exemplar programme

BACKGROUND: There is currently a strong drive internationally towards creating digitally advanced healthcare systems through coordinated efforts at a national level. The English Global Digital Exemplar (GDE) programme is a large-scale national health information technology change programme aiming to promote digitally-enabled transformation in secondary healthcare provider organisations by supporting relatively digitally mature provider organisations to become international centres of excellence. AIM: To qualitatively evaluate the impact of the GDE programme in promoting digital transformation in provider organisations that took part in the programme. METHODS: We conducted a series of in-depth case studies in 12 purposively selected provider organisations and a further 24 wider case studies of the remaining organisations participating in the GDE programme. Data collected included 628 interviews, non-participant observations of 190 meetings and workshops and analysis of 9 documents. We used thematic analysis aided by NVivo software and drew on sociotechnical theory to analyse the data. RESULTS: We found the GDE programme accelerated digital transformation within participating provider organisations. This acceleration was triggered by: (1) dedicated funding and the associated requirement for matched internal funding, which in turn helped to prioritise digital transformation locally; (2) governance requirements put in place by the programme that helped strengthen existing local governance and project management structures and supported the emergence of a cadre of clinical health informatics leaders locally; and (3) reputational benefits associated with being recognised as a centre of digital excellence, which facilitated organisational buy-in for digital transformation and increased negotiating power with vendors. CONCLUSION: The GDE programme has been successful in accelerating digital transformation in participating provider organisations. Large-scale digital transformation programmes in healthcare can stimulate local progress through protected funding, putting in place governance structures and leveraging reputational benefits for participating provider organisations, around a coherent vision of transformation

Interorganizational Knowledge Sharing to Establish Digital Health Learning Ecosystems:Qualitative Evaluation of a National Digital Health Transformation Program in England

BACKGROUND: The English Global Digital Exemplar (GDE) program is one of the first concerted efforts to create a digital health learning ecosystem across a national health service. OBJECTIVE: This study aims to explore mechanisms that support or inhibit the exchange of interorganizational digital transformation knowledge. METHODS: We conducted a formative qualitative evaluation of the GDE program. We used semistructured interviews with clinical, technical, and managerial staff; national program managers and network leaders; nonparticipant observations of knowledge transfer activities through attending meetings, workshops, and conferences; and documentary analysis of policy documents. The data were thematically analyzed by drawing on a theory-informed sociotechnical coding framework. We used a mixture of deductive and inductive methods, supported by NVivo software, to facilitate coding. RESULTS: We conducted 341 one-on-one and 116 group interviews, observed 86 meetings, and analyzed 245 documents from 36 participating provider organizations. We also conducted 51 high-level interviews with policy makers and vendors; performed 77 observations of national meetings, workshops, and conferences; and analyzed 80 national documents. Formal processes put in place by the GDE program to initiate and reinforce knowledge transfer and learning have accelerated the growth of informal knowledge networking and helped establish the foundations of a learning ecosystem. However, formal networks were most effective when supported by informal networking. The benefits of networking were enhanced (and costs reduced) by geographical proximity, shared culture and context, common technological functionality, regional and strategic alignments, and professional agendas. CONCLUSIONS: Knowledge exchange is most effective when sustained through informal networking driven by the mutual benefits of sharing knowledge and convergence between group members in their organizational and technological setting and goals. Policy interventions need to enhance incentives and reduce barriers to sharing across the ecosystem, be flexible in tailoring formal interventions to emerging needs, and promote informal knowledge sharing

Using Blueprints to promote interorganizational knowledge transfer in digital health initiatives—a qualitative exploration of a national change program in English hospitals

OBJECTIVE: The Global Digital Exemplar (GDE) Program is a national attempt to accelerate digital maturity in healthcare providers through promoting knowledge transfer across the English National Health Service (NHS). “Blueprints”—documents capturing implementation experience—were intended to facilitate this knowledge transfer. Here we explore how Blueprints have been conceptualized, produced, and used to promote interorganizational knowledge transfer across the NHS. MATERIALS AND METHODS: We undertook an independent national qualitative evaluation of the GDE Program. This involved collecting data using semistructured interviews with implementation staff and clinical leaders in provider organizations, nonparticipant observation of meetings, and key documents. We also attended a range of national meetings and conferences, interviewed national program managers, and analyzed a range of policy documents. Our analysis drew on sociotechnical principles, combining deductive and inductive methods. RESULTS: Data comprised 508 interviews, 163 observed meetings, and analysis of 325 documents. We found little evidence of Blueprints being adopted in the manner originally conceived by national program managers. However, they proved effective in different ways to those planned. As well as providing a helpful initial guide to a topic, we found that Blueprints served as a method of identifying relevant expertise that paved the way for subsequent discussions and richer knowledge transfers amongst provider organizations. The primary value of Blueprinting, therefore, seemed to be its role as a networking tool. Members of different organizations came together in developing, applying, and sustaining Blueprints through bilateral conversations—in some circumstances also fostering informal communities of practice. CONCLUSIONS: Blueprints may be effective in facilitating knowledge transfer among healthcare organizations, but need to be accompanied by other evolving methods, such as site visits and other networking activities, to iteratively transfer knowledge and experience

Theoretical and methodological considerations in evaluating large-scale health information technology change programmes

BACKGROUND: Attempts to achieve digital transformation across the health service have stimulated increasingly large-scale and more complex change programmes. These encompass a growing range of functions in multiple locations across the system and may take place over extended timeframes. This calls for new approaches to evaluate these programmes. MAIN BODY: Drawing on over a decade of conducting formative and summative evaluations of health information technologies, we here build on previous work detailing evaluation challenges and ways to tackle these. Important considerations include changing organisational, economic, political, vendor and markets necessitating tracing of evolving networks, relationships, and processes; exploring mechanisms of spread; and studying selected settings in depth to understand local tensions and priorities. CONCLUSIONS: Decision-makers need to recognise that formative evaluations, if built on solid theoretical and methodological foundations, can help to mitigate risks and help to ensure that programmes have maximum chances of success

Formative independent evaluation of a digital change programme in the English National Health Service:Study protocol for a longitudinal qualitative study

INTRODUCTION: Many countries are launching large-scale, digitally enabled change programmes as part of efforts to improve the quality, safety and efficiency of care. We have been commissioned to conduct an independent evaluation of a major national change programme, the Global Digital Exemplar (GDE) Programme, which aims to develop exemplary digital health solutions and encourage their wider adoption by creating a learning ecosystem across English National Health Service (NHS) provider organisations. METHODS AND ANALYSIS: This theoretically informed, qualitative, longitudinal formative evaluation comprises five inter-related work packages. We will conduct a combination of 12 in-depth and 24 broader qualitative case studies in GDE sites exploring digital transformation, local learning and mechanisms of spread of knowledge within the Programme and across the wider NHS. Data will be collected through a combination of semistructured interviews with managers, implementation staff (clinical and non-clinical), vendors and policymakers, plus non-participant observations of meetings, site visits, workshops and documentary analysis of strategic local and national plans. Data will be analysed through inductive and deductive methods, beginning with in-depth case study sites and testing the findings against data from the wider sample and national stakeholders. ETHICS AND DISSEMINATION: This work is commissioned as part of a national change programme and is therefore a service evaluation. We have ethical approval from the University of Edinburgh. Results will be disseminated at six monthly intervals to national policymakers, and made available via our publicly accessible website. We will also identify lessons for the management and evaluation of large-scale evolving digital health change programmes that are of international relevance

Promoting inter-organisational knowledge sharing:A qualitative evaluation of England's Global Digital Exemplar and Fast Follower Programme

BACKGROUND: The Global Digital Exemplar (GDE) Programme was designed to promote the digitisation of hospital services in England. Selected provider organisations that were reasonably digitally-mature were funded with the expectation that they would achieve internationally recognised levels of excellence and act as exemplars (‘GDE sites’) and share their learning with somewhat less digitally-mature Fast Follower (FF) sites. AIMS: This paper explores how partnerships between GDE and FF sites have promoted knowledge sharing and learning between organisations. METHODS: We conducted an independent qualitative longitudinal evaluation of the GDE Programme, collecting data across 36 provider organisations (including acute, mental health and speciality), 12 of which we studied as in-depth ethnographic case studies. We used a combination of semi-structured interviews with programme leads, vendors and national policy leads, non-participant observations of meetings and workshops, and analysed national and local documents. This allowed us to explore both how inter-organisational learning and knowledge sharing was planned, and how it played out in practice. Thematic qualitative analysis, combining findings from diverse data sources, was facilitated by NVivo 11 and drew on sociotechnical systems theory. RESULTS: Formally established GDE and FF partnerships were perceived to enhance learning and accelerate adoption of technologies in most pairings. They were seen to be most successful where they had encouraged, and were supported by, informal knowledge networking, driven by the mutual benefits of information sharing. Informal networking was enhanced where the benefits were maximised (for example where paired sites had implemented the same technological system) and networking costs minimised (for example by geographical proximity, prior links and institutional alignment). Although the intervention anticipated uni-directional learning between exemplar sites and ‘followers’, in most cases we observed a two-way flow of information, with GDEs also learning from FFs, through informal networking which also extended to other health service providers outside the Programme. The efforts of the GDE Programme to establish a learning ecosystem has enhanced the profile of shared learning within the NHS. CONCLUSIONS: Inter-organisational partnerships have produced significant gains for both follower (FF) and exemplar (GDE) sites. Formal linkages were most effective where they had facilitated, and were supported by, informal networking. Informal networking was driven by the mutual benefits of information sharing and was optimised where sites were well aligned in terms of technology, geography and culture. Misalignments that created barriers to networking between organisations in a few cases were attributed to inappropriate choice of partners. Policy makers seeking to promote learning through centrally directed mechanisms need to create a framework that enables networking and informal knowledge transfer, allowing local organisations to develop bottom-up collaboration and exchanges, where they are productive, in an organic manner

Benefits realization management in the context of a national digital transformation initiative in English provider organizations

BACKGROUND: The Global Digital Exemplar (GDE) Programme is a national initiative to promote digitally enabled transformation in English provider organizations. The Programme applied benefits realization management techniques to promote and demonstrate transformative outcomes. This work was part of an independent national evaluation of the GDE Programme. AIMS: We explored how benefits realization management was approached and conceptualized in the GDE Programme. METHODS: We conducted a series of 36 longitudinal case studies of provider organizations participating in the GDE Programme, 12 of which were in depth. Data collection included a combination of 628 interviews (with implementation staff in provider organizations, national programme management staff, and suppliers), 499 documents (of national and local implementation plans and lessons learned), and 190 nonparticipant observations (of national and local programme management meetings to develop insights into the broader context of benefits realization activities, tensions arising, and how these were negotiated). Data were coded drawing on a sociotechnical framework developed in related work and thematically analyzed, initially within and then across cases, with the help of NVivo 11 software. RESULTS: Most stakeholders broadly agreed with the rationale of benefits realization in the GDE Programme to show due diligence that public money was appropriately spent, and to develop an evidence base supporting the value of digitally enabled transformation. Differing national and local reporting purposes, however, created tensions. Central requirements, for progress reporting and tracking high-level benefits, had limited perceived local value and were seen to impose an unnecessary burden on provider organizations. This was accentuated by the lack of harmonization of reporting requirements to different stakeholders (which differed in content and timing). There were tensions between the desire for early evidence of outcomes and the slow processes of infrastructural change (which created problems of attribution of benefits to causes as benefits emerged gradually and over long timeframes), and also between reporting immediately visible local changes and showing how these flowed through to high level organization wide benefits (eg, in terms of health outcomes or cost savings/return on investment). The attempt to fulfill these diverging agendas and informational needs within a single reporting tool had limited success. These difficulties were mitigated by efforts to simplify reporting requirements and to support targeted collection of key national outcome measures. Although progress was hampered by an initial lack of benefits realization expertise in provider organizations, some providers subsequently retained these skills for their own change management purposes. CONCLUSIONS: There is a need to recognize the limitations and cost of benefits realization management practices in the context of healthcare digitalization where benefits may materialize over long timeframes and in unanticipated ways. Although diverse stakeholder information needs may create tensions, prior agreement about rationales for collecting information and a targeted approach to tracking local and high-level benefits may enhance local relevance, reduce perceived reporting burdens, and improve acceptance/effectiveness. A single integrated reporting mechanism is unlikely to fulfill both national and local requirements

Streetsport: Supporting and facilitating the development of enhanced graduate attributes.

Streetsport is a programme that aims to exercise social innovation by reducing instances of youth crime and anti-social behaviour; whilst promoting health and wellbeing through sport, physical activity and creative endeavour. As a vehicle for delivery the initiative facilitates work based educational experiences that are embedded within disadvantaged communities; supporting the development of enhanced graduate attributes by way of collaborative teaching and learning support. Adopting a collaborative partnership model, the programme brings together multiple courses, students and stakeholders to work within communities resulting in activities that react and respond to local needs, interests and demand

Technological capabilities to assess digital excellence in hospitals in high performing healthcare systems::an international eDelphi exercise

Background: Hospitals worldwide are developing ambitious digital transformation programs as part of broader efforts to create digitally advanced health care systems. However, there is as yet no consensus on how best to characterize and assess digital excellence in hospitals. Objective: Our aim was to develop an international agreement on a defined set of technological capabilities to assess digital excellence in hospitals. Methods: We conducted a two-stage international modified electronic Delphi (eDelphi) consensus-building exercise, which included a qualitative analysis of free-text responses. In total, 31 international health informatics experts participated, representing clinical, academic, public, and vendor organizations. Results: We identified 35 technological capabilities that indicate digital excellence in hospitals. These are divided into two categories: (a) capabilities within a hospital (n=20) and (b) capabilities enabling communication with other parts of the health and social care system, and with patients and carers (n=15). The analysis of free-text responses pointed to the importance of nontechnological aspects of digitally enabled change, including social and organizational factors. Examples included an institutional culture characterized by a willingness to transform established ways of working and openness to risk-taking. The availability of a range of skills within digitization teams, including technological, project management and business expertise, and availability of resources to support hospital staff, were also highlighted. Conclusions: We have identified a set of criteria for assessing digital excellence in hospitals. Our findings highlight the need to broaden the focus from technical functionalities to wider digital transformation capabilities