Evaluation of random forest and ensemble methods at predicting complications following cardiac surgery

Cardiac patients undergoing surgery face increased risk of postoperative complications, due to a combination of factors, including higher risk surgery, their age at time of surgery and the presence of co-morbid conditions. They will therefore require high levels of care and clinical resources throughout their perioperative journey (i.e. before, during and after surgery). Although surgical mortality rates in the UK have remained low, postoperative complications on the other hand are common and can have a significant impact on patients’ quality of life, increase hospital length of stay and healthcare costs. In this study we used and compared several machine learning methods – random forest, AdaBoost, gradient boosting model and stacking – to predict severe postoperative complications after cardiac surgery based on preoperative variables obtained from a surgical database of a large acute care hospital in Scotland. Our results show that AdaBoost has the best overall performance (AUC = 0.731), and also outperforms EuroSCORE and EuroSCORE II in other studies predicting postoperative complications. Random forest (Sensitivity = 0.852, negative predictive value = 0.923), however, and gradient boosting model (Sensitivity = 0.875 and negative predictive value = 0.920) have the best performance at predicting severe postoperative complications based on sensitivity and negative predictive value

Implementation factors affecting the large-scale deployment of digital health and well-being technologies : a qualitative study of the initial phases of the ‘Living-It-Up’programme

Little is known about the factors which facilitate or impede the large-scale deployment of health and well-being consumer technologies. The Living-It-Up project is a large-scale digital intervention led by NHS 24, aiming to transform health and well-being services delivery throughout Scotland. We conducted a qualitative study of the factors affecting the implementation and deployment of the Living-It-Up services. We collected a range of data during the initial phase of deployment, including semi-structured interviews (N = 6); participant observation sessions (N = 5) and meetings with key stakeholders (N = 3). We used the Normalisation Process Theory as an explanatory framework to interpret the social processes at play during the initial phases of deployment.Initial findings illustrate that it is clear - and perhaps not surprising - that the size and diversity of the Living-It-Up consortium made implementation processes more complex within a 'multi-stakeholder' environment. To overcome these barriers, there is a need to clearly define roles, tasks and responsibilities among the consortium partners. Furthermore, varying levels of expectations and requirements, as well as diverse cultures and ways of working, must be effectively managed. Factors which facilitated implementation included extensive stakeholder engagement, such as co-design activities, which can contribute to an increased 'buy-in' from users in the long term. An important lesson from the Living-It-Up initiative is that attempting to co-design innovative digital services, but at the same time, recruiting large numbers of users is likely to generate conflicting implementation priorities which hinder - or at least substantially slow down - the effective rollout of services at scale.The deployment of Living-It-Up services is ongoing, but our results to date suggest that - in order to be successful - the roll-out of digital health and well-being technologies at scale requires a delicate and pragmatic trade-off between co-design activities, the development of innovative services and the efforts allocated to widespread marketing and recruitment initiatives

Haemodialysis Electronic Patient Portal: A Design Requirements Analysis and Feasibility Study with Domain Experts

In 2013, the UK national renal registry established 57,000 adults in the UK were treated for advanced kidney failure, 23,683 (42%) receiving haemodialysis. Haemodialysis patients face some of the highest treatment burden in the National Health Service (NHS) and are among the most 'expensive' to treat. In addition, patients endure complex treatment trajectories. In this study, we have sought to gather and synthesise the opinion of clinical and Human Computer Interaction (HCI) domain experts (n=9) to establish a set of initial design requirements in order to test the feasibility of developing a digital aid (i.e. electronic haemodialysis patient portal) to support patients in the course of their treatment. Expert feedback was gathered by means of interviews and focus groups in order to instruct design requirements for a haemodialysis patient portal

Optimising ePrescribing in hospitals through the interoperability of systems and processes:a qualitative study in the UK, US, Norway and the Netherlands

Abstract Background Investment in the implementation of hospital ePrescribing systems has been a priority in many economically-developed countries in order to modernise the delivery of healthcare. However, maximum gains in the safety, quality and efficiency of care are unlikely to be fully realised unless ePrescribing systems are further optimised in a local context. Typical barriers to optimal use are often encountered in relation to a lack of systemic capacity and preparedness to meet various levels of interoperability requirements, including at the data, systems and services levels. This lack of systemic interoperability may in turn limit the opportunities and benefits potentially arising from implementing novel digital heath systems. Methods We undertook n = 54 qualitative interviews with key stakeholders at nine digitally advanced hospital sites across the UK, US, Norway and the Netherlands. We included hospitals featuring ‘standalone, best of breed’ systems, which were interfaced locally, and multi-component and integrated electronic health record enterprise systems. We analysed the data inductively, looking at strategies and constraints for ePrescribing interoperability within and beyond hospital systems. Results Our thematic analysis identified 4 main drivers for increasing ePrescribing systems interoperability: (1) improving patient safety (2) improving integration & continuity of care (3) optimising care pathways and providing tailored decision support to meet local and contextualised care priorities and (4) to enable full patient care services interoperability in a variety of settings and contexts. These 4 interoperability dimensions were not always pursued equally at each implementation site, and these were often dependent on the specific national, policy, organisational or technical contexts of the ePrescribing implementations. Safety and efficiency objectives drove optimisation targeted at infrastructure and governance at all levels. Constraints to interoperability came from factors such as legacy systems, but barriers to interoperability of processes came from system capability, hospital policy and staff culture. Conclusions Achieving interoperability is key in making ePrescribing systems both safe and useable. Data resources exist at macro, meso and micro levels, as do the governance interventions necessary to achieve system interoperability. Strategic objectives, most notably improved safety, often motivated hospitals to push for evolution across the entire data architecture of which they formed a part. However, hospitals negotiated this terrain with varying degrees of centralised coordination. Hospitals were heavily reliant on staff buy-in to ensure that systems interoperability was built upon to achieve effective data sharing and use. Positive outcomes were founded on a culture of agreement about the usefulness of access by stakeholders, including prescribers, policymakers, vendors and lab technicians, which was reflected in an alignment of governance goals with system design

Evidence based evaluation of eHealth interventions: A systematic literature review

Background: Until now, the use of technology in health care was driven mostly by the assumptions about the benefits of electronic health (eHealth) rather than its evidence. It is noticeable that the magnitude of evidence of effectiveness and efficiency of eHealth is not proportionate to the number of interventions that are regularly conducted. Reliable evidence generated through comprehensive evaluation of eHealth interventions may accelerate the growth of eHealth for long-term successful implementation and help to experience eHealth benefits in an enhanced way.Objective: This study aimed to understand how the evidence of effectiveness and efficiency of eHealth can be generated through evaluation. Hence, we aim to discern (1) how evaluation is conducted in distinct eHealth intervention phases, (2) the aspects of effectiveness and efficiency that are typically evaluated during eHealth interventions, and (3) how eHealth interventions are evaluated in practice.Methods: A systematic literature review was conducted to explore the evaluation methods for eHealth interventions. Preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines were followed. We searched Google Scholar and Scopus for the published papers that addressed the evaluation of eHealth or described an eHealth intervention study. A qualitative analysis of the selected papers was conducted in several steps.Results: We intended to see how the process of evaluation unfolds in distinct phases of an eHealth intervention. We revealed that in practice and in several conceptual papers, evaluation is performed at the end of the intervention. There are some studies that discuss the importance of conducting evaluation throughout the intervention; however, in practice, we found no case study that followed this. For our second research question, we discovered aspects of efficiency and effectiveness that are proposed to be assessed during interventions. The aspects that were recurrent in the conceptual papers include clinical, human and social, organizational, technological, cost, ethical and legal, and transferability. However, the case studies reviewed only evaluate the clinical and human and social aspects. At the end of the paper, we discussed a novel approach to look into the evaluation. Our intention was to stir up a discussion around this approach with the hope that it might be able to gather evidence in a comprehensive and credible way.Conclusions: The importance of evidence in eHealth has not been discussed as rigorously as have the diverse evaluation approaches and evaluation frameworks. Further research directed toward evidence-based evaluation can not only improve the quality of intervention studies but also facilitate successful long-term implementation of eHealth in general. We conclude that the development of more robust and comprehensive evaluation of eHealth studies or an improved validation of evaluation methods could ease the transferability of results among similar studies. Thus, the resources can be used for supplementary research in eHealth

Researching big IT in the UK National Health Service:A systematic review of theory-based studies

ObjectiveTo identify and discuss theory-based studies of large-scale health information technology programs in the UK National Health Service.Materials and MethodsUsing the PRISMA systematic review framework, we searched Scopus, PubMed and CINAHL databases from inception to March 2022 for theory-based studies of large-scale health IT implementations. We undertook detailed full-text analyses of papers meeting our inclusion criteria.ResultsForty-six studies were included after assessment for eligibility, of which twenty-five applied theories from the information systems arena (socio-technical approaches, normalization process theory, user acceptance theories, diffusion of innovation), twelve from sociology (structuration theory, actor-network theory, institutional theory), while nine adopted other theories. Most investigated England’s National Program for IT (2002–2011), exploring various technologies among which electronic records predominated. Research themes were categorized into user factors, program factors, process outcomes, clinical impact, technology, and organizational factors. Most research was qualitative, often using a case study strategy with a longitudinal or cross-sectional approach. Data were typically collected through interviews, observation, and document analysis; sampling was generally purposive; and most studies used thematic or related analyses. Theories were generally applied in a superficial or fragmentary manner; and articles frequently lacked detail on how theoretical constructs and relationships aided organization, analysis, and interpretation of data.ConclusionTheory-based studies of large NHS IT programs are relatively uncommon. As large healthcare programs evolve over a long timeframe in complex and dynamic environments, wider adoption of theory-based methods could strengthen the explanatory and predictive utility of research findings across multiple evaluation studies. Our review has confirmed earlier suggestions for theory selection, and we suggest there is scope for more explicit use of such theoretical constructs to strengthen the conceptual foundations of health informatics research. Additionally, the challenges of large national health informatics programs afford wide-ranging opportunities to test, refine, and adapt sociological and information systems theories

Success and failure in eHealth

Introduction In the field of eHealth, there seems to be a gap between promising research and clinical reality. This master thesis aims to give insight in patterns that can be found regarding the possible outcome in terms of success and/or failure. An in-depth review of workflow will be done, to get an understanding of the implications of eHealth on workflow. Methods Using a systematic article search, papers have been collected regarding the subject of this thesis. Through multiple search strategies, one final search string has been formulated. This final search string led to 903 papers. These papers have been assessed on relevance using qualitative methods. This resulted in 258 papers, which have been categorised by topic, entity and success or failure. After categorisation, the topic of workflow has been selected for an additional in-depth full-text review. Results The categorisation led to 27 categories. The categories are separated among the following entities: patient, health professional, health system and all. The first three have been separated in terms of success and failure as well. This led to a quantitative overview of different categories, for different actors in terms of success and failure. Workflow appeared to be essential for the possible success or failure of eHealth implementations. It is important to include workflow in the design of the tool as well. Conclusion Different categories show a unique combination in success and failure, and to what entity they belong. The category costs appeared to be mostly based on the health system and is attributed to failure. Therefore it is a pre-requisite for the implementation of eHealth. Other categories like quality healthcare and user expectations seem to target on success. The category legal was smaller than anticipated, which could have been caused by categories that are closely linked to each other

Paediatric Surgical Outcomes Across Ontario

BACKGROUND: The purpose of this study was to determine the volume of three complex paediatric procedures at the tertiary care centres in Ontario over the last 20 years in three disciplines, analyze outcomes and explore the possibility of an outcome volume relationship. METHODS: A population based cohort study of patients undergoing TEF repair, Pyeloplasty and V-P shunt insertion at 4 paediatric centres in Ontario between 1993 and 2013 was carried out. Administrative data sets were linked using unique encoded identifiers and analyzed at the Institute for Clinical Evaluative Sciences (ICES). RESULTS: In TEF repair and Pyeloplasty, there was significant difference in hospital volume between institutions and no significant difference in main outcome–reoperation rate between them. The data for VPS need to be completed. CONCLUSIONS: This study demonstrated that although the institutional volume and surgeon volume is different, the reoperation rate between institutions is similar in tertiary paediatric Ontario centres

Implementation of an integrated preoperative care pathway and regional electronic clinical portal for preoperative assessment

Background: Effective surgical pre-assessment will depend upon the collection of relevant medical information, good data management and communication between the members of the preoperative multi-disciplinary team. NHS Greater Glasgow and Clyde has implemented an electronic preoperative integrated care pathway (eForm) allowing all hospitals to access a comprehensive patient medical history via a clinical portal on the health-board intranet. Methods: We conducted six face-to-face semi-structured interviews and participated in one focus group and two workshops with key stakeholders involved in the Planned Care Improvement (PCIP) and Electronic Patient Record programmes. We used qualitative methods and Normalisation Process Theory in order to identify the key factors which led to the successful deployment of the preoperative eForm in the health-board. Results: In January 2013, more than 90,000 patient preoperative assessments had been completed via the electronic portal. Two complementary strategic efforts were instrumental in the successful deployment of the preoperative eForm. At the local health-board level: the PCIP led to the rationalisation of surgical pre-assessment clinics and the standardisation of preoperative processes. At the national level: the eHealth programme selected portal technology as an iterative strategic technology solution towards a virtual electronic patient record. Our study has highlighted clear synergies between these two standardisation efforts. Conclusion: The adoption of the eForm into routine preoperative work practices can be attributed to: (i) a policy context – including performance targets – promoting the rationalisation of surgical pre-assessment pathways, (ii) financial and organisational resources to support service redesign and the use of information technology for operationalising the standardisation of preoperative processes, (iii) a sustained engagement with stakeholders throughout the iterative phases of the preoperative clinics redesign, guidelines standardisation and the eForm development, (iv) the use of a pragmatic and domain-agnostic technology solution and finally: (v) a consensual and contextualised implementation

A Feasibility Study to Measure the Efficacy of a Pediatric Anesthesiology Preoperative Clinic

Introduction: Preoperative assessments by anesthesiologists have the potential to improve the care of pediatric surgical patients. At UNC, the PreCare clinic is being piloted to provide this assessment service. To facilitate its ongoing development, I developed a feasibility study involving stakeholder interviews and preoperative tasks assessments to determine how to measure the efficacy of the PreCare clinic. Methods: I reviewed patient charts and EMR messages to identify tasks completed by the clinic, and stratified tasks based on the type of patient evaluation being conducted by the clinic. Key stakeholders, including PreCare staff and referring surgeons, were asked to participate in open-ended, in-depth interviews to determine their professional opinions of the clinic. Results: I encountered significant barriers in the preoperative task assessment due to inconsistent documentation in the EMR. However, the charts of the patients seen at the clinic had clear documentation showing that patient education and preoperative testing were consistently completed at the clinic. Stakeholder interviews revealed that the goals of the clinic are to improve preoperative management and reduce family anxiety. However, the actual functions of the clinic are more complex, and it still suffers from a number of logistic growing pains. Stakeholders indicated that measuring surgical delays and cancellations (and a number of secondary measures) was the best way to evaluate the clinic. Discussion: Based on the collected data, I concluded that a study measuring the secondary outcomes of surgical delays and cancellations is the best way to measure the efficacy of the PreCare clinic. Due to sparse EMR documentation and the small scale of the clinic, this study will need to rely on data generated for research purposes and will need to use a single-subject study design so that participants can act as their own controls.Master of Public Healt