Effect of predefined order sets and usability problems on efficiency of computerized medication ordering

Objectives: To study the effect of predefined order sets on the efficiency of computerized medication ordering, and to analyze the effect of different types of usability problems on ordering efficiency. Methods: Crossover study to comparing the efficiency of two methods of ordering (with and without use of predefined order sets) in a laboratory setting using a computerized physician order entry system (CPOE). The excess number of mouse clicks and keystrokes (the difference in number of mouse clicks and keystrokes needed by each physician and the minimally required numbers to accomplish the ordering tasks) for each method was measured and per physician, occurrences of usability problems during the task sessions were recorded. Observed usability problems were categorized using Zhang et al.’s heuristic principles of good user interface design. The effect of different types of usability problems on the excess number of mouse clicks and keystrokes was statistically analyzed. Results: The median excess number of mouse clicks and keystrokes needed by physicians was 6.2 times lower in the method with predefined order sets (p < 0.01). The excess number of mouse clicks and keystrokes was significantly increased by vague and erroneous system messages with a factor of 2.62 (95% CI 2.24–3.07), the use of unfamiliar language and terminology by a factor of 1.28 (95% CI 1.14–1.43), and non-informative system feedback by a factor of 1.15 (95% CI 1.03–1.28), respectively. Other categories of usability problems had little influence on ordering efficiency. Conclusions: Predefined order sets can improve the efficiency of computerized ordering by reducing the excess number of mouse clicks and keystrokes. However, the efficiency of computerized ordering can be significantly impaired by usability problems due to vague and incorrect system messages, unfamiliar language, and non-informative system feedback

Master of Science

thesisElectronic Health Record (EHR) adoption rates have been low in the United States. A key reason for this low adoption rate is poor EHR usability. Currently no standards exist for design, testing and monitoring the usability of EHRs. Therefore, we conducted a usability evaluation of a vendor's product in the Emergency Department at the University of Utah. In the first objective of this study, we evaluated a newly implemented computerized provider order entry application. Four usability experts used the Zhang et al 14 heuristics and 23 predefined tasks to perform the evaluation. The experts found 48 usability problems categorized into 51 heuristic violations. There were 4 cosmetic, 120 minor, 64 major, and 4 catastrophic problems identified. The interrater reliability was 0.81 using Fleis' Kappa, showing a high level of consistency in ratings across evaluators. For the second objective, we used an electronic version of Questionnaire of User Interaction Satisfaction (QUIS 7.0) to evaluate physician satisfaction with the CPOE application in the ED. The physician response rate was 50% (25/50). The total survey mean was 4.87, lower than the -a priori‖ definition for acceptable satisfaction score of 5.0 (of a possible 9). The lowest scale scores were for overall user reaction and learning iv and the highest were for screen, terminology and system capabilities. Further analyses were completed to determine any differences for satisfaction scores between physician trainees and attending. A multifactor ANOVA was performed to examine the combined effect of the different experience levels and sections of the QUIS. The results were significant at -1.43 (p < 0.05) for screen and terminology and system capabilities. In this setting, the ED CPOE application had a high level of usability issues and low mean satisfaction scores among physician end-users. The responsibility for improved usability lies with both vendors developing the product and facilities implementing the product and both should be educated on usability principles. The combination of a user-based and expert-based inspection method yielded congruent findings and was an accurate and efficient means of evaluation

Impact of computerized order sets on practitioner performance.

Order sets have the potential to provide evidence at the point of care and improve healthcare practice. In this study we reviewed the literature to assess the effect of computerized order sets on practitioner performance. Our search in PubMed and Science direct identified 442 studies of which 16 met our inclusion criteria. In 15 studies order sets contributed to the improvement of at least one performance outcome. The effect of order sets was evaluated on 34 performance outcomes, of which 19 were improved and the rest remained unchanged. The results provide evidence that the use of order sets can improve health care practice

A Fit between Clinical Workflow and Health Care Information Systems: Not waiting for Godot but making the journey

Health care has long suffered from inefficiencies due to the fragmentation of patient care information and the lack of coordination between health professionals [1]. Health care information systems (HISs) have been lauded as tools to remedy such inefficiencies [2, 3]. The primary idea behind the support of their implementation in health care is that these systems support clinical workflow and thereby decrease medical errors [2]. However, their introduction to health care settings have been accompanied by a transformation of the way their primary users, care providers, carry out clinical tasks and establish or maintain work relationships [4]. Studies have shown that these transformations have not always been productive [5, 6]

The Impact of CPOE Medication Systems’ Design Aspects on Usability, Workflow and Medication Orders

Objectives: To examine the impact of design aspects of computerized physician order entry (CPOE) systems for medication ordering on usability, physicians’ workflow and on medication orders. Methods: We systematically searched Pub - Med, EMBASE and Ovid MEDLINE for articles published from 1986 to 2007. We also evaluated reference lists of reviews and relevant articles captured by our search strategy, and the web-based inventory of evaluation studies in medical informatics 1982–2005. Data about design aspects were extracted from the relevant articles. Identified design aspects were categorized in groups derived from principles for computer screen and dialogue design and user guidance from the International Stan-dard Organization, and if CPOE-specific, from the collected data. Results: A total of 19 papers met our inclusion criteria. Sixteen studies used qualitative evaluation methods and the rest both qualitative and quantitative. In total 42 CPOE design aspects were identified and categorized in seven groups: 1) documentation and data entry components, 2) alerting, 3) visual clues and icons, 4) drop-down lists and menus, 5) safeguards, 6) screen displays, and 7) auxiliary functions. Conclusions: Beside the range of functionalities provided by a CPOE system, their subtle design is important to increase physicians’ adoption and to reduce medication errors. This requires continuous evaluations to investigate whether interfaces of CPOE systems follow normal flow of actions in the ordering process and if they are cognitively easy to understand and use for physicians. This paper provides general recommendations for CPOE (re)design based on the characteristics of CPOE design aspects found

COMBINING HUMAN FACTORS AND DATA SCIENCE METHODS TO EVALUATE THE USE OF FREE TEXT COMMUNICATION ORDERS IN ELECTRONIC HEALTH RECORDS

Medication errors are a leading cause of death in the United States. Electronic Health Records (EHR) along with Computerized Provider Order Entry (CPOE) are considered promising ways to reduce these errors. However, EHR systems have not eliminated medication errors. Moreover, in some cases they have facilitated errors due to issues such as poor usability and negative effects on clinical workflows. The use of unexpected free text within a CPOE system can serve as a marker that the system does not adequately support clinical workflow. Prior studies have looked at the use of free text within medication orders, but the inclusion of medication related information in communication for non-medication orders (CNMOs), a type of free text order, has not been adequately studied. This mixed-methods study identified the prevalence, nature and reasons for the inclusion of medication related information in CNMOs using a large sample of CNMOs placed at a mid-Atlantic hospital system in 2017, and via interviews with physicians. The study found that more than 42% of CNMOs contain medication related information. Moreover, the use of CNMOs varied significantly across provider types, hospital locations, patient settings and other factors. The study found 10 themes that might cause providers to adopt such workarounds, including missing functionality and poor usability. The viii study also identified several general challenges in communicating medication information in the EHR, and potential solutions to mitigate these challenges. This dissertation also demonstrates how natural language processing could be used to identify medication related CNMOs

An investigation of healthcare professionals’ experiences of training and using electronic prescribing systems: four literature reviews and two qualitative studies undertaken in the UK hospital context

Electronic prescribing (ePrescribing) is the process of ordering medicines electronically for a patient and has been associated with reduced medication errors and improved patient safety. However, these systems have also been associated with unintended adverse consequences. There is a lack of published research about users’ experiences of these systems in UK hospitals. The aim of this research was therefore to firstly describe the literature pertaining to the recent developments and persisting issues with ePrescribing and clinical decision support systems (CDS) (chapter 2). Two further systematic literature reviews (chapters 3 and 4) were then conducted to understand the unintended consequences of ePrescribing and clinical decision support (CDS) systems across both adult and paediatric patients. These revealed a taxonomy of factors, which have contributed to errors during use of these systems e.g., the screen layout, default settings and inappropriate drug-dosage support. The researcher then conducted a qualitative study (chapters 7-10) to explore users’ experiences of using and being trained to use ePrescribing systems. This study involved conducting semi-structured interviews and observations, which revealed key challenges facing users, including issues with using the ‘Medication List’ and how information was presented. Users experienced benefits and challenges when customising the system, including the screen display; however, the process was sometimes overly complex. Users also described the benefits and challenges associated with different forms of interruptive and passive CDS. Order sets, for instance, encouraged more efficient prescribing, yet users often found them difficult to find within the system. A lack of training resulted in users failing to use all features of the ePrescribing system and left some healthcare staff feeling underprepared for using the system in their role. A further literature review (chapter 5) was then performed to complement emerging themes relating to how users were trained to use ePrescribing systems, which were generated as part of a qualitative study. This review revealed the range of approaches used to train users and the need for further research in this area. The literature review and qualitative study-based findings led to a follow-on study (chapter 10), whereby the researcher conducted semi-structured interviews to examine how users were trained to use ePrescribing systems across four NHS Hospital Trusts. A range of approaches were used to train users; tailored training, using clinically specific scenarios or matching the user’s profession to that of the trainer were preferred over lectures and e-learning may offer an efficient way of training large numbers of staff. However, further research is needed to investigate this and whether alternative approaches such as the use of students as trainers could be useful. This programme of work revealed the importance of human factors and user involvement in the design and ongoing development of ePrescribing systems. Training also played a role in users’ experiences of using the system and hospitals should carefully consider the training approaches used. This thesis provides recommendations gathered from the literature and primary data collection that can help inform organisations, system developers and further research in this area

ارزیابی مشکلات تعاملی کاربران با سیستم اطلاعات فیزیوتراپی

زمینه و هدف:سیستم اطلاعات فیزیوتراپی یکی از سیستمهای اطلاعات بهداشتی است که برای افزایش کارآیی و اثربخشی خدمات این بخش بکار می رود. مطالعات نشان داده اند که تعامل کاربران با برخی از سیستم های اطلاعاتی به دلیل مشکلات کاربردپذیری دشوار است. هدف این مطالعه ارزیابی مشکلات تعاملی کاربران با سیستم اطلاعات فیزیوتراپی می باشد. مواد و روش ها:این مطالعه یک پژوهش توصیفی˗ مقطعی است که در آن با استفاده از روش ارزیابی هیورستیک مشکلات تعاملی کاربران سیستم اطلاعات فیزیوتراپی مورد استفاده در 105 بیمارستان کشور توسط سه ارزیاب شناسایی، طبقه بندی و تعیین شدت شدند. داده ها با استفاده از یک فرم گردآوری طراحی شده در نرم افزار اکسل گردآوری شدند. نتایج: در این مطالعه 82 مشکل تعاملی منفرد شناسایی گردید که بیشترین آنها با 28% (23n=) مربوط به مغایرت طراحی سیستم با اصول رایج در دنیای واقعی با میانگین شدت بزرگ بود. کمترین آنها مربوط به مشکلات عدم وجود قسمت راهنمایی و مستندسازی و جلوگیری از خطا در سیستم به ترتیب با میانگین شدت بزرگ و کوچک بودند. نتیجه گیری: برخی از سیستم هایی که امروزه در بسیاری از مراکز بهداشتی درمانی مورد استفاده قرار می گیرند، از جمله سیستم مورد بررسی در این مطالعه، دارای مشکلات متعددی می باشند که تعامل کاربران با آنها را دچار مشکل کرده و می تواند باعث ایجاد خطا و آسیب به بیماران شوند. برای شناسایی و رفع این گونه مشکلات توصیه می شود ارزیابی کاربردپذیری این سیستم ها به صورت مستمر انجام شود تا باعث افزایش رضایتمندی کاربران، بهبود روندکاری و ارتقای امنیت بیماران شود