CPOE System Design Aspects and Their Qualitative Effect on Usability

Although many studies have discussed the benefits of Computerized Provider Order Entry (CPOE) systems, their configuration can have a great impact on clinicians’ adoption of these systems. Poorly designed CPOE systems can lead to usability problems, users’ dissatisfaction and may disrupt normal flow of clinical activities. This paper reports on a literature review focused on the identification of CPOE medication systems’ design aspects that impact CPOE systems’ usability and create opportunities for medication errors. Our review is based on a systematic literature search in PubMed, EMBASE and Ovid MEDLINE for relevant publications from 1986-2006. We categorized the design aspects extracted from relevant publications into six different groups: 1) timing of alerts, 2) log in/out procedures, 3) pick lists and drop down menus, 4) clues and guidelines, 5) documentation and data entry options, and 6) screen display and layout. Our review shows that the manner in which a CPOE system is configured can have a high impact on ease of system use, task behavior of clinicians in ordering drugs, and medication errors. Characterization of consequences associated with certain CPOE design aspects provides insight into how CPOE system designs can be improved to enhance physicians’ adoption of these systems and their success. Recommendations are provided to enable CPOE system designers to create CPOE systems that are not only more user friendly and efficient but safer

Usability Evaluation of a Computerized Physician Order Entry for Medication Ordering

Despite CPOE (Computerized Physician Order Entry) systems’ potential to enhance patient safety by reducing medication errors, recent studies have cast some doubts on their role in error reduction. CPOE systems with poorly designed interfaces have proven to cause users dissatisfaction and to introduce new kind of errors in the ordering process, suggesting a threat instead of an enhancement of patient safety. The main objective of this study is to identify usability problems related to a CPOE medication system’s design and determining their severities. Two experts completed a cognitive walkthrough (CW) of an ordering task based on a clinical scenario for ordering the consolidation phase of chemotherapy for a leukemic patient. Fifty five usability problems were found and classified into eleven categories. CW identified cosmetic to catastrophic problems leading to inefficient use of the CPOE system and potentially resulting in users’ confusion, longer ordering duration, and medication errors. The complexity of the CPOE design, its rigidness and lack of user guidance suggests the necessity to redesign the current user interface in order to match clinicians’ ordering behaviors and to fully support them in the medication ordering process

A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence

OBJECTIVE: Usability evaluation is now widely recognized as critical to the success of interactive health care applications. However, the broad range of usability inspection and testing methods available may make it difficult to decide on a usability assessment plan. To guide novices in the human-computer interaction field, we provide an overview of the methodological and empirical research available on the three usability inspection and testing methods most often used. METHODS: We describe two 'expert-based' and one 'user-based' usability method: (1) the heuristic evaluation, (2) the cognitive walkthrough, and (3) the think aloud. RESULTS: All three usability evaluation methods are applied in laboratory settings. Heuristic evaluation is a relatively efficient usability evaluation method with a high benefit-cost ratio, but requires high skills and usability experience of the evaluators to produce reliable results. The cognitive walkthrough is a more structured approach than the heuristic evaluation with a stronger focus on the learnability of a computer application. Major drawbacks of the cognitive walkthrough are the required level of detail of task and user background descriptions for an adequate application of the latest version of the technique. The think aloud is a very direct method to gain deep insight in the problems end users encounter in interaction with a system but data analyses is extensive and requires a high level of expertise both in the cognitive ergonomics and in computer system application domain. DISCUSSION AND CONCLUSIONS: Each of the three usability evaluation methods has shown its usefulness, has its own advantages and disadvantages; no single method has revealed any significant results indicating that it is singularly effective in all circumstances. A combination of different techniques that compliment one another should preferably be used as their collective application will be more powerful than applied in isolation. Innovative mobile and automated solutions to support end-user testing have emerged making combined approaches of laboratory, field and remote usability evaluations of new health care applications more feasibl

The redesign of the medical informatics master of science course at the University of Amsterdam

The University of Amsterdam redesigned its former 4 years Medical Informatics university program into a Dutch 3 years BSc program and a 2 years English MSc program. The new MSc program is aimed at (international) baccalaureates in medical informatics, computer science, medicine, health sciences, and biology. Besides, health care professionals or professionals with a background in computer science may enter the program. We present our new MSc program shortl

The new set-up of the medical informatics Master of Science program at the University of Amsterdam

OBJECTIVES: To describe the new set-up of our Master of Science program in medical informatics that started in September 2006 at the University of Amsterdam-Academic Medical Center, The Netherlands. METHODS: To harmonize with the Bologna declaration, we transformed our former medical informatics program from a 4-year course into a 3-year bachelor and a 2-year (English) Master of Science Medical Informatics program. We describe the objectives, organizational structure and contents of this new medical informatics master program. RESULTS: The new master program now is aimed at (international) baccalaureates in medical informatics, computer science, medicine and other health-related studies and professionals from these disciplines. The master course comprises four study semesters of 30 EC each, equal to 120 EC in total. Central to the program is the context of hospital organization, encompassing medical practice and patient care, information and communication technologies and (logistic) structuring of health care and health care processes. The program comprises core education (60 EC), discipline-related in-depth-study (12 EC) and an academic work placement (48 EC). Students from a range of other disciplines will be admitted to the master program after successful completion of an individualized conversion program (a maximum of 30 EC). CONCLUSIONS: With the new set-up of our master course and by offering individualized conversion programs, we hope to both accommodate the learning needs of our own medical informatics baccalaureates and to attract other (international) students and professionals to our new program. Our ultimate aim is to bring forth medical informatics specialists who can make significant contributions to the fiel

The significance of a usability evaluation of an emerging laboratory order entry system

OBJECTIVES: To assess the usability of an emerging POE system, OM/Lab, for the computer-supported ordering of laboratory tests. We were more specifically interested in the relation of the usability problems detected in the user testing sessions with the order behaviour in terms of efficiency and errors in ordering. METHODS: A cognitive walkthrough of the OM/Lab system was conducted by two analysts using four real-life scenarios for ordering laboratory tests, which were reviewed for comprehensiveness by an expert clinician. Thereafter, the OM/Lab system was evaluated on its usability in testing sessions with seven potential end-users of the system performing these same four scenarios. The results of these end-user testing sessions were used to analyze the effect of usability flaws on the quality of ordering in terms of omissions, errors in orders and cancelled orders. RESULTS: The analyses revealed a total of 33 usability problems, of which 25 problems were revealed both by the cognitive walkthrough and in the end-user sessions. These 25 usability problems indeed led to inefficient order behaviour, omissions and errors in orders and even to cancelled orders. DISCUSSION: Our results revealed that the OM/Lab system suffered from a high number of usability flaws. The interface design flaws were, among other things, related to misallocation of buttons on the screen, incomprehensibility of button labels and feedback containing no relevant information to the user about the cause of errors made and consequences of a user's action. Additionally, our user test session results indicated that the OM/Lab system also suffered from user interaction problems of a more socio-technical nature. These sessions revealed, among other things, that the more specific action sequences to be executed within the ordering model of the OM/Lab system did not correspond to the daily working routines of end-users and that the grouping of laboratory tests within clusters did not match the paper-based order forms. The seemingly negative effects of these usability flaws on the quality of ordering and the inefficiency of work processes during the pilot implementation finally lead to withdrawal of the OM/Lab system from clinical practice. Though the system implementation failed, our usability study brought research on (re)designing and evaluating clinical computer applications at a higher status in our institution. It is now recognized that usability evaluation studies that will support good quality of clinical practice are highly importan

Usability evaluation of a laboratory order entry system: cognitive walkthrough and think aloud combined

Though designing physician order entry (POE) systems has been a highly discussed research topic in the last decade, it seems that engineering POE systems that truly optimise the quality and efficiency of ordering is still a challenge. This study addresses a usability evaluation of an emerging POE for the electronically ordering of laboratory tests. By applying two complementary cognitive approaches, specifically the cognitive walkthrough and think aloud method with seven potential end users of the system, we analyzed usability problems in the prototype system and their potential effect on the quality of orders in terms of efficiency and errors in ordering. The cognitive walkthrough provided a coding scheme that was used to analyse in more detail usability errors encountered during the think aloud tests with the seven end users. The analyses revealed a total of 33 usability problems, which indeed led to inefficiency, omissions in ordering and even to cancelled orders. Most of these usability problems referred to incomprehensiveness of required actions by the user and incomprehensiveness of text used in the system. Next to the discussion of the reasons for these usability problems, the surplus value of the think aloud method as supplementary to the cognitive walkthrough in evaluating physician order entry systems is outlined in this pape

Considerate mHealth design for older adults with Alzheimer's disease and related dementias (ADRD): A scoping review on usability barriers and design suggestions

Background: The number of older adults with Alzheimer's disease and related dementias (ADRD) is increasing worldwide. This offers ample opportunities for mobile health (mHealth) apps, for example to support them in performing daily activities or monitoring their health status. However, how to design these apps taking into consideration ADRD related barriers remains a challenge. Objective: To provide a synthesis of mHealth usability barriers of older adults with ADRD and design suggestions to enhance user-friendliness of mHealth apps. Methods: We performed a scoping review of mHealth studies focused on older adults with ADRD and: (1) usability barriers experienced, and/or (2) implementation or design suggestions. PubMed, Medline, EMBASE, PsycINFO, and Web of Science were searched. Clinical and grey literature on ADRD diminishments was explored to identify potential barriers to using mHealth. Detected usability barriers caused by ADRD impairments were mapped onto the MOLD-US framework through validation, calibration, and adaption. MOLD-US was initially developed to provide an overview of barriers influencing mHealth usability for the general aging population. Results: Title and abstract of 792 unique citations were scanned of which 69 were included for full text review. Fifteen studies matched inclusion criteria. In total 42 barriers were identified to influence mobile health use for older adults with ADRD. Twenty design suggestions were extracted from the studies. Conclusions: The identified usability barriers were classified in five categories: cognition, perception, physical ability, frame of mind, and speech- and language. In addition, the design suggestions were categorized as evidence- or expert-based. Evidence-based design suggestions include showing limited information, repeating instructions multiple times and breaking instructions into simple steps given one at a time. This research provides a first step for further collaboration between ADRD experts and designers to support the development of effective mHealth apps with high user-friendliness