Usability analysis of contending electronic health record systems

In this paper, we report measured usability of two leading EHR systems during procurement. A total of 18 users participated in paired-usability testing of three scenarios: ordering and managing medications by an outpatient physician, medicine administration by an inpatient nurse and scheduling of appointments by nursing staff. Data for audio, screen capture, satisfaction rating, task success and errors made was collected during testing. We found a clear difference between the systems for percentage of successfully completed tasks, two different satisfaction measures and perceived learnability when looking at the results over all scenarios. We conclude that usability should be evaluated during procurement and the difference in usability between systems could be revealed even with fewer measures than were used in our study. © 2019 American Psychological Association Inc. All rights reserved.Peer reviewe

Electronic Health Record Optimization for Cardiac Care

Electronic health record (EHR) systems have been studied for over 30 years, and despite the benefits of information technology in other knowledge domains, progress has been slow in healthcare. A growing body of evidence suggests that dissatisfaction with EHR systems was not simply due to resistance to adoption of new technology but also due to real concerns about the adverse impact of EHRs on the delivery of patient care. Solutions for EHR improvement require an approach that combines an understanding of technology adoption with the complexity of the social and technical elements of the US healthcare system. Several studies are presented to clarify and propose a new framework to study EHR-provider interaction. Four focus areas were defined - workflow, communication, medical decision-making and patient care. Using Human Computer Interaction best practices, an EHR usability framework was designed to include a realistic clinical scenario, a cognitive walkthrough, a standardized simulated patient actor, and a portable usability lab. Cardiologists, fellows and nurse practitioners were invited to participate in a simulation to use their institution’s EHR system for a routine cardiac visit. Using a mixed methods approach, differences in satisfaction and effectiveness were identified. Cardiologists were dissatisfied with EHR functionality, and were critical of the potential impact of the communication of incorrect information, while displaying the highest level of success in completing the tasks. Fellows were slightly less dissatisfied with their EHR interaction, and demonstrated a preference for tools to improve workflow and support decision-making, and showed less success in completing the tasks in the scenario. Nurse practitioners were also dissatisfied with their EHR interaction, and cited poor organization of data, yet demonstrated more success than fellows in successful completion of tasks. Study results indicate that requirements for EHR functionality differ by type of provider. Cardiologists, cardiology fellows, and nurse practitioners required different levels of granularity of patient data for use in medical decision-making, defined different targets for communication, sought different solutions to workflow which included distribution of data input, and requested technical solutions to ensure valid and relevant patient data. These findings provide a foundation for future work to optimize EHR functionality

Examining perceptions of the usefulness and usability of a mobile-based system for pharmacogenomics clinical decision support: A mixed methods study

Background. Pharmacogenomic testing has the potential to improve the safety and efficacy of pharmacotherapy, but clinical application of pharmacogenetic knowledge has remained uncommon. Clinical Decision Support (CDS) systems could help overcome some of the barriers to clinical implementation. The aim of this study was to evaluate the perception and usability of a web- and mobile-enabled CDS system for pharmacogenetics-guided drug therapy-the Medication Safety Code (MSC) system-among potential users (i.e., physicians and pharmacists). Furthermore, this study sought to collect data on the practicability and comprehensibility of potential layouts of a proposed personalized pocket card that is intended to not only contain the machine-readable data for use with the MSC system but also humanreadable data on the patient's pharmacogenomic profile. Methods. We deployed an emergent mixed methods design encompassing (1) qualitative interviews with pharmacists and pharmacy students, (2) a survey among pharmacogenomics experts that included both qualitative and quantitative elements and (3) a quantitative survey among physicians and pharmacists. The interviews followed a semistructured guide including a hypothetical patient scenario that had to be solved by using the MSC system. The survey among pharmacogenomics experts focused on what information should be printed on the card and how this information should be arranged. Furthermore, the MSC system was evaluated based on two hypothetical patient scenarios and four follow-up questions on the perceived usability. The second survey assessed physicians' and pharmacists' attitude towards the MSC system. Results. In total, 101 physicians, pharmacists and PGx experts coming from various relevant fields evaluated the MSC system. Overall, the reaction to the MSC system was positive across all investigated parameters and among all user groups. The majority of participants were able to solve the patient scenarios based on the recommendations displayed on the MSC interface. A frequent request among participants was to provide specific listings of alternative drugs and concrete dosage instructions. Negligence of other patient-specific factors for choosing the right treatment such as renal function and co-medication was a common concern related to the MSC system, while data privacy and cost-benefit considerations emerged as the participants' major concerns regarding pharmacogenetic testing in general. The results of the card layout evaluation indicate that a gene-centered and tabulated presentation of the patient's pharmacogenomic profile is helpful and well-accepted. Conclusions. We found that the MSC system was well-received among the physicians and pharmacists included in this study. A personalized pocket card that lists a patient's metabolizer status along with critically affected drugs can alert physicians and pharmacists to the availability of essential therapy modifications

Strategies for Applying Electronic Health Records to Achieve Cost Saving Benefits

The American Recovery and Reinvestment Act (ARRA) of 2009 authorized the distribution of about $30 billion incentive funds to accelerate electronic health record (EHR) applications to improve the quality of care, safety, privacy, care coordination, and patients\u27 involvement in healthcare. EHR use has the potential of saving$731 in costs for hospitals per patient admission; however, most hospitals are not applying EHR to reach the level at which cost savings are possible. The purpose of this single case study was to explore strategies that IT leaders in hospitals can use to apply EHR to achieve the cost saving benefits. The participants were IT leaders and EHR super users at a large hospital in Texas with successful experience in applying EHR. Information systems success model formed the conceptual framework for the study. I conducted face-to-face interviews and analyzed organizational documents. I used qualitative textual data analysis method to identify themes. Five themes emerged from this study, which are ensuring information quality, ensuring system quality, assuring service quality, promoting usability, and maximizing net benefits of the EHR system. The findings of this study included four strategies to apply EHR; these strategies include engaging training staff, documenting accurately and in a timely manner, protecting patient data, and enforcing organizational best practice policies to maximize reimbursement and cost savings. The findings of this study could contribute to positive social change for the communities because EHR successful application includes lower cost for hospitals that may lead to the provision of affordable care to more low-income patients

Perception gaps and the adoption of information technology in the clinical healthcare environment

Implementation of information systems has lagged in many areas of clinical healthcare for a variety of reasons. Economics, data complexity and resistance are among the often quoted roadblocks. Research suggests that physicians play a major part in the adoption, use and diffusion of information technology (IT) in clinical settings. There are also other healthcare professionals, clinical and non-clinical, who play important roles in making decisions about the acquisition of information technology. In addition to these groups there are information technology professionals providing the services required within the healthcare field. Finally within this group are those IT professionals who have sufficient cross training to understand specific needs. Each member of these groups brings a different perspective to both needs assessments as well as implementation of clinical systems. This study considers the idea that there are preconceived differences of opinion of the information needs of clinical healthcare by the clinical community and the information technology professionals. Are these differences significant enough to create a barrier to implementation? A questionnaire was developed from preliminary data to assess multiple parameters which could impact implementation of a clinical information technology solution. A Web of System Performance (WOSP) model was created to map each of the following eight areas of concern: functionality, usability, extendibility, connectivity, flexibility, reliability, privacy and security. Responses to the questions were related to professional roles, age and experience. There were no differences seen in the perceived need for secure systems by either healthcare workers or IT professionals. The variance of perceived need was greatest among the various non-physician healthcare workers when compared to physicians or information technology professions. This was a consistent pattern for the otherparameters with the exception of the usability of the electronic health record. In this area all groups disagreed significantly. The study, though limited by its small sample, still suggests that the resistance by healthcare professionals is not a significant barrier to successful information technology implementation

Optimizing Electronic Medication Prior Authorization: Reducing Prescription Delays

Background: Within the United States, chronic disease in children has doubled over the last 20 years. Many diseases defined as chronic (attention deficit, epilepsy, and diabetes) require daily medication regimens for optimal management. To be covered by insurance, many of these medications require prior authorization (PA) from the patients’ pharmacy benefits policy. Delays in processing and receiving PA orders can lead to worsening disease and inadequate disease management. In 2014, a pediatric academic medical center in the Midwest found that processing medications from prescription order to PA approval took nurses an average of over 90 hours. In August 2020, the organization implemented an electronic prior authorization (ePA) system that interfaced with the organization’s electronic health record (EHR). The primary goals of this implementation were to reduce medication PA turnaround times and to increase employee engagement with the ePA system. The goals of this quality improvement (QI) project are to optimize the existing ePA system with the medication PA process to reduce average medication PA turnaround times and to increase the approval rates for medication PAs by five percent. Project Design: Three interventions support the outputs of this QI project. Increase the availability of the ePA system by changing the patient and pharmacy benefits insurance matching interface logic. Reduce the number of medications falsely requiring PA by removing them from the ePA system. Increase PA processing efficiency by improving the workflow for attaching documents required for PA approval. To accomplish and measure these interventions, data reports and surveys were developed to establish baselines and to measure ePA turnaround times, PA approval rates, and user satisfaction both pre- and post-intervention. User satisfaction was measured utilizing a secure online survey emailed to ambulatory division nurses. Results: The median medication ePA turnaround pre- and post-interventions was unchanged at 36 hours. The ePA approval percentage dropped from 55.7% in June 2021 to 46.9% in August 2021. The primary QI project outcomes of reducing turnaround time and increasing the approval rate by 5% were not met. A user involvement survey was sent to 194 nurses with a response rate of 29% pre intervention and 8% post intervention. Overall user satisfaction was measured using a net promotor score which registered scores of –70 pre- and –82 post-intervention, revealing overall dissatisfaction with the ePA system. The use of an alternative ePA system outside the organization’s EHR was discovered after the QI project data was reviewed and showed that roughly 45% of ePAs were completed using this alternative system during the QI project timeframe. Recommendations: User involvement surveys measure user engagement with electronic systems and measuring user satisfaction is beneficial to providing direction for interventions as well as predicting future utilization of healthcare informatics projects. Conclusion: Though most of the goals for this QI project outcome were not met, use of the alternative ePA system confirmed the Technology Acceptance Model that users prefer the electronic system that they perceive as being the most useful. Nurses using ePA will use the system that best addresses their own user experiences regarding content, accuracy, format, timeliness, ease of use, and overall satisfaction

Usability in healthcare : overcoming the mismatch between information systems and clinical work

Usability of clinical information technology (IT) systems is an ongoing topic of discussion. The systems should support healthcare professionals in their daily work with patients. However, critics indicate the prevalence of negative experiences and use related problems. The overall goal of the thesis is to examine the usability of current clinical IT systems from the viewpoint of physicians and nurses for the purposes of further user-centred system development. The thesis includes three empirical studies: a digital dictation study, evaluation of nursing documentation systems, and a national usability questionnaire study with physicians. The research was carried out utilizing contextual inquiry, interaction sequence illustration analysis, and tailored usability questionnaire methods. The research resulted in the following findings and conclusions. Currently used IT systems do not support the daily work and clinical tasks of clinicians well. This is due to numerous usability problems, and lack of computer support for multi-professional and cross-organizational collaboration between clinicians. Major improvements are needed to achieve the potential benefits clinical information and communication technology systems offer. Based on empirical studies, themes for potential improvements are: development of efficient and mobile documentation solutions, redesign of system user interfaces, solutions to support communication and collaboration, customisable and context-specific clinical IT systems, and conceptual redesign of nursing documentation system. In the field of health informatics, a need exists to broaden the scope of usability work. Usability is closely associated with evaluation and testing activities instead of design activities. Hence, the scope of usability is more restricted than it is in user-centred design and usability research fields. In order to overcome the current mismatch between IT systems and clinical work, it is important to understand that usability is extremely context-sensitive by nature. The study results indicated shortcomings in user-centred healthcare IT systems design and end-users' abilities to contribute to development work. User-centred design methods provide a variety of means to analyse, design, and evaluate information and communication systems for clinical purposes. However, the characteristics of the clinical context (e.g. privacy and data security issues, and the wide range of IT systems in use) need to be taken into account when applying the methods and performing research in real-life clinical surroundings

Holistic System Design for Distributed National eHealth Services

publishedVersio

A process model for quality in use evaluation on clinical decision support systems

Developing or purchasing software is an expensive investment and needs to be justified. Furthermore, the software must be useful in its purpose, reliable, efficient and, among other characteristics, meet the expectations of users [1, 2]. It would be no different in the case of a clinical Decision Support System - CDSS. CDSS are systems developed to support clinicians and other health professionals in a medical decision making [3]. They are developed within a clinical context, following medical guidelines, with varied purposes such as diagnoses [4, 5, 6] patient monitoring [7, 8, 9], prevention [10] and disease treatment [11, 12]. Conversely, even with all the benefits offered by a CDSS, its acceptance in the medical field is still a matter of debate [13, 14]. The CDSS acceptance is linked to the perception of the end user, such as 1) the system’s ease of use and utility, 2) the quality of its results and its reliability [14], 3) the contextual accessibility of the system, sometimes not included in the health professional’s routine and workflow, and 4) the fact that numerous CDSSs are not integrated with existing systems [15]. One manner to extend the use and disseminate positive contributions of CDSSs to the medical world is to develop them in a reliable and useful way. For this, one must follow the best practices of software engineering (SE, acronym in English) [16] and be concerned with its quality, both in the design and development process and in its effective use. Evaluating the quality of the software is to measure its characteristics and sub-characteristics of quality. In order to better structure the assessment, a series of international standards, with models and frameworks, were developed for assisting software developers in assessing the quality of software products. The latest series is the ISO/IEC 25000 - System and Software Quality Requirements and Evaluation (SQuaRE) [17]. Two of the SQuaRE divisions are addressed in this thesis: 1) Division of quality models standard (ISO/IEC 25010) [18], and 2) Quality measurement division standard (ISO/IEC 25022) [19]. The ISO/IEC 25010 are divided in product quality model and the quality model in use. Quality in use (QiU), a model of ISO/IEC 25010, is the focus of this study, through its evaluation in the context of a CDSS. The quality in use model refers to the quality of the software when executed, mentioning the result of the interaction between users and the software system/product in a specific context. This model consists of five quality characteristics: • Effectiveness - means the level of precision and completeness with which users achieve their specific goals when using the system; • Efficiency - refers to the resources spent to achieve the goals and its measure is related to the level of effectiveness achieved with the consumed resources; • Satisfaction - refers to whether user requirements are satisfied in a particular context of system use; • Freedom from risk - refers to the degree to which the quality of a system reduces or avoids potential risks to human life, the economic situation, and health of the environment; • Context coverage - deals with the use of the system in all specific contexts and/or in contexts that extend beyond the initially identified contexts. Context completeness and flexibility are the sub-characteristics that represent context coverage. Thus, when measuring the quality of a CDSS, we must consider both the context of use and the choice of the characteristic and sub-characteristic that best suits the purpose of the measurement [20]. The QiU model provides a powerful contribution to the practice of evaluating a system and determining its quality. According to Harrison et al. [21], Effectiveness, Efficiency and Satisfaction are considered the key criteria to reflect the quality of use. Therefore, these QiU characteristics meet the needs and expectations of the users of the systems, in our case of CDSSs, as they consider the user experience. As a contribution, we proposed a process model to evaluate two QiU characteristics in a CDSS: satisfaction and efficiency. We believe these characteristics are important in the evaluation of a CDSS because, due to its links with the user experience and the usability of the system, when measured, can corroborate the quality of the CDSS and mitigate the non-use and non-acceptance of this type of software. Other contributions from our work are 1) in the academic context, a significant study in the area of software quality, focusing on its characteristics, especially on the quality in use. A guideline for collecting and measuring these characteristics was built into our process model; 2) in the area of software development, professionals can make use of a simple and adaptable process, applicable to other types of systems, to measure the quality-in-use characteristics of their products.Desenvolver ou adquirir software é um investimento caro e precisa ser justificado. Além de útil, o sistema deve ser confiável, eficiente e, entre outras características, atender às expectativas dos usuários [1, 2]. Não seria diferente no caso de um sistema de apoio à decisão clínica (CDSS, acrônimo em inglês), sistemas desenvolvidos para apoiar médicos e outros profissionais de saúde na tomada de uma decisão médica [3]. CDSSs são elaborados dentro de um contexto clínico, seguindo guidelines com propósitos variados, sejam para diagnósticos [4, 5, 6], acompanhamento do paciente [7, 8, 9], na prevenção [10] e tratamento de doenças [11, 12]. No entanto, apesar de todo os benefícios oferecidos por um CDSS, sua aceitação na área médica ainda é motivo de debate [13, 14]. Essa aceitação está ligada à percepção do usuário final, como 1) a facilidade de uso e utilidade do sistema; 2) a qualidade dos resultados produzidos e sua confiabilidade [14]; 3) a acessibilidade contextual do sistema, muitas vezes não incluída na rotina e no fluxo de trabalho do profissional de saúde, e 4) o fato de muitos CDSSs não estarem integrados aos sistemas existentes [15]. Uma forma de estender o uso de CDSSs e disseminar suas contribuições positivas entre os profissionais de saúde é garantir a confiabilidade de seus resultados e a satisfação do usuáriofinal. Para tal deve-se seguir as melhores práticas da engenharia de software (SE, acrônimo em inglês) em sua concepção [16]. Isso implica em preocupar-se com a qualidade do sistema tanto no processo do projeto e desenvolvimento quanto em sua efetiva utilização. Uma forma de certificar se um software obedece a essa premissa é realizando avaliações de qualidade. Avaliar a qualidade do software é medir suas características e subcaracterísticas de qualidade. Para uma melhor estruturação desta medição foram desenvolvidos séries de padrões internacionais como guidelines de avaliação de qualidade de produtos de software. A série mais recente trata-se da ISO/IEC 25000 System and Software Quality Requirements and Evaluation (SQUARE) [17]. Dois padrões desta série foram abordadas nesta tese, sendo 1) o modelos de qualidade de software e sistemas (ISO/IEC 25010) [18], no qual trabalhamos especificamente com o modelo de qualidade em uso, e 2) o padrão de medição da qualidade em uso (ISO/IEC 25022) [19]. Qualidade em uso é o foco desta tese, através de sua avaliação no contexto de utilização de um CDSS. O Modelo de qualidade em uso trata da qualidade do software quando em execução, referindose ao resultado da interação dos usuários e o software em um cenário específico. Este modelo é composto de cinco características de qualidade: • Eficácia (ou efetividade) - esta característica representa o nível de precisão e completude com que os usuários alcançam os objetivos específicos, durante a utilização do sistema ou produto de software; • Eficiência - sua medição representa o nível de eficácia alcançada em relação aos recursos consumidos para o alcance das metas; • Satisfação - trata do quanto as necessidades do usuário são satisfeitas dentro de um determinado contexto de uso do sistema ou produto de software. Esta característica é composta pelas subcaracterísticas Utilidade, Confiança, Prazer e Conforto do usuário em relação ao sistema; • Livre de risco - trata do grau em que a qualidade de um sistema ou produto permite mitigar ou evitar riscos potenciais à vida humana, à situação econômica, à saúde ou ao meio ambiente, sendo estas suas três subcaracterísticas; • Cobertura de contexto - trata do uso do sistema em todos os contextos específicos e/ou em contextos além dos inicialmente identificados, sendo composta pelas subcaracterísticas completude de contexto e flexibilidade do sistema. Assim, para se medir a qualidade de um CDSS deve-se considerar tanto o contexto de utilização quanto a escolha da característica e subcaracterística que melhor condizem ao propósito da avaliação [20]. De acordo com Harrison et al. [21], Eficácia, Eficiência e Satisfação são considerados os principais critérios a serem avaliados para refletir a qualidade de uso. Tais características de qualidades em uso refletem o atendimento das necessidades e expectativas dos usuários dos sistemas, em especial ao usuário primário ou final, uma vez que estão diretamente relacionadas com a experiência do usuário. O modelo de qualidade em uso fornece uma contribuição poderosa para a prática de avaliar um sistema e determinar sua qualidade. Como contribuição, propusemos um modelo de processo para avaliação de qualidade em uso de um CDSS através da medição, a priori, de duas características de qualidade - satisfação e eficiência. Acreditamos que tais características são importantes na avaliação de um CDSS devido estreita relação destas com a experiência do usuário-final e a usabilidade do sistema. Assim, quando mensuradas, tais características podem corroborar com a qualidade do CDSS e mitigar a não utilização e não aceitação desse tipo de software. Nosso modelo proposto é definido por cinco (5) fases, a saber: 1) Identificação de cenário e contexto de uso do sistema, 2) seleção das medidas, métricas e métodos para mensurar as características, 3) a medição da qualidade, 4) a análise dos valores encontrados na medição e 5) a apresentação dos resultados obtidos. O resultado da aplicação do modelo de processo traduz-se em um conjunto de informações que nortearão um melhoramento do software, caso a medição das características fique abaixo de um padrão pré-definido pelos atores envolvidos no processo de medição do sistema. Por outro lado, se a medição for positiva, isso vem ratificar a qualidade do sistema e ações poderão ser tomadas para disseminar esse bom resultado, buscando a adesão de mais utilizadores. Como forma de validação do modelo proposto, após sua utilização para identificação de cenários e contexto-de-uso possíveis de serem mensurados, foi apresentado um CDSS da área oncológica a profissionais de saúde, estudantes de medicina e profissionais da área de qualidade de software que, ao final de sua utilização, responderam a um inquérito com o objetivo de avaliar o sistema. A aplicação se deu de forma online, dado a necessidade de mantermos o distanciamento social e o de cumprirmos as orientações sanitárias. As respostas serviram como fonte de dados para a medição das características de qualidadeem- uso do sistema. Os resultados da aplicação revelou que nosso modelo de processo de avaliação é válido, relevante e de fácil utilização para identificar as características importantes em um sistema, bem como suas medições por meio das funções matemáticas do modelo ISO/IEC 25022. Outras contribuições do nosso trabalho, temos 1) no âmbito acadêmico, um estudo significativo na área de qualidade de software, com foco em suas características, especialmente na qualidade em uso. Uma guideline para a coleta e mensuração dessas características foi construída em nosso modelo de processo; 2) na área de desenvolvimento de software, os profissionais podem contar com um processo simples e adaptável, aplicável a outros tipos de sistema, para mensuração da qualidade em uso de seus produtos.The research has been partially funded by the FCT/MCTES through national funds, and when applicable, co-funded EU funds under the project UIDB/EEA/50008/2020 and Operação Centro 01-0145-FEDER-000019 – C4 – Centro de Competências em Cloud Computing, co-financed by the Programa Operacional Regional do Centro (CENTRO 2020), through the Sistema de Apoio à Investigação Científica e Tecnológica – Programas Integrados de IC&DT. I would also like to acknowledge the contribution of the COST Action IC1303: AAPELE—Archi- tectures, Algorithms and Protocols for Enhanced Living Environments and COST Action CA16226; SHELD-ON—Indoor living space improvement: Smart Habitat for the Elderly, supported by COST (European Cooperation in Science and Technology)