Automation bias in electronic prescribing

© 2017 The Author(s). Background: Clinical decision support (CDS) in e-prescribing can improve safety by alerting potential errors, but introduces new sources of risk. Automation bias (AB) occurs when users over-rely on CDS, reducing vigilance in information seeking and processing. Evidence of AB has been found in other clinical tasks, but has not yet been tested with e-prescribing. This study tests for the presence of AB in e-prescribing and the impact of task complexity and interruptions on AB. Methods: One hundred and twenty students in the final two years of a medical degree prescribed medicines for nine clinical scenarios using a simulated e-prescribing system. Quality of CDS (correct, incorrect and no CDS) and task complexity (low, low + interruption and high) were varied between conditions. Omission errors (failure to detect prescribing errors) and commission errors (acceptance of false positive alerts) were measured. Results: Compared to scenarios with no CDS, correct CDS reduced omission errors by 38.3% (p < .0001, n = 120), 46.6% (p < .0001, n = 70), and 39.2% (p < .0001, n = 120) for low, low + interrupt and high complexity scenarios respectively. Incorrect CDS increased omission errors by 33.3% (p < .0001, n = 120), 24.5% (p < .009, n = 82), and 26.7% (p < .0001, n = 120). Participants made commission errors, 65.8% (p < .0001, n = 120), 53.5% (p < .0001, n = 82), and 51.7% (p < .0001, n = 120). Task complexity and interruptions had no impact on AB. Conclusions: This study found evidence of AB omission and commission errors in e-prescribing. Verification of CDS alerts is key to avoiding AB errors. However, interventions focused on this have had limited success to date. Clinicians should remain vigilant to the risks of CDS failures and verify CDS

Measurement Error in Performance Studies of Health Information Technology: Lessons from the Management Literature

Just as researchers and clinicians struggle to pin down the benefits attendant to health information technology (IT), management scholars have long labored to identify the performance effects arising from new technologies and from other organizational innovations, namely the reorganization of work and the devolution of decision-making authority. This paper applies lessons from that literature to theorize the likely sources of measurement error that yield the weak statistical relationship between measures of health IT and various performance outcomes. In so doing, it complements the evaluation literature’s more conceptual examination of health IT’s limited performance impact. The paper focuses on seven issues, in particular, that likely bias downward the estimated performance effects of health IT. They are 1.) negative self-selection, 2.) omitted or unobserved variables, 3.) mis-measured contextual variables, 4.) mismeasured health IT variables, 5.) lack of attention to the specific stage of the adoption-to-use continuum being examined, 6.) too short of a time horizon, and 7.) inappropriate units-of-analysis. The authors offer ways to counter these challenges. Looking forward more broadly, they suggest that researchers take an organizationally-grounded approach that privileges internal validity over generalizability. This focus on statistical and empirical issues in health IT-performance studies should be complemented by a focus on theoretical issues, in particular, the ways that health IT creates value and apportions it to various stakeholders

Automation bias and prescribing decision support – rates, mediators and mitigators

Purpose: Computerised clinical decision support systems (CDSS) are implemented within healthcare settings as a method to improve clinical decision quality, safety and effectiveness, and ultimately patient outcomes. Though CDSSs tend to improve practitioner performance and clinical outcomes, relatively little is known about specific impact of inaccurate CDSS output on clinicians. Although there is high heterogeneity between CDSS types and studies, reviews of the ability of CDSS to prevent medication errors through incorrect decisions have generally been consistently positive, working by improving clinical judgement and decision making. However, it is known that the occasional incorrect advice given may tempt users to reverse a correct decision, and thus introduce new errors. These systematic errors can stem from Automation Bias (AB), an effect which has had little investigation within the healthcare field, where users have a tendency to use automated advice heuristically. Research is required to assess the rate of AB, identify factors and situations involved in overreliance and propose says to mitigate risk and refine the appropriate usage of CDSS; this can provide information to promote awareness of the effect, and ensure the maximisation of the impact of benefits gained from the implementation of CDSS. Background: A broader literature review was carried out coupled with a systematic review of studies investigating the impact of automated decision support on user decisions over various clinical and non-clinical domains. This aimed to identify gaps in the literature and build an evidence-based model of reliance on Decision Support Systems (DSS), particularly a bias towards over-using automation. The literature review and systematic review revealed a number of postulates - that CDSS are socio-technical systems, and that factors involved in CDSS misuse can vary from overarching social or cultural factors, individual cognitive variables to more specific technology design issues. However, the systematic review revealed there is a paucity of deliberate empirical evidence for this effect. The reviews identified the variables involved in automation bias to develop a conceptual model of overreliance, the initial development of an ontology for AB, and ultimately inform an empirical study to investigate persuasive potential factors involved: task difficulty, time pressure, CDSS trust, decision confidence, CDSS experience and clinical experience. The domain of primary care prescribing was chosen within which to carry out an empirical study, due to the evidence supporting CDSS usefulness in prescribing, and the high rate of prescribing error. Empirical Study Methodology: Twenty simulated prescribing scenarios with associated correct and incorrect answers were developed and validated by prescribing experts. An online Clinical Decision Support Simulator was used to display scenarios to users. NHS General Practitioners (GPs) were contacted via emails through associates of the Centre for Health Informatics, and through a healthcare mailing list company. Twenty-six GPs participated in the empirical study. The study was designed so each participant viewed and gave prescriptions for 20 prescribing scenarios, 10 coded as “hard” and 10 coded as “medium” prescribing scenarios (N = 520 prescribing cases were answered overall). Scenarios were accompanied by correct advice 70% of the time, and incorrect advice 30% of the time (in equal proportions in either task difficulty condition). Both the order of scenario presentation and the correct/incorrect nature of advice were randomised to prevent order effects. The planned time pressure condition was dropped due to low response rate. Results: To compare with previous literature which took overall decisions into account, taking individual cases into account (N=520), the pre advice accuracy rate of the clinicians was 50.4%, which improved to 58.3% post advice. The CDSS improved the decision accuracy in 13.1% of prescribing cases. The rate of AB, as measured by decision switches from correct pre advice, to incorrect post advice was 5.2% of all cases at a CDSS accuracy rate of 70% - leading to a net improvement of 8%. However, the above by-case type of analysis may not enable generalisation of results (but illustrates rates in this specific situation); individual participant differences must be taken into account. By participant (N = 26) when advice was correct, decisions were more likely to be switched to a correct prescription, when advice was incorrect decisions were more likely to be switched to an incorrect prescription. There was a significant correlation between decision switching and AB error. By participant, more immediate factors such as trust in the specific CDSS, decision confidence, and task difficulty influenced rate of decision switching. Lower clinical experience was associated with more decision switching (but not higher AB rate). The rate of AB was somewhat problematic to analyse due to low number of instances – the effect could potentially have been greater. The between subjects effect of time pressure could not be investigated due to low response rate. Age, DSS experience and trust in CDSS generally were not significantly associated with decision switching. Conclusion: There is a gap in the current literature investigating inappropriate CDSS use, but the general literature supports an interactive multi-factorial aetiology for automation misuse. Automation bias is a consistent effect with various potential direct and indirect causal factors. It may be mitigated by altering advice characteristics to aid clinicians’ awareness of advice correctness and support their own informed judgement – this needs further empirical investigation. Users’ own clinical judgement must always be maintained, and systems should not be followed unquestioningly

Medical errors : Healthcare professionals’ perspective at a tertiary hospital in Kuwait

Medical errors are of economic importance and can contribute to serious adverse events for patients. Medical errors refer to preventable events resulting from healthcare interactions, whether these events harm the patient or not. In Kuwait, there is a paucity literature detailing the causes, forms, and risks of medical errors in their state-funded healthcare facilities. This study aimed to explore medical errors, their causes and preventive strategies in a Kuwait tertiary hospital based on the perceptions and experience of a cross-section of healthcare professionals using a questionnaire with 27 open (n = 10) and closed (n = 17) questions. The recruited healthcare professionals in this study included pharmacists, nurses, physicians, dentists, radiographers, hospital administrators, surgeons, nutritionists, and physiotherapists. The collected data were analysed quantitatively using descriptive statistics. A total of 203 participants filled and completed the survey questionnaire. The frequency of medical errors in Kuwait was found to be high at 60.3% ranging from incidences of prolonged hospital stays (32.9%), adverse events and life-threatening complications (32.3%), and fatalities (20.9%). The common medical errors result from incomplete instructions, incorrect dosage, and incorrect route of administration, diagnosis errors, and labelling errors. The perceived causes of these medical errors include high workload, lack of support systems, stress, medical negligence, inadequate training, miscommunication, poor collaboration, and non-adherence to safety guidelines among the healthcare professionals.Peer reviewe

The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support

Background: Computerised decision support (CDS) based on trustworthy clinical guidelines is a key component of a learning healthcare system. Research shows that the effectiveness of CDS is mixed. Multifaceted context, system, recommendation and implementation factors may potentially affect the success of CDS interventions. This paper describes the development of a checklist that is intended to support professionals to implement CDS successfully. Methods: We developed the checklist through an iterative process that involved a systematic review of evidence and frameworks, a synthesis of the success factors identified in the review, feedback from an international expert panel that evaluated the checklist in relation to a list of desirable framework attributes, consultations with patients and healthcare consumers and pilot testing of the checklist. Results: We screened 5347 papers and selected 71 papers with relevant information on success factors for guideline-based CDS. From the selected papers, we developed a 16-factor checklist that is divided in four domains, i.e. the CDS context, content, system and implementation domains. The panel of experts evaluated the checklist positively as an instrument that could support people implementing guideline-based CDS across a wide range of settings globally. Patients and healthcare consumers identified guideline-based CDS as an important quality improvement intervention and perceived the GUIDES checklist as a suitable and useful strategy. Conclusions: The GUIDES checklist can support professionals in considering the factors that affect the success of CDS interventions. It may facilitate a deeper and more accurate understanding of the factors shaping CDS effectiveness. Relying on a structured approach may prevent that important factors are missed

Human Factors Standards and the Hard Human Factor Problems: Observations on Medical Usability Standards

With increasing variety and sophistication of computer-based medical devices, and more diverse users and use environments, usability is essential, especially to ensure safety. Usability standards and guidelines play an important role. We reviewed several, focusing on the IEC 62366 and 60601 sets. It is plausible that these standards have reduced risks for patients, but we raise concerns regarding: (1) complex design trade-offs that are not addressed, (2) a focus on user interface design (e.g., making alarms audible) to the detriment of other human factors (e.g., ensuring users actually act upon alarms they hear), and (3) some definitions and scope restrictions that may create “blind spots”. We highlight potential related risks, e.g. that clear directives on “easier to understand” risks, though useful, may preclude mitigating other, more “difficult” ones; but ask to what extent these negative effects can be avoided by standard writers, given objective constraints. Our critique is motivated by current research and incident reports, and considers standards from other domains and countries. It is meant to highlight problems, relevant to designers, standards committees, and human factors researchers, and to trigger discussion about the potential and limits of standards

체계적 문헌고찰과 메타분석을 통한 전산처방자동화시스템과 관련된 처방오류 평가 연구

학위논문 (석사) -- 서울대학교 대학원 : 약학대학 약학과, 2020. 8. 김은경.Computerized Physician Order Entry (CPOE) systems and Clinical Decision Support Systems (CDSS) have been proven to contribute to improve patients safety and quality of care; however, the adoption of computerization introduced a new type of error, called system-related or technology-induced errors. A comprehensive evaluation regarding the prevalence of CPOE-related errors (CRE) is lacking. The aim of this study was to describe the prevalence of CRE evaluated by pharmacists and to evaluate the association between the introduction of CPOE and prescribing errors. A systematic review and meta-analysis were conducted of studies retrieved from the MEDLINE, Embase, Cochrane, and Scopus up to March 2020. All studies reporting the rate of prescribing errors related to CPOE were included. The prevalence of CRE among overall prescribing errors occurred in the hospitals was estimated using pooled prevalence estimate with a 95% confidence interval (CI) and relative risk (RR) was calculated for the subgroup analysis. A total of 14 studies were identified and included in the systematic review and meta-analysis. In the meta-analysis of 13 data of estimate, the overall pooled prevalence of CRE across studies were 32.36% (95% CI 22.87 – 42.62). Among the 6 types of error identified throughout the studies: omission, wrong drug, wrong dose, wrong route/form, wrong time, and monitoring error, the main type of error related to CPOE were wrong dose (47.28%, 95% CI 38.38-56.26), followed by wrong drug (14.45%, 95% CI 7.96-22.40). The subgroup analysis revealed that the risk of error was not significantly reduced with CPOE (RR 0.842, 95% CI 0.559 – 1.268), except omission which was significantly reduced after the implementation of CPOE (RR 0.484, 95% CI 0.282 – 0.831). Our study findings support that system-related errors were a major reason for CPOE not delivering a significant reduction in the overall rate of clinical errors. A considerable risk for prescribing errors still exists, which healthcare professionals should be aware that CPOE could also lead to a new type of medication errors. In order to reduce the prescribing error related to CPOE, the system should be continually examined and users should receive periodic and multidisciplinary training on the use of CPOE and CDSS.처방자동화시스템(Computerized Physician Order Entry, CPOE)과 임상의사결정지원시스템(Clinical Decision Support System)의 활성화로 전체적인 처방오류의 비율은 감소하였지만, CPOE와 같은 새로운 시스템으로 인하여 새로운 오류가 출현되었다. 본 연구는 원내 CPOE와 관련된 약물 처방오류 중 약사가 평가한 처방오류의 발생률과 CPOE 도입 전후 오류유형의 변화를 파악하고자 선행연구들을 대상으로 체계적 문헌고찰과 메타분석을 수행하였다. PubMed, EMBASE, Cochrane Register of Controlled Trials, Scopus에서 2020년 3월까지 검색되는 문헌 중 CPOE 도입 후 발생한 처방오류에 해당하는 문헌을 추출하였고 선정 및 제외기준에 따라 총 14개의 최종 문헌을 선정하였다. 처방오류의 합동 발생률 수치와 CPOE 도입 전과 후 유형 별 처방오류 발생의 상대 위험도 및 95% 신뢰 구간은 랜덤 효과 모델을 적용하여 제시하였다. CPOE 도입 후 전체 처방오류 중 CPOE로 인한 처방오류의 발생률 추정치 범위는 12.78%에서 58.54% 사이였고 랜덤 효과 모델에서 계산된 합동 발생률은 32.36%였다 (95% 신뢰 구간 22.87-42.62). National Coordinating Council for Medication Error Reporting and Prevention 분류체계에 기반하여 문헌에서 추출 가능한 처방오류의 유형을 처방 누락오류, 약물 오류, 용량오류, 제형 및 투여경로 오류, 투여 시간 오류, 약물 모니터링과 같이 총 6개 유형으로 분류하였을 때, 용량오류가 47.28% (95% 신뢰 구간 38.38-56.26)로 가장 높았고 그 다음은 약물 오류가 14.45% (95% 신뢰 구간 7.96-22.40)으로 높았다. CPOE 도입 전과 후의 처방오류 유형별 발생을 비교하기 위하여 하위그룹 메타 분석을 하였을 때, CPOE 도입 후 전체적인 처방오류의 발생률은 CPOE 도입 전에 비해 통계적으로 유의하게 증가하지 않았으나 (Relative risk, RR 0.842, 95% 신뢰 구간 0.559-1.168), 6개 처방오류 유형 중 메타분석이 가능한 5개 오류 유형 중 (처방 누락오류, 약물 오류, 용량오류, 제형 및 투여경로 오류, 약물 모니터링) 처방 누락오류만 CPOE 도입 후 유의하게 줄어들었다 (RR 0.484, 95% 신뢰 구간 0.282-0.831). 체계적 문헌고찰 및 메타분석을 통해 새로운 기술인 CPOE 도입 후 CPOE와 관련된 처방오류가 전체 처방오류 중 1/3의 빈도로 발생하는 것으로 파악되었다. 처방오류의 유형 중 처방 누락오류, 약물 오류, 용량오류, 제형 및 투여경로 오류, 약물 모니터링의 오류의 발생 비율은 CPOE 도입 전과 후에 유의한 변화를 보이지 않았으나, 처방 누락의 비율은 CPOE 도입 후에 낮아진 것으로 나타났다. 약물처방의 전자화와 처방 지원 시스템과 같은 새로운 기술의 도입으로 단순 실수로 인한 처방오류는 방지되었으나 다양한 처방오류가 지속해서 발생함으로 환자의 안전을 위한 시스템 사용자의 지속적인 교육과 시스템의 기술적 개선으로 처방오류의 예방, 감지, 및 모니터링의 노력이 필요하다.1. Introduction 1 2. Methods 3 3. Results 8 4. Discussion 25 5. Conclusion 31 References 32 Appendix 40 요약 (국문초록) 48Maste

Electronic Prescribing and Robotic Dispensing: The Impact of Integrating Together on Practice and Professionalism

Technology developments offer hospital pharmacies opportunities to enhance efficiency and safety in the dispensing process. Adoption of technology potentially allows release of resources to develop more patient-focused activities. Resource release can be achieved via a variety of impacts, such as efficiency of the dispensing process, reduction of potential for dispensing errors and potential to adjust skill mix. Developing more patient-focused activities can enhance pharmacy development in a broader sense, and as this happens, changes can occur in professional identities across a range of job roles within the pharmacy. These changes offer benefits to the development of the professional model

Association of Electronic Health Records with Methicillin-Resistant Staphylococcus aureus Infection in a National Sample

This study examined the relationship between advanced electronic health record (EHR) use in hospitals and rates of Methicillin-resistant Staphylococcus aureus (MRSA) infection in an inpatient setting. National Inpatient Sample (NIS) and Health Information Management Systems Society (HIMSS) Annual Survey are combined in the retrospective, cross-sectional analysis. A twenty percent simple random sample of the combined 2009 NIS and HIMSS datasets included a total of 1,032,905 patient cases of MRSA in 550 hospitals. Results of the propensity-adjusted logistic regression model revealed a statistically significant association between advanced EHR and MRSA, with patient cases from an advanced EHR being less likely to report a MRSA diagnosis code

Optimising hospital electronic prescribing systems:A Systematic Scoping Review

Considerable international investment in hospital electronic prescribing (ePrescribing) systems has been made, but despite this, it is proving difficult for most organizations to realize safety, quality, and efficiency gains in prescribing. The objective of this work was to develop policy-relevant insights into the optimization of hospital ePrescribing systems to maximize the benefits and minimize the risks of these expensive digital health infrastructures. METHODS: We undertook a systematic scoping review of the literature by searching MEDLINE, Embase, and CINAHL databases. We searched for primary studies reporting on ePrescribing optimization strategies and independently screened and abstracted data until saturation was achieved. Findings were theoretically and thematically synthesized taking a medicine life-cycle perspective, incorporating consultative phases with domain experts. RESULTS: We identified 23,609 potentially eligible studies from which 1367 satisfied our inclusion criteria. Thematic synthesis was conducted on a data set of 76 studies, of which 48 were based in the United States. Key approaches to optimization included the following: stakeholder engagement, system or process redesign, technological innovations, and education and training packages. Single-component interventions (n = 26) described technological optimization strategies focusing on a single, specific step in the prescribing process. Multicomponent interventions (n = 50) used a combination of optimization strategies, typically targeting multiple steps in the medicines management process. DISCUSSION: We identified numerous optimization strategies for enhancing the performance of ePrescribing systems. Key considerations for ePrescribing optimization include meaningful stakeholder engagement to reconceptualize the service delivery model and implementing technological innovations with supporting training packages to simultaneously impact on different facets of the medicines management process