Measuring agreement between decision support reminders: the cloud vs. the local expert

Background: A cloud-based clinical decision support system (CDSS) was implemented to remotely provide evidence-based guideline reminders in support of preventative health. Following implementation, we measured the agreement between preventive care reminders generated by an existing, local CDSS and the new, cloud-based CDSS operating on the same patient visit data. Methods: Electronic health record data for the same set of patients seen in primary care were sent to both the cloud-based web service and local CDSS. The clinical reminders returned by both services were captured for analysis. Cohen’s Kappa coefficient was calculated to compare the two sets of reminders. Kappa statistics were further adjusted for prevalence and bias due to the potential effects of bias in the CDS logic and prevalence in the relative small sample of patients. Results: The cloud-based CDSS generated 965 clinical reminders for 405 patient visits over 3 months. The local CDSS returned 889 reminders for the same patient visit data. When adjusted for prevalence and bias, observed agreement varied by reminder from 0.33 (95% CI 0.24 – 0.42) to 0.99 (95% CI 0.97 – 1.00) and demonstrated almost perfect agreement for 7 of the 11 reminders. Conclusions: Preventive care reminders delivered by two disparate CDS systems show substantial agreement. Subtle differences in rule logic and terminology mapping appear to account for much of the discordance. Cloud-based CDSS therefore show promise, opening the door for future development and implementation in support of health care providers with limited resources for knowledge management of complex logic and rules

Using a Service Oriented Architecture Approach to Clinical Decision Support: Performance Results from Two CDS Consortium Demonstrations

The Clinical Decision Support Consortium has completed two demonstration trials involving a web service for the execution of clinical decision support (CDS) rules in one or more electronic health record (EHR) systems. The initial trial ran in a local EHR at Partners HealthCare. A second EHR site, associated with Wishard Memorial Hospital, Indianapolis, IN, was added in the second trial. Data were gathered during each 6 month period and analyzed to assess performance, reliability, and response time in the form of means and standard deviations for all technical components of the service, including assembling and preparation of input data. The mean service call time for each period was just over 2 seconds. In this paper we report on the findings and analysis to date while describing the areas for further analysis and optimization as we continue to expand our use of a Services Oriented Architecture approach for CDS across multiple institutions

The state of the art in clinical knowledge management: An inventory of tools and techniques

Purpose To explore the need for, and use of, high-quality, collaborative, clinical knowledge management (CKM) tools and techniques to manage clinical decision support (CDS) content. Methods In order to better understand the current state of the art in CKM, we developed a survey of potential CKM tools and techniques. We conducted an exploratory study by querying a convenience sample of respondents about their use of specific practices in CKM. Results The following tools and techniques should be priorities in organizations interested in developing successful computer-based provider order entry (CPOE) and CDS implementations: (1) a multidisciplinary team responsible for creating and maintaining the clinical content; (2) an external organizational repository of clinical content with web-based viewer that allows anyone in the organization to review it; (3) an online, collaborative, interactive, Internet-based tool to facilitate content development; (4) an enterprise-wide tool to maintain the controlled clinical terminology concepts. Even organizations that have been successfully using computer-based provider order entry with advanced clinical decision support features for well over 15 years are not using all of the CKM tools or practices that we identified. Conclusions If we are to further stimulate progress in the area of clinical decision support, we must continue to develop and refine our understanding and use of advanced CKM capabilities

Lessons Learned from Implementing Service-Oriented Clinical Decision Support at Four Sites: A Qualitative Study

Objective To identify challenges, lessons learned and best practices for service-oriented clinical decision support, based on the results of the Clinical Decision Support Consortium, a multi-site study which developed, implemented and evaluated clinical decision support services in a diverse range of electronic health records. Methods Ethnographic investigation using the rapid assessment process, a procedure for agile qualitative data collection and analysis, including clinical observation, system demonstrations and analysis and 91 interviews. Results We identified challenges and lessons learned in eight dimensions: (1) hardware and software computing infrastructure, (2) clinical content, (3) human-computer interface, (4) people, (5) workflow and communication, (6) internal organizational policies, procedures, environment and culture, (7) external rules, regulations, and pressures and (8) system measurement and monitoring. Key challenges included performance issues (particularly related to data retrieval), differences in terminologies used across sites, workflow variability and the need for a legal framework. Discussion Based on the challenges and lessons learned, we identified eight best practices for developers and implementers of service-oriented clinical decision support: (1) optimize performance, or make asynchronous calls, (2) be liberal in what you accept (particularly for terminology), (3) foster clinical transparency, (4) develop a legal framework, (5) support a flexible front-end, (6) dedicate human resources, (7) support peer-to-peer communication, (8) improve standards. Conclusion The Clinical Decision Support Consortium successfully developed a clinical decision support service and implemented it in four different electronic health records and four diverse clinical sites; however, the process was arduous. The lessons identified by the Consortium may be useful for other developers and implementers of clinical decision support services

Comparison of clinical knowledge management capabilities of commercially-available and leading internally-developed electronic health records

<p>Abstract</p> <p>Background</p> <p>We have carried out an extensive qualitative research program focused on the barriers and facilitators to successful adoption and use of various features of advanced, state-of-the-art electronic health records (EHRs) within large, academic, teaching facilities with long-standing EHR research and development programs. We have recently begun investigating smaller, community hospitals and out-patient clinics that rely on commercially-available EHRs. We sought to assess whether the current generation of commercially-available EHRs are capable of providing the clinical knowledge management features, functions, tools, and techniques required to deliver and maintain the clinical decision support (CDS) interventions required to support the recently defined "meaningful use" criteria.</p> <p>Methods</p> <p>We developed and fielded a 17-question survey to representatives from nine commercially available EHR vendors and four leading internally developed EHRs. The first part of the survey asked basic questions about the vendor's EHR. The second part asked specifically about the CDS-related system tools and capabilities that each vendor provides. The final section asked about clinical content.</p> <p>Results</p> <p>All of the vendors and institutions have multiple modules capable of providing clinical decision support interventions to clinicians. The majority of the systems were capable of performing almost all of the key knowledge management functions we identified.</p> <p>Conclusion</p> <p>If these well-designed commercially-available systems are coupled with the other key socio-technical concepts required for safe and effective EHR implementation and use, and organizations have access to implementable clinical knowledge, we expect that the transformation of the healthcare enterprise that so many have predicted, is achievable using commercially-available, state-of-the-art EHRs.</p

Continuity of Care Document (CCD) Enables Delivery of Medication Histories to the Primary Care Clinician

INTRODUCTION: The goal of the Enhanced Medication History (EMH) project is to provide medication histories to ambulatory primary care practices in the Indiana Network for Patient Care. METHODS: Medications were aggregated from three different sources of pharmacy data (Medicaid, SureScripts, and the county health system of Indianapolis). Dispensing events were assembled into the Continuity of Care Document (CCD), and presented to clinicians as RxNorm Clinical Drugs. RESULTS: The EMH project completed 46 weeks of operation in a community health center in Indianapolis. Medication Histories were generated for 10498 office visits for 4449 distinct patients. Seven (of nine) attending physicians responded to a written survey and found the Medication Histories useful (3.9±0.4 on a scale of 1 to 5). CONCLUSION: Implementation of the EMH project demonstrated the successful use (as well as the challenging aspects) of the CCD and the RxNorm terminology in the outpatient clinical setting

Building a Production-Ready Infrastructure to Enhance Medication Management: Early Lessons from the Nationwide Health Information Network

Poor medication management practices can lead to serious erosion of health care quality and safety. The DHHS Medication Management Use Case outlines methods for the exchange of electronic health information to improve medication management practices. In this case report, the authors describe initial development of Nationwide Health Information Network (NHIN) services to support the Medication Management Use Case. The technical approach and core elements of medication management transactions involved in the NHIN are presented. Early lessons suggest the pathway to improvements in quality and safety are achievable, yet there are challenges for the medical informatics community to address through future research and development activities