Developing a manually annotated clinical document corpus to identify phenotypic information for inflammatory bowel disease

<p>Abstract</p> <p>Background</p> <p>Natural Language Processing (NLP) systems can be used for specific Information Extraction (IE) tasks such as extracting phenotypic data from the electronic medical record (EMR). These data are useful for translational research and are often found only in free text clinical notes. A key required step for IE is the manual annotation of clinical corpora and the creation of a reference standard for (1) training and validation tasks and (2) to focus and clarify NLP system requirements. These tasks are time consuming, expensive, and require considerable effort on the part of human reviewers.</p> <p>Methods</p> <p>Using a set of clinical documents from the VA EMR for a particular use case of interest we identify specific challenges and present several opportunities for annotation tasks. We demonstrate specific methods using an open source annotation tool, a customized annotation schema, and a corpus of clinical documents for patients known to have a diagnosis of Inflammatory Bowel Disease (IBD). We report clinician annotator agreement at the document, concept, and concept attribute level. We estimate concept yield in terms of annotated concepts within specific note sections and document types.</p> <p>Results</p> <p>Annotator agreement at the document level for documents that contained concepts of interest for IBD using estimated Kappa statistic (95% CI) was very high at 0.87 (0.82, 0.93). At the concept level, F-measure ranged from 0.61 to 0.83. However, agreement varied greatly at the specific concept attribute level. For this particular use case (IBD), clinical documents producing the highest concept yield per document included GI clinic notes and primary care notes. Within the various types of notes, the highest concept yield was in sections representing patient assessment and history of presenting illness. Ancillary service documents and family history and plan note sections produced the lowest concept yield.</p> <p>Conclusion</p> <p>Challenges include defining and building appropriate annotation schemas, adequately training clinician annotators, and determining the appropriate level of information to be annotated. Opportunities include narrowing the focus of information extraction to use case specific note types and sections, especially in cases where NLP systems will be used to extract information from large repositories of electronic clinical note documents.</p

Doctor of Philosophy

DissertationHealth information technology (HIT) in conjunction with quality improvement (QI) methodologies can promote higher quality care at lower costs. Unfortunately, most inpatient hospital settings have been slow to adopt HIT and QI methodologies. Successful adoption requires close attention to workflow. Workflow is the sequence of tasks, processes, and the set of people or resources needed for those tasks that are necessary to accomplish a given goal. Assessing the impact on workflow is an important component of determining whether a HIT implementation will be successful, but little research has been conducted on the impact of eMeasure (electronic performance measure) implementation on workflow. One solution to addressing implementation challenges such as the lack of attention to workflow is an implementation toolkit. An implementation toolkit is an assembly of instruments such as checklists, forms, and planning documents. We developed an initial eMeasure Implementation Toolkit for the heart failure (HF) eMeasure to allow QI and information technology (IT) professionals and their team to assess the impact of implementation on workflow. During the development phase of the toolkit, we undertook a literature review to determine the components of the toolkit. We conducted stakeholder interviews with HIT and QI key informants and subject matter experts (SMEs) at the US Department of Veteran Affairs (VA). Key informants provided a broad understanding about the context of workflow during eMeasure implementation. Based on snowball sampling, we also interviewed other SMEs based on the recommendations of the key informants who suggested tools and provided information essential to the toolkit development. The second phase involved evaluation of the toolkit for relevance and clarity, by experts in non-VA settings. The experts evaluated the sections of the toolkit that contained the tools, via a survey. The final toolkit provides a distinct set of resources and tools, which were iteratively developed during the research and available to users in a single source document. The research methodology provided a strong unified overarching implementation framework in the form of the Promoting Action on Research Implementation in Health Services (PARIHS) model in combination with a sociotechnical model of HIT that strengthened the overall design of the study

BMC Bioinformatics

BackgroundNatural Language Processing (NLP) systems can be used for specific Information Extraction (IE) tasks such as extracting phenotypic data from the electronic medical record (EMR). These data are useful for translational research and are often found only in free text clinical notes. A key required step for IE is the manual annotation of clinical corpora and the creation of a reference standard for (1) training and validation tasks and (2) to focus and clarify NLP system requirements. These tasks are time consuming, expensive, and require considerable effort on the part of human reviewers.MethodsUsing a set of clinical documents from the VA EMR for a particular use case of interest we identify specific challenges and present several opportunities for annotation tasks. We demonstrate specific methods using an open source annotation tool, a customized annotation schema, and a corpus of clinical documents for patients known to have a diagnosis of Inflammatory Bowel Disease (IBD). We report clinician annotator agreement at the document, concept, and concept attribute level. We estimate concept yield in terms of annotated concepts within specific note sections and document types.ResultsAnnotator agreement at the document level for documents that contained concepts of interest for IBD using estimated Kappa statistic (95% CI) was very high at 0.87 (0.82, 0.93). At the concept level, F-measure ranged from 0.61 to 0.83. However, agreement varied greatly at the specific concept attribute level. For this particular use case (IBD), clinical documents producing the highest concept yield per document included GI clinic notes and primary care notes. Within the various types of notes, the highest concept yield was in sections representing patient assessment and history of presenting illness. Ancillary service documents and family history and plan note sections produced the lowest concept yield.ConclusionChallenges include defining and building appropriate annotation schemas, adequately training clinician annotators, and determining the appropriate level of information to be annotated. Opportunities include narrowing the focus of information extraction to use case specific note types and sections, especially in cases where NLP systems will be used to extract information from large repositories of electronic clinical note documents.1 PO1 CD000284-01/CD/ODCDC CDC HHS/United States19761566PMC274568

Electronic Health Record Optimization for Cardiac Care

Electronic health record (EHR) systems have been studied for over 30 years, and despite the benefits of information technology in other knowledge domains, progress has been slow in healthcare. A growing body of evidence suggests that dissatisfaction with EHR systems was not simply due to resistance to adoption of new technology but also due to real concerns about the adverse impact of EHRs on the delivery of patient care. Solutions for EHR improvement require an approach that combines an understanding of technology adoption with the complexity of the social and technical elements of the US healthcare system. Several studies are presented to clarify and propose a new framework to study EHR-provider interaction. Four focus areas were defined - workflow, communication, medical decision-making and patient care. Using Human Computer Interaction best practices, an EHR usability framework was designed to include a realistic clinical scenario, a cognitive walkthrough, a standardized simulated patient actor, and a portable usability lab. Cardiologists, fellows and nurse practitioners were invited to participate in a simulation to use their institution’s EHR system for a routine cardiac visit. Using a mixed methods approach, differences in satisfaction and effectiveness were identified. Cardiologists were dissatisfied with EHR functionality, and were critical of the potential impact of the communication of incorrect information, while displaying the highest level of success in completing the tasks. Fellows were slightly less dissatisfied with their EHR interaction, and demonstrated a preference for tools to improve workflow and support decision-making, and showed less success in completing the tasks in the scenario. Nurse practitioners were also dissatisfied with their EHR interaction, and cited poor organization of data, yet demonstrated more success than fellows in successful completion of tasks. Study results indicate that requirements for EHR functionality differ by type of provider. Cardiologists, cardiology fellows, and nurse practitioners required different levels of granularity of patient data for use in medical decision-making, defined different targets for communication, sought different solutions to workflow which included distribution of data input, and requested technical solutions to ensure valid and relevant patient data. These findings provide a foundation for future work to optimize EHR functionality

Progress Report No. 15

Progress report of the Biomedical Computer Laboratory, covering period 1 July 1978 to 30 June 1979

Architectural Design of the National Health Information System for Rwanda

The use of information technology in healthcare services can improve the quality of care. The large amount of research has demonstrated the role of the use of Information and communication technology (ICT) solutions to overcome the challenges in patient information management. One of the challenges is the healthcare information sharing between providers. In high income countries, the challenge of exchanging information is almost solved. Nearly all high income countries have implemented a national healthcare network which connects healthcare providers in the whole country. Furthermore, European Union (EU) aims at the point of cross-border healthcare information exchange which supports the mobility of EU citizens. However, in developing countries, they are not yet ready to take the full advantage of ICT in their healthcare systems. The main objective of the thesis was to design the architecture of a national health information system for Rwanda, which is a developing country with limited resources. The research was based on three main issues: One was to determine existing health IT solutions in the healthcare system of Rwanda. The second one was to explore how other countries have developed their national health information systems (NHISs). The third was to find out how open source solutions can build a national network for a country. From the research, the components of the architecture have been defined and finally the architecture was designed. The research started by examining the current situation of ICT solutions in the healthcare system of Rwanda. This showed the progress in implementing certain electronic medical record systems in certain health facilities. However, there is no single hospital with a fully functional system. This step was followed by exploring how other countries implemented their NHIS and it showed that the process varies country by country. It was clear that in developing countries, open source solutions got a large market share contrary to developed countries where proprietary systems are the most used. Finally, open source solutions proved the capability to build a NHIS with different examples of robust open source solutions available in health IT nowadays. Although it would have been interesting, the thesis does not estimate the financial resources needed for the implementation of the architecture. It is possible to implement the NHIS for Rwanda by using both proprietary and open source solutions. However, the interoperability issue can be mitigated by minimizing different types of electronic medical records in healthcare facilities

Adoption of Electronic Health Record Systems Within Primary Care Practices

Primary care physicians (PCPPs) have been slow to implement electronic health records (EHRs), even though there is a U.S. federal requirement to implement EHRs. The purpose of this phenomenological study was to determine why PCPPs have been slow to adopt electronic health record (EHR) systems despite the potential to increase efficiency and quality of health care. The complex adaptive systems theory (CAS) served as the conceptual framework for this study. Twenty-six PCPPs were interviewed from primary care practices (PCPs) based in southwestern Ohio. The data were collected through a semistructured interview format and analyzed using a modified van Kaam method. Several themes emerged as barriers to EHR implementation, including staff training on the new EHR system, the decrease in productivity experienced by primary care practice (PCP) staff adapting to the new EHR system, and system usability and technical support after adoption. The findings may contribute to the body of knowledge regarding EHR system implementation and assist healthcare providers who are slow to adopt EHRs. Additionally, findings could contribute to social change by reducing healthcare costs, increasing patient access to care, and improving the efficacy of patient diagnosis and treatment