Formative evaluation of a patient-specific clinical knowledge summarization tool

To iteratively design a prototype of a computerized clinical knowledge summarization (CKS) tool aimed at helping clinicians finding answers to their clinical questions; and to conduct a formative assessment of the usability, usefulness, efficiency, and impact of the CKS prototype on physicians’ perceived decision quality compared with standard search of UpToDate and PubMed

Evaluation of A Prototype of Computerized Health Knowledge Summaries

Physicians raise two questions in every three patients they see and around 50% of those questions are not even pursued due to various reasons. These unanswered questions represent huge knowledge gap and could result in less than desirable treatment outcomes. The situation becomes even worse with the emergence of internet technologies which brought explosively increasing information and knowledge into everybody's lives. To make medical information more readily available and to facilitate physicians' decision making process, we designed and developed a medical knowledge summary system that automatically extract and synthesize relevant medical evidence from major resources including UpToDate and PubMed. We performed a pilot usability study to evaluate the effectiveness of the system and used the feedback from physicians to further the development effort. Physicians in general found our system intuitive to use and information delivered very valuable in filling in their knowledge gaps.Master of Science in Information Scienc

Providing better clinical reference sources at the point of care

Physicians frequently need clinical reference material at the time they are taking care of patients, but despite the widespread availability of seemingly appropriate resources, physicians do not consistently avail themselves. Previous research has shown that the main obstacle to resource use is lack of time to search, followed by inability to find the information even with extended searching. Using a randomly selected sample of previously published questions (Clinical Questions Collection at the National Library of Medicine), current online electronic clinical reference resources were searched. Search times, success rates, and result quality were compared. Retrieval of drug information was dominated by navigation time, whereas searches for other materials were dominated by search time. It was rare that actually reading the material required significant time. Based on the results, an improved interface for accessing clinical reference data is proposed. Key features include a "Fast" interface that minimizes navigation time to easily found resources and a "Flex" interface that minimizes search time for more arcane material.Master of Science in Information Scienc

Doctor of Philosophy

dissertationMedical knowledge learned in medical school can become quickly outdated given the tremendous growth of the biomedical literature. It is the responsibility of medical practitioners to continuously update their knowledge with recent, best available clinical evidence to make informed decisions about patient care. However, clinicians often have little time to spend on reading the primary literature even within their narrow specialty. As a result, they often rely on systematic evidence reviews developed by medical experts to fulfill their information needs. At the present, systematic reviews of clinical research are manually created and updated, which is expensive, slow, and unable to keep up with the rapidly growing pace of medical literature. This dissertation research aims to enhance the traditional systematic review development process using computer-aided solutions. The first study investigates query expansion and scientific quality ranking approaches to enhance literature search on clinical guideline topics. The study showed that unsupervised methods can improve retrieval performance of a popular biomedical search engine (PubMed). The proposed methods improve the comprehensiveness of literature search and increase the ratio of finding relevant studies with reduced screening effort. The second and third studies aim to enhance the traditional manual data extraction process. The second study developed a framework to extract and classify texts from PDF reports. This study demonstrated that a rule-based multipass sieve approach is more effective than a machine-learning approach in categorizing document-level structures and iv that classifying and filtering publication metadata and semistructured texts enhances the performance of an information extraction system. The proposed method could serve as a document processing step in any text mining research on PDF documents. The third study proposed a solution for the computer-aided data extraction by recommending relevant sentences and key phrases extracted from publication reports. This study demonstrated that using a machine-learning classifier to prioritize sentences for specific data elements performs equally or better than an abstract screening approach, and might save time and reduce errors in the full-text screening process. In summary, this dissertation showed that there are promising opportunities for technology enhancement to assist in the development of systematic reviews. In this modern age when computing resources are getting cheaper and more powerful, the failure to apply computer technologies to assist and optimize the manual processes is a lost opportunity to improve the timeliness of systematic reviews. This research provides methodologies and tests hypotheses, which can serve as the basis for further large-scale software engineering projects aimed at fully realizing the prospect of computer-aided systematic reviews

Extractive Summarization : Experimental work on nursing notes in Finnish

Natural Language Processing (NLP) is a subfield of artificial intelligence and linguistics that is concerned with how a computer machine interacts with human language. With the increasing computational power and the advancement in technologies, researchers have been successful at proposing various NLP tasks that have already been implemented as real-world applications today. Automated text summarization is one of the many tasks that has not yet completely matured particularly in health sector. A success in this task would enable healthcare professionals to grasp patient's history in a minimal time resulting in faster decisions required for better care. Automatic text summarization is a process that helps shortening a large text without sacrificing important information. This could be achieved by paraphrasing the content known as the abstractive method or by concatenating relevant extracted sentences namely the extractive method. In general, this process requires the conversion of text into numerical form and then a method is executed to identify and extract relevant text. This thesis is an attempt of exploring NLP techniques used in extractive text summarization particularly in health domain. The work includes a comparison of basic summarizing models implemented on a corpus of patient notes written by nurses in Finnish language. Concepts and research studies required to understand the implementation have been documented along with the description of the code. A python-based project is structured to build a corpus and execute multiple summarizing models. For this thesis, we observe the performance of two textual embeddings namely Term Frequency - Inverse Document Frequency (TF-IDF) which is based on simple statistical measure and Word2Vec which is based on neural networks. For both models, LexRank, an unsupervised stochastic graph-based sentence scoring algorithm, is used for sentence extraction and a random selection method is used as a baseline method for evaluation. To evaluate and compare the performance of models, summaries of 15 patient care episodes of each model were provided to two human beings for manual evaluations. According to the results of the small sample dataset, we observe that both evaluators seem to agree with each other in preferring summaries produced by Word2Vec LexRank over the summaries generated by TF-IDF LexRank. Both models have also been observed, by both evaluators, to perform better than the baseline model of random selection

Biomedical Literature Mining and Knowledge Discovery of Phenotyping Definitions

Indiana University-Purdue University Indianapolis (IUPUI)Phenotyping definitions are essential in cohort identification when conducting clinical research, but they become an obstacle when they are not readily available. Developing new definitions manually requires expert involvement that is labor-intensive, time-consuming, and unscalable. Moreover, automated approaches rely mostly on electronic health records’ data that suffer from bias, confounding, and incompleteness. Limited efforts established in utilizing text-mining and data-driven approaches to automate extraction and literature-based knowledge discovery of phenotyping definitions and to support their scalability. In this dissertation, we proposed a text-mining pipeline combining rule-based and machine-learning methods to automate retrieval, classification, and extraction of phenotyping definitions’ information from literature. To achieve this, we first developed an annotation guideline with ten dimensions to annotate sentences with evidence of phenotyping definitions' modalities, such as phenotypes and laboratories. Two annotators manually annotated a corpus of sentences (n=3,971) extracted from full-text observational studies’ methods sections (n=86). Percent and Kappa statistics showed high inter-annotator agreement on sentence-level annotations. Second, we constructed two validated text classifiers using our annotated corpora: abstract-level and full-text sentence-level. We applied the abstract-level classifier on a large-scale biomedical literature of over 20 million abstracts published between 1975 and 2018 to classify positive abstracts (n=459,406). After retrieving their full-texts (n=120,868), we extracted sentences from their methods sections and used the full-text sentence-level classifier to extract positive sentences (n=2,745,416). Third, we performed a literature-based discovery utilizing the positively classified sentences. Lexica-based methods were used to recognize medical concepts in these sentences (n=19,423). Co-occurrence and association methods were used to identify and rank phenotype candidates that are associated with a phenotype of interest. We derived 12,616,465 associations from our large-scale corpus. Our literature-based associations and large-scale corpus contribute in building new data-driven phenotyping definitions and expanding existing definitions with minimal expert involvement

User Interfaces for Patient-Centered Communication of Health Status and Care Progress

The recent trend toward patients participating in their own healthcare has opened up numerous opportunities for computing research. This dissertation focuses on how technology can foster this participation, through user interfaces to effectively communicate personal health status and care progress to hospital patients. I first characterize the design space for electronic information communication to patients through field studies conducted in multiple hospital settings. These studies utilize a combination of survey instruments, and low- and high-fidelity prototypes, including a document-editing prototype through which users can view and manage clinical data to automatically associate it with progress notes. The prototype, activeNotes, includes the first known techniques supporting clinical information requests directly within a document editor. A usage study with ICU physicians at New York-Presbyterian Hospital (NYP) substantiated our design and revealed how electronic information related to patient status and care progress is derived from a typical Electronic Health Record system. Insights gained from this study informed following studies to understand how to design abstracted, plain-language views suitable for patients. We gauged both patient and physician responses to information display prototypes deployed in patient rooms for a formative study exploring their design. Following my reports on this study, I discuss the design, development and pilot evaluations of a prototype Personal Health Record application providing live, abstracted clinical information for patients at NYP. The portal, evaluated by cardiothoracic surgery patients, is the first of its kind to allow patients to capture and monitor live data related to their care. Patient use of the portal influenced the subsequent design of tools to support users in making sense of online medication information. These tools, designed with nurses and pharmacists and evaluated by cardiothoracic surgery patients at NYP, were developed using topic modeling approaches and text analysis techniques. Embodied in a prototype called Remedy, they enable rapid filtering and comparison of medication-related search results, based on a number of website features and content topics. I conclude by discussing how findings from this series of studies can help shape the ongoing design and development of patient-centered technology

From temporal expressions to symptom onset date identification in emergency department notes: a temporal information extraction process

A patient's visit to the Emergency Department (ED) starts with the triage nurse making a note of the patient's account of the reason for the visit. This triage note (TN) contains symptoms the patient is suffering from, prior treatments if any, related events and sometimes the nurse's evaluation of the situation. Public health officials may use these TNs to identify features of disease outbreaks. Here we present a system that processes triage notes, producing a timeline of events leading to the ED visit and identifying patterns in occurrence of symptoms across patients. This system is designed as an initial step in the process of automatically extracting signals/symptoms defining a disease outbreak based on the details (symptoms and temporal information) associated with ED visits