Enhance Representation Learning of Clinical Narrative with Neural Networks for Clinical Predictive Modeling

Medicine is undergoing a technological revolution. Understanding human health from clinical data has major challenges from technical and practical perspectives, thus prompting methods that understand large, complex, and noisy data. These methods are particularly necessary for natural language data from clinical narratives/notes, which contain some of the richest information on a patient. Meanwhile, deep neural networks have achieved superior performance in a wide variety of natural language processing (NLP) tasks because of their capacity to encode meaningful but abstract representations and learn the entire task end-to-end. In this thesis, I investigate representation learning of clinical narratives with deep neural networks through a number of tasks ranging from clinical concept extraction, clinical note modeling, and patient-level language representation. I present methods utilizing representation learning with neural networks to support understanding of clinical text documents. I first introduce the notion of representation learning from natural language processing and patient data modeling. Then, I investigate word-level representation learning to improve clinical concept extraction from clinical notes. I present two works on learning word representations and evaluate them to extract important concepts from clinical notes. The first study focuses on cancer-related information, and the second study evaluates shared-task data. The aims of these two studies are to automatically extract important entities from clinical notes. Next, I present a series of deep neural networks to encode hierarchical, longitudinal, and contextual information for modeling a series of clinical notes. I also evaluate the models by predicting clinical outcomes of interest, including mortality, length of stay, and phenotype predictions. Finally, I propose a novel representation learning architecture to develop a generalized and transferable language representation at the patient level. I also identify pre-training tasks appropriate for constructing a generalizable language representation. The main focus is to improve predictive performance of phenotypes with limited data, a challenging task due to a lack of data. Overall, this dissertation addresses issues in natural language processing for medicine, including clinical text classification and modeling. These studies show major barriers to understanding large-scale clinical notes. It is believed that developing deep representation learning methods for distilling enormous amounts of heterogeneous data into patient-level language representations will improve evidence-based clinical understanding. The approach to solving these issues by learning representations could be used across clinical applications despite noisy data. I conclude that considering different linguistic components in natural language and sequential information between clinical events is important. Such results have implications beyond the immediate context of predictions and further suggest future directions for clinical machine learning research to improve clinical outcomes. This could be a starting point for future phenotyping methods based on natural language processing that construct patient-level language representations to improve clinical predictions. While significant progress has been made, many open questions remain, so I will highlight a few works to demonstrate promising directions

A Frame-Based NLP System for Cancer-Related Information Extraction.

We propose a frame-based natural language processing (NLP) method that extracts cancer-related information from clinical narratives. We focus on three frames: cancer diagnosis, cancer therapeutic procedure, and tumor description. We utilize a deep learning-based approach, bidirectional Long Short-term Memory (LSTM) Conditional Random Field (CRF), which uses both character and word embeddings. The system consists of two constituent sequence classifiers: a frame identification (lexical unit) classifier and a frame element classifier. The classifier achieves an

Deep Patient Representation of Clinical Notes via Multi-Task Learning for Mortality Prediction.

We propose a deep learning-based multi-task learning (MTL) architecture focusing on patient mortality predictions from clinical notes. The MTL framework enables the model to learn a patient representation that generalizes to a variety of clinical prediction tasks. Moreover, we demonstrate how MTL enables small but consistent gains on a single classification task (e.g., in-hospital mortality prediction) simply by incorporating related tasks (e.g., 30-day and 1-year mortality prediction) into the MTL framework. To accomplish this, we utilize a multi-level Convolutional Neural Network (CNN) associated with a MTL loss component. The model is evaluated with 3, 5, and 20 tasks and is consistently able to produce a higher-performing model than a single-task learning (STL) classifier. We further discuss the effect of the multi-task model on other clinical outcomes of interest, including being able to produce high-quality representations that can be utilized to great effect by simpler models. Overall, this study demonstrates the efficiency and generalizability of MTL across tasks that STL fails to leverage

Generalized and Transferable Patient Language Representation for Phenotyping with Limited Data

The paradigm of representation learning through transfer learning has the potential to greatly enhance clinical natural language processing. In this work, we propose a multi-task pre-training and fine-tuning approach for learning generalized and transferable patient representations from medical language. The model is first pre-trained with different but related high-prevalence phenotypes and further fine-tuned on downstream target tasks. Our main contribution focuses on the impact this technique can have on low-prevalence phenotypes, a challenging task due to the dearth of data. We validate the representation from pre-training, and fine-tune the multi-task pre-trained models on low-prevalence phenotypes including 38 circulatory diseases, 23 respiratory diseases, and 17 genitourinary diseases. We find multi-task pre-training increases learning efficiency and achieves consistently high performance across the majority of phenotypes. Most important, the multi-task pre-training is almost always either the best-performing model or performs tolerably close to the best-performing model, a property we refer to as robust. All these results lead us to conclude that this multi-task transfer learning architecture is a robust approach for developing generalized and transferable patient language representations for numerous phenotypes.Comment: Journal of Biomedical Informatics (in press

Enhancing Clinical Concept Extraction with Contextual Embeddings

Neural network-based representations ("embeddings") have dramatically advanced natural language processing (NLP) tasks, including clinical NLP tasks such as concept extraction. Recently, however, more advanced embedding methods and representations (e.g., ELMo, BERT) have further pushed the state-of-the-art in NLP, yet there are no common best practices for how to integrate these representations into clinical tasks. The purpose of this study, then, is to explore the space of possible options in utilizing these new models for clinical concept extraction, including comparing these to traditional word embedding methods (word2vec, GloVe, fastText). Both off-the-shelf open-domain embeddings and pre-trained clinical embeddings from MIMIC-III are evaluated. We explore a battery of embedding methods consisting of traditional word embeddings and contextual embeddings, and compare these on four concept extraction corpora: i2b2 2010, i2b2 2012, SemEval 2014, and SemEval 2015. We also analyze the impact of the pre-training time of a large language model like ELMo or BERT on the extraction performance. Last, we present an intuitive way to understand the semantic information encoded by contextual embeddings. Contextual embeddings pre-trained on a large clinical corpus achieves new state-of-the-art performances across all concept extraction tasks. The best-performing model outperforms all state-of-the-art methods with respective F1-measures of 90.25, 93.18 (partial), 80.74, and 81.65. We demonstrate the potential of contextual embeddings through the state-of-the-art performance these methods achieve on clinical concept extraction. Additionally, we demonstrate contextual embeddings encode valuable semantic information not accounted for in traditional word representations.Comment: Journal of the American Medical Informatics Associatio

Understanding Spatial Language in Radiology: Representation Framework, Annotation, and Spatial Relation Extraction from Chest X-ray Reports using Deep Learning

We define a representation framework for extracting spatial information from radiology reports (Rad-SpRL). We annotated a total of 2000 chest X-ray reports with 4 spatial roles corresponding to the common radiology entities. Our focus is on extracting detailed information of a radiologist's interpretation containing a radiographic finding, its anatomical location, corresponding probable diagnoses, as well as associated hedging terms. For this, we propose a deep learning-based natural language processing (NLP) method involving both word and character-level encodings. Specifically, we utilize a bidirectional long short-term memory (Bi-LSTM) conditional random field (CRF) model for extracting the spatial roles. The model achieved average F1 measures of 90.28 and 94.61 for extracting the Trajector and Landmark roles respectively whereas the performance was moderate for Diagnosis and Hedge roles with average F1 of 71.47 and 73.27 respectively. The corpus will soon be made available upon request

Identification of the chemical components of ethanol extract of Chenopodium ambrosioides and evaluation of their in vitro antioxidant and anti tumor activities

Purpose: To determine the characteristic chemical components of the ethanol extract of Chenopodium ambrosioides and evaluate their antioxidant and anti-tumor effects in vitro. Methods: The plant powder (5 g) was extracted with 1 L of 80 % ethanol at room temperature for 45 min, and then placed at 60 oC at varying microwave power and duration to obtain optimal extraction conditions. Characteristic chemical components were detected using ultra-high performance liquid chromatography quadrupole time of flight mass spectrometry (UPLC-Q-TOF-MS/MS). Kaempferitrin was isolated from the 80 % ethanol extract using a D101 macroporous resin column, and its content was assessed by high performance liquid chromatography (HPLC). The antioxidant effect of kaempferitrin was evaluated by its ability to scavenge 2,2-diphenyl-1-picrylhydrazyl (DPPH) and 2,2'-azinobis-(3-ethylbenzthiazoline-6-sulphonate) (ABTS) radicals, while its anti-proliferation activity in human liver cancer cells SMMC-7721 was determined using cell counting kit-8 (CCK-8) reagent. Results: Three characteristic components of ethanol extract of C. ambrosioides were obtained, namely, kaempferitrin, kaempferol-3-O-apigenin-7-O-rhamnoside and kaempferol-3-O-acetylapigenin-7-O-rhamnoside. Kaempferitrin was shown to possess strong DPPH radical and moderate ABTS radical scavenging activities. Kaempferitrin significantly inhibited the proliferation of SMMC-7721 cells at doses of 4 and 8 μg/mL, with half-maximal concentration (IC50) of 0.38 μM (p < 0.05). Conclusion: Kaempferitrin extracted from C. ambrosioides has antioxidant and anti-tumor activities. The results reported here indicate that C. ambrosioides may have potential use in herbal medicine practice

Concomitant valve surgery is associated with worse outcomes in surgical treatments of post-infarction ventricular aneurysm

ObjectiveTo evaluate the impact of concomitant valve surgery on the prognosis of patients who experienced coronary artery bypass graft (CABG) with/without ventricular reconstruction for the ventricular aneurysm.MethodsIn our department, 354 patients underwent CABG with/without ventricular reconstruction for a ventricular aneurysm from July 23rd, 2000 to December 23rd, 2022. A total of 77 patients received concomitant valve surgery, 37 of whom underwent replacement, and 40 of whom underwent repair. The baseline characteristics, prognostic, and follow-up information were statically analyzed. Univariate and multivariate Cox regression analyses were applied to identify the risk factors of long-term outcomes.ResultsCompared with patients who did not undergo valvular surgery, patients who experienced concomitant valve surgical treatments had a significantly lower survival rate (p = 0.00022) and a longer total mechanical ventilation time. Subgroup analysis indicated that the options of repair or replacement exhibited no statistically significant difference in postoperative mortality (p = 0.44) and prognosis. The multivariate Cox regression analysis suggested that the pre-operative cholesterol level (HR = 1.68), postoperative IABP (HR = 6.29), NYHA level (HR = 2.84), and pre-operative triglyceride level (HR = 1.09) were independent and significant predictors for overall all-cause mortality after surgery.ConclusionConcomitant valve surgery was considerably related to a higher risk of postoperative mortality in patients with post-infarction ventricle aneurysms who underwent surgical treatments. No significant difference in the prognosis outcomes was observed between the operating methods of repair or replacement valve surgery