AD-BERT: Using Pre-trained contextualized embeddings to Predict the Progression from Mild Cognitive Impairment to Alzheimer's Disease

Objective: We develop a deep learning framework based on the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model using unstructured clinical notes from electronic health records (EHRs) to predict the risk of disease progression from Mild Cognitive Impairment (MCI) to Alzheimer's Disease (AD). Materials and Methods: We identified 3657 patients diagnosed with MCI together with their progress notes from Northwestern Medicine Enterprise Data Warehouse (NMEDW) between 2000-2020. The progress notes no later than the first MCI diagnosis were used for the prediction. We first preprocessed the notes by deidentification, cleaning and splitting, and then pretrained a BERT model for AD (AD-BERT) based on the publicly available Bio+Clinical BERT on the preprocessed notes. The embeddings of all the sections of a patient's notes processed by AD-BERT were combined by MaxPooling to compute the probability of MCI-to-AD progression. For replication, we conducted a similar set of experiments on 2563 MCI patients identified at Weill Cornell Medicine (WCM) during the same timeframe. Results: Compared with the 7 baseline models, the AD-BERT model achieved the best performance on both datasets, with Area Under receiver operating characteristic Curve (AUC) of 0.8170 and F1 score of 0.4178 on NMEDW dataset and AUC of 0.8830 and F1 score of 0.6836 on WCM dataset. Conclusion: We developed a deep learning framework using BERT models which provide an effective solution for prediction of MCI-to-AD progression using clinical note analysis

Dynamical Mean-Field Theory - from Quantum Impurity Physics to Lattice Problems

Since the first investigation of the Hubbard model in the limit of infinite dimensions by Metzner and Vollhardt, dynamical mean-field theory (DMFT) has become a very powerful tool for the investigation of lattice models of correlated electrons. In DMFT the lattice model is mapped on an effective quantum impurity model in a bath which has to be determined self-consistently. This approach lead to a significant progress in our understanding of typical correlation problems such as the Mott transition; furthermore, the combination of DMFT with ab-initio methods now allows for a realistic treatment of correlated materials. The focus of these lecture notes is on the relation between quantum impurity physics and the physics of lattice models within DMFT. Issues such as the observability of impurity quantum phase transitions in the corresponding lattice models are discussed in detail.Comment: 18 pages, 5 figures, invited paper for the Proceedings of the "3rd International Summer School on Strongly Correlated Systems, Debrecen, 2004

Operationalizing Healthcare Big Data in the Electronic Health Records using a Heatmap Visualization Technique

Background: The majority of the electronic health record (EHR) contains a wealth of information, including unstructured notes. Healthcare professionals may be missing substantial portions of essential diagnostic and treatment information by not focusing on unstructured texts. The objective of this study is to present progress notes data using heatmap visualization. Methods: In this study, the research team used the unstructured text from the progress notes of deidentified patient data. The research team conducted qualitative content-coding based on the clinical complexity model and developed a heatmap based on the processed frequency data. Result: The researchers developed a color-coded heatmap focusing on the severity and acuity of patients’ status accumulated through multiple previous patient’s visits. Conclusions: Future research into creating an automated process to generate the heatmap from an unstructured dataset can open up opportunities to operationalize big data in healthcare

Extracting detailed oncologic history and treatment plan from medical oncology notes with large language models

Both medical care and observational studies in oncology require a thorough understanding of a patient's disease progression and treatment history, often elaborately documented in clinical notes. Despite their vital role, no current oncology information representation and annotation schema fully encapsulates the diversity of information recorded within these notes. Although large language models (LLMs) have recently exhibited impressive performance on various medical natural language processing tasks, due to the current lack of comprehensively annotated oncology datasets, an extensive evaluation of LLMs in extracting and reasoning with the complex rhetoric in oncology notes remains understudied. We developed a detailed schema for annotating textual oncology information, encompassing patient characteristics, tumor characteristics, tests, treatments, and temporality. Using a corpus of 10 de-identified breast cancer progress notes at University of California, San Francisco, we applied this schema to assess the abilities of three recently-released LLMs (GPT-4, GPT-3.5-turbo, and FLAN-UL2) to perform zero-shot extraction of detailed oncological history from two narrative sections of clinical progress notes. Our team annotated 2750 entities, 2874 modifiers, and 1623 relationships. The GPT-4 model exhibited overall best performance, with an average BLEU score of 0.69, an average ROUGE score of 0.72, and an average accuracy of 67% on complex tasks (expert manual evaluation). Notably, it was proficient in tumor characteristic and medication extraction, and demonstrated superior performance in inferring symptoms due to cancer and considerations of future medications. The analysis demonstrates that GPT-4 is potentially already usable to extract important facts from cancer progress notes needed for clinical research, complex population management, and documenting quality patient care.Comment: Source code available at: https://github.com/MadhumitaSushil/OncLLMExtractio