PeTailor: Improving Large Language Model by Tailored Chunk Scorer in Biomedical Triple Extraction

The automatic extraction of biomedical entities and their interaction from unstructured data remains a challenging task due to the limited availability of expert-labeled standard datasets. In this paper, we introduce PETAI-LOR, a retrieval-based language framework that is augmented by tailored chunk scorer. Unlike previous retrieval-augmented language models (LM) that retrieve relevant documents by calculating the similarity between the input sentence and the candidate document set, PETAILOR segments the sentence into chunks and retrieves the relevant chunk from our pre-computed chunk-based relational key-value memory. Moreover, in order to comprehend the specific requirements of the LM, PETAI-LOR adapt the tailored chunk scorer to the LM. We also introduce GM-CIHT, an expert annotated biomedical triple extraction dataset with more relation types. This dataset is centered on the non-drug treatment and general biomedical domain. Additionally, we investigate the efficacy of triple extraction models trained on general domains when applied to the biomedical domain. Our experiments reveal that PETAI-LOR achieves state-of-the-art performance on GM-CIHTComment: this is the first preprint versio

EHRTutor: Enhancing Patient Understanding of Discharge Instructions

Large language models have shown success as a tutor in education in various fields. Educating patients about their clinical visits plays a pivotal role in patients' adherence to their treatment plans post-discharge. This paper presents EHRTutor, an innovative multi-component framework leveraging the Large Language Model (LLM) for patient education through conversational question-answering. EHRTutor first formulates questions pertaining to the electronic health record discharge instructions. It then educates the patient through conversation by administering each question as a test. Finally, it generates a summary at the end of the conversation. Evaluation results using LLMs and domain experts have shown a clear preference for EHRTutor over the baseline. Moreover, EHRTutor also offers a framework for generating synthetic patient education dialogues that can be used for future in-house system training.Comment: To appear in NeurIPS'23 Workshop on Generative AI for Education (GAIED

W-procer: Weighted Prototypical Contrastive Learning for Medical Few-Shot Named Entity Recognition

Contrastive learning has become a popular solution for few-shot Name Entity Recognization (NER). The conventional configuration strives to reduce the distance between tokens with the same labels and increase the distance between tokens with different labels. The effect of this setup may, however, in the medical domain, there are a lot of entities annotated as OUTSIDE (O), and they are undesirably pushed apart to other entities that are not labeled as OUTSIDE (O) by the current contrastive learning method end up with a noisy prototype for the semantic representation of the label, though there are many OUTSIDE (O) labeled entities are relevant to the labeled entities. To address this challenge, we propose a novel method named Weighted Prototypical Contrastive Learning for Medical Few Shot Named Entity Recognization (W-PROCER). Our approach primarily revolves around constructing the prototype-based contractive loss and weighting network. These components play a crucial role in assisting the model in differentiating the negative samples from OUTSIDE (O) tokens and enhancing the discrimination ability of contrastive learning. Experimental results show that our proposed W-PROCER framework significantly outperforms the strong baselines on the three medical benchmark datasets

Privacy-preserving Fine-tuning of Large Language Models through Flatness

The privacy concerns associated with the use of Large Language Models (LLMs) have grown recently with the development of LLMs such as ChatGPT. Differential Privacy (DP) techniques are explored in existing work to mitigate their privacy risks at the cost of generalization degradation. Our paper reveals that the flatness of DP-trained models' loss landscape plays an essential role in the trade-off between their privacy and generalization. We further propose a holistic framework to enforce appropriate weight flatness, which substantially improves model generalization with competitive privacy preservation. It innovates from three coarse-to-grained levels, including perturbation-aware min-max optimization on model weights within a layer, flatness-guided sparse prefix-tuning on weights across layers, and weight knowledge distillation between DP \& non-DP weights copies. Comprehensive experiments of both black-box and white-box scenarios are conducted to demonstrate the effectiveness of our proposal in enhancing generalization and maintaining DP characteristics. For instance, on text classification dataset QNLI, DP-Flat achieves similar performance with non-private full fine-tuning but with DP guarantee under privacy budget $\epsilon=3$, and even better performance given higher privacy budgets. Codes are provided in the supplement.Comment: Accepted to ICLR 2024 SeT LLM Worksho

Integration of Cadmium Accumulation, Subcellular Distribution, and Physiological Responses to Understand Cadmium Tolerance in Apple Rootstocks

Cadmium (Cd) is a nonessential and highly toxic element causing agricultural problems. However, little information is available about the variation in Cd tolerance among apple rootstocks and its underlying physiological regulation mechanisms. This study investigated Cd accumulation, subcellular distribution, and chemical forms as well as physiological changes among four apple rootstocks exposed to either 0 or 300 μM CdCl2. The results showed that variations in Cd tolerance existed among these rootstocks. Cd exposure caused decline in photosynthesis, chlorophyll and biomass in four apple rootstocks, which was less pronounced in M. baccata, indicating its higher Cd tolerance. This finding was corroborated with higher Cd tolerance indexes (TIs) of the whole plant in M. baccata than those in the other three apple rootstocks. Among the four apple rootstocks, M. baccata displayed the lowest Cd concentrations in roots, wood, and leaves, the smallest total Cd amounts as well as the lowest BCF. In apple rootstocks, it was found that to immobilize Cd in cell wall and soluble fraction (most likely in vacuole) and to convert it into pectate- or protein- integrated forms and undissolved Cd phosphate forms may be the primary strategies to reduce Cd mobility and toxicity. The physiological changes including ROS, carbohydrates and antioxidants were in line with the variations of Cd tolerance among four apple rootstocks. In comparison with the other three apple rootstocks, M. baccata had lower concentrations of ROS in roots and bark, H2O2 in roots and leaves and MDA in roots, wood and bark, but higher concentrations of soluble sugars in bark and starch in roots and leaves, and enhanced antioxidants. These results indicate that M. baccata are more tolerant to Cd stress than the other three apple rootstocks under the current experiment conditions, which is probably related to Cd accumulation, subcellular partitioning and chemical forms of Cd and well-coordinated antioxidant defense mechanisms

Complementary and Integrative Health Lexicon (CIHLex) and Entity Recognition in the Literature

Objective: Our study aimed to construct an exhaustive Complementary and Integrative Health (CIH) Lexicon (CIHLex) to better represent the often underrepresented physical and psychological CIH approaches in standard terminologies. We also intended to apply advanced Natural Language Processing (NLP) models such as Bidirectional Encoder Representations from Transformers (BERT) and GPT-3.5 Turbo for CIH named entity recognition, evaluating their performance against established models like MetaMap and CLAMP. Materials and Methods: We constructed the CIHLex by integrating various resources, compiling and integrating data from biomedical literature and relevant knowledge bases. The Lexicon encompasses 198 unique concepts with 1090 corresponding unique terms. We matched these concepts to the Unified Medical Language System (UMLS). Additionally, we developed and utilized BERT models and compared their efficiency in CIH named entity recognition to that of other models such as MetaMap, CLAMP, and GPT3.5-turbo. Results: From the 198 unique concepts in CIHLex, 62.1% could be matched to at least one term in the UMLS. Moreover, 75.7% of the mapped UMLS Concept Unique Identifiers (CUIs) were categorized as "Therapeutic or Preventive Procedure." Among the models applied to CIH named entity recognition, BLUEBERT delivered the highest macro average F1-score of 0.90, surpassing other models. Conclusion: Our CIHLex significantly augments representation of CIH approaches in biomedical literature. Demonstrating the utility of advanced NLP models, BERT notably excelled in CIH entity recognition. These results highlight promising strategies for enhancing standardization and recognition of CIH terminology in biomedical contexts

A Review of Reinforcement Learning for Natural Language Processing, and Applications in Healthcare

Reinforcement learning (RL) has emerged as a powerful approach for tackling complex medical decision-making problems such as treatment planning, personalized medicine, and optimizing the scheduling of surgeries and appointments. It has gained significant attention in the field of Natural Language Processing (NLP) due to its ability to learn optimal strategies for tasks such as dialogue systems, machine translation, and question-answering. This paper presents a review of the RL techniques in NLP, highlighting key advancements, challenges, and applications in healthcare. The review begins by visualizing a roadmap of machine learning and its applications in healthcare. And then it explores the integration of RL with NLP tasks. We examined dialogue systems where RL enables the learning of conversational strategies, RL-based machine translation models, question-answering systems, text summarization, and information extraction. Additionally, ethical considerations and biases in RL-NLP systems are addressed

PaniniQA: Enhancing Patient Education Through Interactive Question Answering

Patient portal allows discharged patients to access their personalized discharge instructions in electronic health records (EHRs). However, many patients have difficulty understanding or memorizing their discharge instructions. In this paper, we present PaniniQA, a patient-centric interactive question answering system designed to help patients understand their discharge instructions. PaniniQA first identifies important clinical content from patients' discharge instructions and then formulates patient-specific educational questions. In addition, PaniniQA is also equipped with answer verification functionality to provide timely feedback to correct patients' misunderstandings. Our comprehensive automatic and human evaluation results demonstrate our PaniniQA is capable of improving patients' mastery of their medical instructions through effective interactionsComment: Accepted to TACL 2023. Equal contribution for the first two authors. This arXiv version is a pre-MIT Press publication versio

The association between dietary protein intake and colorectal cancer risk: a meta-analysis

Abstract Background Association between dietary protein intake and colorectal cancer risk has not been fully quantified, while the results were controversial. This study aimed to evaluate the role of protein intake in the development of colorectal cancer. Methods PUBMED and EMBASE were searched up to December 2016. Two independent reviewers independently extracted data from eligible studies. Relative risk (RR) with 95% confidence intervals (CI) was pooled using random-effects model to estimate the result. Besides, publication bias and sensitivity analysis were conducted. Results Thirteen articles involving 21 studies comprising 8187 cases were included in this report. The pooled RR of colorectal cancer was 1.006 (95% CI = 0.857–1.179) indicating that there is no significant association between dietary protein intake and colorectal cancer risk. Furthermore, the pooled RRs for colon cancer and rectum cancer were 1.135(95% CI = 0.871–1.480) and 0.773(95% CI = 0.538–1.111), respectively, with the highest category of dietary protein intake. The association was not significant either in subgroup analysis of study design, protein type (animal protein or vegetable protein), sex, and or geographic locations. Conclusions The present study indicated that the highest category compared to the lowest category of protein intake had no significant association on colorectal cancer risk. Dose-response analysis was not conducted due to limited information provided. Therefore, more studies with large cases and participants as well as detailed amounts of dietary protein intake are wanted to confirm this result