Continuous Prompt Tuning Based Textual Entailment Model for E-commerce Entity Typing

The explosion of e-commerce has caused the need for processing and analysis of product titles, like entity typing in product titles. However, the rapid activity in e-commerce has led to the rapid emergence of new entities, which is difficult to be solved by general entity typing. Besides, product titles in e-commerce have very different language styles from text data in general domain. In order to handle new entities in product titles and address the special language styles problem of product titles in e-commerce domain, we propose our textual entailment model with continuous prompt tuning based hypotheses and fusion embeddings for e-commerce entity typing. First, we reformulate the entity typing task into a textual entailment problem to handle new entities that are not present during training. Second, we design a model to automatically generate textual entailment hypotheses using a continuous prompt tuning method, which can generate better textual entailment hypotheses without manual design. Third, we utilize the fusion embeddings of BERT embedding and CharacterBERT embedding with a two-layer MLP classifier to solve the problem that the language styles of product titles in e-commerce are different from that of general domain. To analyze the effect of each contribution, we compare the performance of entity typing and textual entailment model, and conduct ablation studies on continuous prompt tuning and fusion embeddings. We also evaluate the impact of different prompt template initialization for the continuous prompt tuning. We show our proposed model improves the average F1 score by around 2% compared to the baseline BERT entity typing model

DALA: A Distribution-Aware LoRA-Based Adversarial Attack against Pre-trained Language Models

Pre-trained language models (PLMs) that achieve success in applications are susceptible to adversarial attack methods that are capable of generating adversarial examples with minor perturbations. Although recent attack methods can achieve a relatively high attack success rate (ASR), our observation shows that the generated adversarial examples have a different data distribution compared with the original examples. Specifically, these adversarial examples exhibit lower confidence levels and higher distance to the training data distribution. As a result, they are easy to detect using very simple detection methods, diminishing the actual effectiveness of these attack methods. To solve this problem, we propose a Distribution-Aware LoRA-based Adversarial Attack (DALA) method, which considers the distribution shift of adversarial examples to improve attack effectiveness under detection methods. We further design a new evaluation metric NASR combining ASR and detection for the attack task. We conduct experiments on four widely-used datasets and validate the attack effectiveness on ASR and NASR of the adversarial examples generated by DALA on the BERT-base model and the black-box LLaMA2-7b model.Comment: First two authors contribute equall

A novel MIPgene mutation associated with autosomal dominant congenital cataracts in a Chinese family

BACKGROUND: The major intrinsic protein gene (MIP), also known as MIP26 or AQP0, is a member of the water-transporting aquaporin family, which plays a critical role in the maintenance of lifelong lens transparency. To date, several mutations in MIP (OMIM 154050) have been linked to hereditary cataracts in humans. However, more pathogenic mutations remain to be identified. In this study, we describe a four-generation Chinese family with a nonsense mutation in MIP associated with an autosomal dominant congenital cataract (ADCC), thus expanding the mutational spectrum of this gene. METHODS: A large four-generation Chinese family affected with typical Y-suture cataracts combined with punctuate cortical opacities and 100 ethnically matched controls were recruited. Genomic DNA was extracted from peripheral blood leukocytes to analyze congenital cataract-related candidate genes. Effects of the sequence change on the structure and function of proteins were predicted by bioinformatics analysis. RESULTS: Direct sequencing of MIP in all affected members revealed a heterozygous nucleotide exchange c.337C>T predicting an arginine to a stop codon exchange (p.R113X). The substitution co-segregated well in all the affected individuals in the family and was not found in unaffected members or in the 100 unrelated healthy controls. Bioinformatics analysis predicted that the mutation affects the secondary structure and function of the MIP protein. CONCLUSIONS: We identified a novel mutation of MIP (p.R113X) in a Chinese cataract family. This is the first nonsense mutation of MIP identified thus far. This novel mutation is also the first disease-causing mutation located in the loop C domain of MIP. The results add to the list of mutations of the MIP linked to cataracts

Named Entity Recognition via Machine Reading Comprehension: A Multi-Task Learning Approach

Named Entity Recognition (NER) aims to extract and classify entity mentions in the text into pre-defined types (e.g., organization or person name). Recently, many works have been proposed to shape the NER as a machine reading comprehension problem (also termed MRC-based NER), in which entity recognition is achieved by answering the formulated questions related to pre-defined entity types through MRC, based on the contexts. However, these works ignore the label dependencies among entity types, which are critical for precisely recognizing named entities. In this paper, we propose to incorporate the label dependencies among entity types into a multi-task learning framework for better MRC-based NER. We decompose MRC-based NER into multiple tasks and use a self-attention module to capture label dependencies. Comprehensive experiments on both nested NER and flat NER datasets are conducted to validate the effectiveness of the proposed Multi-NER. Experimental results show that Multi-NER can achieve better performance on all datasets

Uncertainty in Graph Neural Networks: A Survey

Graph Neural Networks (GNNs) have been extensively used in various real-world applications. However, the predictive uncertainty of GNNs stemming from diverse sources such as inherent randomness in data and model training errors can lead to unstable and erroneous predictions. Therefore, identifying, quantifying, and utilizing uncertainty are essential to enhance the performance of the model for the downstream tasks as well as the reliability of the GNN predictions. This survey aims to provide a comprehensive overview of the GNNs from the perspective of uncertainty with an emphasis on its integration in graph learning. We compare and summarize existing graph uncertainty theory and methods, alongside the corresponding downstream tasks. Thereby, we bridge the gap between theory and practice, meanwhile connecting different GNN communities. Moreover, our work provides valuable insights into promising directions in this field.Comment: 13 main pages, 3 figures, 1 table. Under revie