Perturbation-based Self-supervised Attention for Attention Bias in Text Classification

In text classification, the traditional attention mechanisms usually focus too much on frequent words, and need extensive labeled data in order to learn. This paper proposes a perturbation-based self-supervised attention approach to guide attention learning without any annotation overhead. Specifically, we add as much noise as possible to all the words in the sentence without changing their semantics and predictions. We hypothesize that words that tolerate more noise are less significant, and we can use this information to refine the attention distribution. Experimental results on three text classification tasks show that our approach can significantly improve the performance of current attention-based models, and is more effective than existing self-supervised methods. We also provide a visualization analysis to verify the effectiveness of our approach

Self-Adaptive Reconstruction with Contrastive Learning for Unsupervised Sentence Embeddings

Unsupervised sentence embeddings task aims to convert sentences to semantic vector representations. Most previous works directly use the sentence representations derived from pretrained language models. However, due to the token bias in pretrained language models, the models can not capture the fine-grained semantics in sentences, which leads to poor predictions. To address this issue, we propose a novel Self-Adaptive Reconstruction Contrastive Sentence Embeddings (SARCSE) framework, which reconstructs all tokens in sentences with an AutoEncoder to help the model to preserve more fine-grained semantics during tokens aggregating. In addition, we proposed a self-adaptive reconstruction loss to alleviate the token bias towards frequency. Experimental results show that SARCSE gains significant improvements compared with the strong baseline SimCSE on the 7 STS tasks.Comment: 8 pages, 3 figure

Balancing the Causal Effects in Class-Incremental Learning

Class-Incremental Learning (CIL) is a practical and challenging problem for achieving general artificial intelligence. Recently, Pre-Trained Models (PTMs) have led to breakthroughs in both visual and natural language processing tasks. Despite recent studies showing PTMs' potential ability to learn sequentially, a plethora of work indicates the necessity of alleviating the catastrophic forgetting of PTMs. Through a pilot study and a causal analysis of CIL, we reveal that the crux lies in the imbalanced causal effects between new and old data. Specifically, the new data encourage models to adapt to new classes while hindering the adaptation of old classes. Similarly, the old data encourages models to adapt to old classes while hindering the adaptation of new classes. In other words, the adaptation process between new and old classes conflicts from the causal perspective. To alleviate this problem, we propose Balancing the Causal Effects (BaCE) in CIL. Concretely, BaCE proposes two objectives for building causal paths from both new and old data to the prediction of new and classes, respectively. In this way, the model is encouraged to adapt to all classes with causal effects from both new and old data and thus alleviates the causal imbalance problem. We conduct extensive experiments on continual image classification, continual text classification, and continual named entity recognition. Empirical results show that BaCE outperforms a series of CIL methods on different tasks and settings

Improving Factual Consistency of Text Summarization by Adversarially Decoupling Comprehension and Embellishment Abilities of LLMs

Despite the recent progress in text summarization made by large language models (LLMs), they often generate summaries that are factually inconsistent with original articles, known as "hallucinations" in text generation. Unlike previous small models (e.g., BART, T5), current LLMs make fewer silly mistakes but more sophisticated ones, such as imposing cause and effect, adding false details, overgeneralizing, etc. These hallucinations are challenging to detect through traditional methods, which poses great challenges for improving the factual consistency of text summarization. In this paper, we propose an adversarially DEcoupling method to disentangle the Comprehension and EmbellishmeNT abilities of LLMs (DECENT). Furthermore, we adopt a probing-based efficient training to cover the shortage of sensitivity for true and false in the training process of LLMs. In this way, LLMs are less confused about embellishing and understanding; thus, they can execute the instructions more accurately and have enhanced abilities to distinguish hallucinations. Experimental results show that DECENT significantly improves the reliability of text summarization based on LLMs

Cellular automaton modeling of semisolid microstructure formation

Computer modeling of semi-solid structure formation is of significance in both understanding the mechanisms of globular structure formation and determining the effect of solidification conditions on final microstructure. A modified cellular automaton (mCA) model has been developed, which is coupled with macroscopic models for heat transfer calculation and microscopic models for nucleation and grain growth. The mCA model is applied to A356 Al alloy – one of the most widely used semi-solid alloys, to predict grain morphology and grain size during semi-solid solidification, and determines the effects of pouring temperature on the final microstructure. The modeling results show that the lower the initial temperature, the finer grain size will be obtained. In addition, the model can be used to predict the solutal micro-segregation

Metal-Free Photoredox Intramolecular Cyclization of N-Aryl Acrylamides

A novel metal-free photoredox-catalyzed cyclization reaction of N-aryl acrylamide is herein reported that provides synthetically valuable oxindole derivatives through the bis-mediation of H2O and aldehyde. In this work, sustainable visible light was used as the energy source, and the organic light-emitting molecule 4CzIPN served as the efficient photocatalyst. The main characteristics of this reaction are environmentally friendly and high yields

Direct and indirect effects and 95% confidence intervals for the final model.

<p>Direct and indirect effects and 95% confidence intervals for the final model.</p

Final Structural Model (N = 296).

<p>Note: all factor loadings were standardized. Stress1–Stress3 = three parcels of stress; Self-efficacy 1–Self-efficacy 3 = three parcels of self-efficacy; Optimism1–Optimism3 = three parcels of dispositional optimism; Suicide ideation1–Suicide ideation3 = three parcels of suicide ideation; ** P<0.01</p

Progress in C-C and C-Heteroatom Bonds Construction Using Alcohols as Acyl Precursors

Acyl moiety is a common structural unit in organic molecules, thus acylation methods have been widely explored to construct various functional compounds. While the traditional Friedel&ndash;Crafts acylation processes work to allow viable construction of arylketones under harsh acid conditions, recent progress on developing acylation methods focused on the new reactivity discovery by exploiting versatile and easily accessible acylating reagents. Of them, alcohols are cheap, have low toxicity, and are naturally abundant feedstocks; thus, they were recently used as ideal acyl precursors in molecule synthesis for ketones, esters, amides, etc. In this review, we display and discuss recent advances in employing alcohols as unusual acyl sources to form C-C and C-heteroatom bonds, with emphasis on the substrate scope, limitations, and mechanism