Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment

Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models. It is helpful to evaluate or even improve the robustness of these models by exposing the maliciously crafted adversarial examples. In this paper, we present TextFooler, a simple but strong baseline to generate natural adversarial text. By applying it to two fundamental natural language tasks, text classification and textual entailment, we successfully attacked three target models, including the powerful pre-trained BERT, and the widely used convolutional and recurrent neural networks. We demonstrate the advantages of this framework in three ways: (1) effective---it outperforms state-of-the-art attacks in terms of success rate and perturbation rate, (2) utility-preserving---it preserves semantic content and grammaticality, and remains correctly classified by humans, and (3) efficient---it generates adversarial text with computational complexity linear to the text length. *The code, pre-trained target models, and test examples are available at https://github.com/jind11/TextFooler.Comment: AAAI 2020 (Oral

Sentiment spin: Attacking financial sentiment with GPT-3

In this study, we explore the susceptibility of financial sentiment analysis to adversarial attacks that manipulate financial texts. With the rise of AI readership in the financial sector, companies are adapting their language and disclosures to fit AI processing better, leading to concerns about the potential for manipulation. In the finance literature, keyword-based methods, such as dictionaries, are still widely used for financial sentiment analysis due to their perceived transparency. However, our research demonstrates the vulnerability of keyword-based approaches by successfully generating adversarial attacks using the sophisticated transformer model, GPT-3. With a success rate of nearly 99% for negative sentences in the Financial Phrase Bank, a widely used database for financial sentiment analysis, we highlight the importance of incorporating robust methods, such as context-aware approaches such as BERT, in financial sentiment analysis

A Review on Identification of Contextual Similar Sentences

The task of identifying contextual similar sentences plays a crucial role in various natural language processing applications such as information retrieval, paraphrase detection, and question answering systems. This paper presents a comprehensive review of the methodologies, techniques, and advancements in the identification of contextual similar sentences. Beginning with an overview of the importance and challenges associated with this task, the paper delves into the various approaches employed, including traditional similarity metrics, deep learning architectures, and transformer-based models. Furthermore, the review explores different datasets and evaluation metrics used to assess the performance of these methods. Additionally, the paper discusses recent trends, emerging research directions, and potential applications in the field. By synthesizing existing literature, this review aims to provide researchers and practitioners with insights into the state-of-the-art techniques and future avenues for advancing the identification of contextual similar sentences

Disentangled Contrastive Learning for Learning Robust Textual Representations

Although the self-supervised pre-training of transformer models has resulted in the revolutionizing of natural language processing (NLP) applications and the achievement of state-of-the-art results with regard to various benchmarks, this process is still vulnerable to small and imperceptible permutations originating from legitimate inputs. Intuitively, the representations should be similar in the feature space with subtle input permutations, while large variations occur with different meanings. This motivates us to investigate the learning of robust textual representation in a contrastive manner. However, it is non-trivial to obtain opposing semantic instances for textual samples. In this study, we propose a disentangled contrastive learning method that separately optimizes the uniformity and alignment of representations without negative sampling. Specifically, we introduce the concept of momentum representation consistency to align features and leverage power normalization while conforming the uniformity. Our experimental results for the NLP benchmarks demonstrate that our approach can obtain better results compared with the baselines, as well as achieve promising improvements with invariance tests and adversarial attacks. The code is available in https://github.com/zjunlp/DCL.Comment: Work in progres

Effects of Data Duplication in Pretraining

학위논문(석사) -- 서울대학교대학원 : 데이터사이언스대학원 데이터사이언스학과, 2023. 2. 이재진.This paper studies the effect of deduplication in training data on language models, such as BERT (the encoder-based model) and GPT-2 (the decoder-based model). Previous studies focus on memorizing duplicates in the training dataset whereas we perform several experiments with data deduplication. The pretraining data is first clustered by MinhashLSH, a stochastic method for finding near-duplicate documents in large corpus data, and then deduplicated by Jaccard similarity with various threshold values. Then, the models are finetuned with different downstream tasks. The experimental result indicates that GPT-2 works better with the deduplication, whereas BERT works differently depending on the tasks. It is due to the difference in self-supervised learning methods between BERT and GPT-2. The duplicated data may work on BERT as data augmentation through random masking in its data preprocessing stage. Data duplication may introduce biases and lead to overfitting, but the effect depends on the amount of duplicated data. To improve performance, data deduplication with proper granularity is essential in language model training.이 연구는 BERT(인코더 기반 모델) 및 GPT-2(디코더 기반 모델)와 같은 언어 모델에 대한 훈련 데이터의 중복 제거 효과를 제시하는 데 목적이 있다. 기존 연구에서는 생성 모델에 한하여 중복 제거의 이점을 밝혔으며, 모델이 암기된 텍스트를 덜 생성하고 모델의 훈련 단계가 더 적게 필요하다는 것을 발견하였다. 이에 덧붙여 현 연구에서는 데이터 중복 제거에 대해 몇 가지 추가적인 실험을 수행한다. 사전 학습 데이터는 우선 MinhashLSH(대규모 말뭉치 데이터에서 유사한 문서를 찾기 위한 확률론적 방법)로 클러스터링 한 다음, 다양한 임계값의 Jaccard 유사성으로 중복 document를 제거하는 전처리 과정을 거친다. 구성된 데이터셋을 기반으로 사전 학습을 진행하고, 이후 다양한 downstream 작업에 finetuning한다. GPT-2는 중복 제거된 모델에서 더 높은 성능을 내는 반면, BERT는 downstream 작업에 따라 다른 성능을 보인다. 이는 BERT와 GPT-2의 self-supervised learning 방식의 차이 때문이다. BERT에서는 데이터 전처리 단계에서 랜덤 마스킹 방식을 통해 중복된 데이터가 오히려 데이터 augmentation으로 작용할 수 있다. 그렇지만 결과적으로 데이터 중복은 편향을 도입하고 과적합으로 이어질 수 있으며, 그 효과는 중복 데이터의 양에 따라 다를 수 있다. 따라서 성능을 향상시키기 위해선 언어 모델 훈련에서 적절한 임계값의 데이터 중복 제거가 필수적이다.Chapter 1. Introduction １ 1.1. Study Background １ 1.2. Purpose of Research ３ 1.3. Related Work ４ Chapter 2. Approach ６ 2.1. Pretraining Models ６ 2.2. Pretraining Dataset ７ 2.3. Near Deduplication ７ 2.4. Injection of Exact Document Duplication １０ Chapter 3. Experiments １２ 3.1. Near Deduplication Results １２ 3.2. Duplication Injection Results １４ Chapter 4. Conclusion １６ 4.1. Discussion and Future work １６석

Token-Modification Adversarial Attacks for Natural Language Processing: A Survey

There are now many adversarial attacks for natural language processing systems. Of these, a vast majority achieve success by modifying individual document tokens, which we call here a \textit{token-modification} attack. Each token-modification attack is defined by a specific combination of fundamental \textit{components}, such as a constraint on the adversary or a particular search algorithm. Motivated by this observation, we survey existing token-modification attacks and extract the components of each. We use an attack-independent framework to structure our survey which results in an effective categorisation of the field and an easy comparison of components. We hope this survey will guide new researchers to this field and spark further research into the individual attack components.Comment: 8 pages, 1 figur