Semantic-Retention Attack for Continual Named Entity Recognition

Abstract

Continual Named Entity Recognition (CNER) aims to learn new entity types while preventing the catastrophic forgetting of previously learned types, expanding the flexibility of NER. However, the robustness of CNER models, which are typically evaluated through adversarial attacks, hasn’t been fully investigated due to two crucial challenges. Firstly, most works only focus on attacking NER models, in which the model is designed for a fixed dataset without considering the dynamic nature of real-world scenarios. These methods are not tailored to the challenges of the continual learning setting. Secondly, current textual attacks discretely change target words, which are widely used in sentence-level or document-level tasks. But in the word-level CNER task, even changes in one character may lead to a shift in the true label, making it difficult to reliably evaluate the results of the attack. Additionally, this type of attack is easily observed and lacks stealthiness. Thus, we propose a novel attack approach named Semantic-Retention Attack (SRA). To fit the continual learning tasks, SRA disrupts CNER models by enhancing catastrophic forgetting and knowledge confusion. To improve the reliability and stealthiness of the proposed attack, we perform a continuous transformation on the discrete texts and then apply a trainable SRA on them, ensuring the retention of the original semantics of the texts and avoiding changes in word-level ground truth labels. Experiments across ten CNER settings show our approach decreases performance to at most 51.63%, reaching the best degradation independent of the initial performance of CNER models while maintaining the best stealthiness, which exposes security vulnerabilities