401 research outputs found
ADVERSARY AWARE CONTINUAL LEARNING
Continual learning approaches are useful as they help the model to learn new information (classes) sequentially, while also retaining the previously acquired information (classes). However, these approaches are adversary agnostic, i.e., they do not consider the possibility of malicious attacks. In this dissertation, we have demonstrated that continual learning approaches are extremely vulnerable to the adversarial backdoor attacks, where an intelligent adversary can introduce small amount of misinformation to the model in the form of imperceptible backdoor pattern during training to cause deliberate forgetting of a specific class at test time. We then propose a novel defensive framework to counter such an insidious attack where, we use the attacker’s primary strength – hiding the backdoor pattern by making it imperceptible to humans – against it and propose to learn a perceptible (stronger) pattern (also during the training) that can overpower the attacker’s imperceptible (weaker) pattern. We demonstrate the effectiveness of the proposed defensive mechanism through various commonly used replay-based (both generative and exact replay-based) continual learning algorithms using CIFAR-10, CIFAR-100, and MNIST benchmark datasets. Most noteworthy, we show that our proposed defensive framework considerably improves the robustness of continual learning algorithms with ZERO knowledge of the attacker’s target task, attacker’s target class, shape, size, and location of the attacker’s pattern. The proposed defensive framework also does not depend on the underlying continual learning algorithm. We term our proposed defensive framework as Adversary Aware Continual Learning (AACL)
Semi-WTC: A Practical Semi-supervised Framework for Attack Categorization through Weight-Task Consistency
Supervised learning has been widely used for attack categorization, requiring
high-quality data and labels. However, the data is often imbalanced and it is
difficult to obtain sufficient annotations. Moreover, supervised models are
subject to real-world deployment issues, such as defending against unseen
artificial attacks. To tackle the challenges, we propose a semi-supervised
fine-grained attack categorization framework consisting of an encoder and a
two-branch structure and this framework can be generalized to different
supervised models. The multilayer perceptron with residual connection is used
as the encoder to extract features and reduce the complexity. The Recurrent
Prototype Module (RPM) is proposed to train the encoder effectively in a
semi-supervised manner. To alleviate the data imbalance problem, we introduce
the Weight-Task Consistency (WTC) into the iterative process of RPM by
assigning larger weights to classes with fewer samples in the loss function. In
addition, to cope with new attacks in real-world deployment, we propose an
Active Adaption Resampling (AAR) method, which can better discover the
distribution of unseen sample data and adapt the parameters of encoder.
Experimental results show that our model outperforms the state-of-the-art
semi-supervised attack detection methods with a 3% improvement in
classification accuracy and a 90% reduction in training time.Comment: Tech repor
Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey
Image classification systems recently made a giant leap with the advancement
of deep neural networks. However, these systems require an excessive amount of
labeled data to be adequately trained. Gathering a correctly annotated dataset
is not always feasible due to several factors, such as the expensiveness of the
labeling process or difficulty of correctly classifying data, even for the
experts. Because of these practical challenges, label noise is a common problem
in real-world datasets, and numerous methods to train deep neural networks with
label noise are proposed in the literature. Although deep neural networks are
known to be relatively robust to label noise, their tendency to overfit data
makes them vulnerable to memorizing even random noise. Therefore, it is crucial
to consider the existence of label noise and develop counter algorithms to fade
away its adverse effects to train deep neural networks efficiently. Even though
an extensive survey of machine learning techniques under label noise exists,
the literature lacks a comprehensive survey of methodologies centered
explicitly around deep learning in the presence of noisy labels. This paper
aims to present these algorithms while categorizing them into one of the two
subgroups: noise model based and noise model free methods. Algorithms in the
first group aim to estimate the noise structure and use this information to
avoid the adverse effects of noisy labels. Differently, methods in the second
group try to come up with inherently noise robust algorithms by using
approaches like robust losses, regularizers or other learning paradigms
AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
Recent advances in artificial intelligence (AI) have significantly
intensified research in the geoscience and remote sensing (RS) field. AI
algorithms, especially deep learning-based ones, have been developed and
applied widely to RS data analysis. The successful application of AI covers
almost all aspects of Earth observation (EO) missions, from low-level vision
tasks like super-resolution, denoising and inpainting, to high-level vision
tasks like scene classification, object detection and semantic segmentation.
While AI techniques enable researchers to observe and understand the Earth more
accurately, the vulnerability and uncertainty of AI models deserve further
attention, considering that many geoscience and RS tasks are highly
safety-critical. This paper reviews the current development of AI security in
the geoscience and RS field, covering the following five important aspects:
adversarial attack, backdoor attack, federated learning, uncertainty and
explainability. Moreover, the potential opportunities and trends are discussed
to provide insights for future research. To the best of the authors' knowledge,
this paper is the first attempt to provide a systematic review of AI
security-related research in the geoscience and RS community. Available code
and datasets are also listed in the paper to move this vibrant field of
research forward
A review of spam email detection: analysis of spammer strategies and the dataset shift problem
.Spam emails have been traditionally seen as just annoying and unsolicited emails containing advertisements, but they increasingly include scams, malware or phishing. In order to ensure the security and integrity for the users, organisations and researchers aim to develop robust filters for spam email detection. Recently, most spam filters based on machine learning algorithms published in academic journals report very high performance, but users are still reporting a rising number of frauds and attacks via spam emails. Two main challenges can be found in this field: (a) it is a very dynamic environment prone to the dataset shift problem and (b) it suffers from the presence of an adversarial figure, i.e. the spammer. Unlike classical spam email reviews, this one is particularly focused on the problems that this constantly changing environment poses. Moreover, we analyse the different spammer strategies used for contaminating the emails, and we review the state-of-the-art techniques to develop filters based on machine learning. Finally, we empirically evaluate and present the consequences of ignoring the matter of dataset shift in this practical field. Experimental results show that this shift may lead to severe degradation in the estimated generalisation performance, with error rates reaching values up to 48.81%.SIPublicación en abierto financiada por el Consorcio de Bibliotecas Universitarias de Castilla y León (BUCLE), con cargo al Programa Operativo 2014ES16RFOP009 FEDER 2014-2020 DE CASTILLA Y LEÓN, Actuación:20007-CL - Apoyo Consorcio BUCL
- …