Decision-BADGE: Decision-based Adversarial Batch Attack with Directional Gradient Estimation

The susceptibility of deep neural networks (DNNs) to adversarial examples has prompted an increase in the deployment of adversarial attacks. Image-agnostic universal adversarial perturbations (UAPs) are much more threatening, but many limitations exist to implementing UAPs in real-world scenarios where only binary decisions are returned. In this research, we propose Decision-BADGE, a novel method to craft universal adversarial perturbations for executing decision-based black-box attacks. To optimize perturbation with decisions, we addressed two challenges, namely the magnitude and the direction of the gradient. First, we use batch loss, differences from distributions of ground truth, and accumulating decisions in batches to determine the magnitude of the gradient. This magnitude is applied in the direction of the revised simultaneous perturbation stochastic approximation (SPSA) to update the perturbation. This simple yet efficient method can be easily extended to score-based attacks as well as targeted attacks. Experimental validation across multiple victim models demonstrates that the Decision-BADGE outperforms existing attack methods, even image-specific and score-based attacks. In particular, our proposed method shows a superior success rate with less training time. The research also shows that Decision-BADGE can successfully deceive unseen victim models and accurately target specific classes.Comment: 9 pages (7 pages except for references), 4 figures, 4 table

Towards Optimization and Robustification of Data-Driven Models

In the past two decades, data-driven models have experienced a renaissance, with notable success achieved through the use of models such as deep neural networks (DNNs) in various applications. However, complete reliance on intelligent machine learning systems is still a distant dream. Nevertheless, the initial success of data-driven approaches presents a promising path for building trustworthy data-oriented models. This thesis aims to take a few steps toward improving the performance of existing data-driven frameworks in both the training and testing phases. Specifically, we focus on several key questions: 1) How to efficiently design optimization methods for learning algorithms that can be used in parallel settings and also when first-order information is unavailable? 2) How to revise existing adversarial attacks on DNNs to structured attacks with minimal distortion of benign samples? 3) How to integrate attention models such as Transformers into data-driven inertial navigation systems? 4) How to address the lack of data problem for existing data-driven models and enhance the performance of existing semi-supervised learning (SSL) methods? In terms of parallel optimization methods, our research focuses on investigating a delay-aware asynchronous variance-reduced coordinate descent approach. Additionally, we explore the development of a proximal zeroth-order algorithm for nonsmooth nonconvex problems when first-order information is unavailable. We also extend our study to zeroth-order stochastic gradient descent problems. As for robustness, we develop a structured white-box adversarial attack to enhance research on robust machine learning schemes. Furthermore, our research investigates a group threat model in which adversaries can only perturb image segments rather than the entire image to generate adversarial examples. We also explore the use of attention models, specifically Transformer models, for deep inertial navigation systems based on the Inertial Measurement Unit (IMU). In addressing the problem of data scarcity during the training process, we propose a solution that involves quantizing the uncertainty from the unlabeled data and corresponding pseudo-labels, and incorporating it into the loss term to compensate for noisy pseudo-labeling. We also extend the generic semi-supervised method for data-driven noise suppression frameworks by utilizing a reinforcement learning (RL) model to learn contrastive features in an SSL fashion. Each chapter of the thesis presents the problem and our solutions using concrete algorithms. We verify our approach through comparisons with existing methods on different benchmarks and discuss future research directions

Adaptive Stochastic Optimisation of Nonconvex Composite Objectives

In this paper, we propose and analyse a family of generalised stochastic composite mirror descent algorithms. With adaptive step sizes, the proposed algorithms converge without requiring prior knowledge of the problem. Combined with an entropy-like update-generating function, these algorithms perform gradient descent in the space equipped with the maximum norm, which allows us to exploit the low-dimensional structure of the decision sets for high-dimensional problems. Together with a sampling method based on the Rademacher distribution and variance reduction techniques, the proposed algorithms guarantee a logarithmic complexity dependence on dimensionality for zeroth-order optimisation problems.Comment: arXiv admin note: substantial text overlap with arXiv:2208.0457