Towards Robust Deep Neural Networks

Deep neural networks (DNNs) enable state-of-the-art performance for most machine learning tasks. Unfortunately, they are vulnerable to attacks, such as Trojans during training and Adversarial Examples at test time. Adversarial Examples are inputs with carefully crafted perturbations added to benign samples. In the Computer Vision domain, while the perturbations being imperceptible to humans, Adversarial Examples can successfully misguide or fool DNNs. Meanwhile, Trojan or backdoor attacks involve attackers tampering with the training process, for example, to inject poisoned training data to embed a backdoor into the network that can be activated during model deployment when the Trojan triggers (known only to the attackers) appear in the model’s inputs. This dissertation investigates methods of building robust DNNs against these training-time and test-time threats. Recognising the threat of Adversarial Examples in the malware domain, this research considers the problem of realising a robust DNN-based malware detector against Adversarial Example attacks by developing a Bayesian adversarial learning algorithm. In contrast to vision tasks, adversarial learning in a domain without a differentiable or invertible mapping function from the problemspace (such as software code inputs) to the feature space is hard. The study proposes an alternative; performing adversarial learning in the feature space and proving the projection of perturbed yet, valid malware, in the problem space into the feature space will be a subset of feature-space adversarial attacks. The Bayesian approach improves benign performance, provably bounds the difference between adversarial risk and empirical risk and improves robustness against increasingly large attack budgets not employed during training. To investigate the problem of improving the robustness of DNNs against Adversarial Examples–carefully crafted perturbation added to inputs—in the Computer Vision domain, the research considers the problem of developing a Bayesian learning algorithm to realise a robust DNN against Adversarial Examples in the CV domain. Accordingly, a novel Bayesian learning method is designed that conceptualises an information gain objective to measure and force the information learned from both benign and Adversarial Examples to be similar. This method proves that minimising this information gain objective further tightens the bound of the difference between adversarial risk and empirical risk to move towards a basis for a principled method of adversarially training BNNs. Recognising the threat from backdoor or Trojan attacks against DNNs, the research considers the problem of finding a robust defence method that is effective against Trojan attacks. The research explores a new idea in the domain; sanitisation of inputs and proposes Februus to neutralise highly potent and insidious Trojan attacks on DNN systems at run-time. In Trojan attacks, an adversary activates a backdoor crafted in a deep neural network model using a secret trigger, a Trojan, applied to any input to alter the model’s decision to a target prediction—a target determined by and only known to the attacker. Februus sanitises the incoming input by surgically removing the potential trigger artifacts and restoring the input for the classification task. Februus enables effective Trojan mitigation by sanitising inputs with no loss of performance for sanitised inputs, trojaned or benign. This method is highly effective at defending against advanced Trojan attack variants as well as challenging, adaptive attacks where attackers have full knowledge of the defence method. Investigating the connections between Trojan attacks and spatially constrained Adversarial Examples or so-called Adversarial Patches in the input space, the research exposes an emerging threat; an attack exploiting the vulnerability of a DNN to generate naturalistic adversarial patches as universal triggers. For the first time, a method based on Generative Adversarial Networks is developed to exploit a GAN’s latent space to search for universal naturalistic adversarial patches. The proposed attack’s advantage is its ability to exert a high level of control, enabling attackers to craft naturalistic adversarial patches that are highly effective, robust against state-of-the-art DNNs, and deployable in the physical world without needing to interfere with the model building process or risking discovery. Until now, this has only been demonstrably possible using Trojan attack methods.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 202

COMPLEX-VALUED APPROACH TO KURAMOTO-LIKE OSCILLATORS

The Kuramoto Model (KM) is a nonlinear model widely used to model synchrony in a network of oscillators – from the synchrony of the flashing fireflies to the hand clapping in an auditorium. Recently, a modification of the KM (complex-valued KM) was introduced with an analytical solution expressed in terms of a matrix exponential, and consequentially, its eigensystem. Remarkably, the analytical KM and the original KM bear significant similarities, even with phase lag introduced, despite being determined by distinct systems. We found that this approach gives a geometric perspective of synchronization phenomena in terms of complex eigenmodes, which in turn offers a unified geometry for synchrony, chimera states, and waves in nonlinear oscillator networks. These insights are presented in Chapter 2 of this thesis. This surprising connection between the eigenspectrum of the adjacency matrix of a ring graph and its Kuramoto dynamics invites the question: what is the eigenspectrum of joins of circulant matrices? We answered this question in Chapter 3 of this thesis

Computation of electromagnetic forces in the windings of amorphous core transformers

Electromagnetic forces generated by the short circuit current and leakage flux in low- and high-voltage windings of distribution transformers as well as amorphous core transformers will cause the translation, destruction, and explosion of the windings. Thus, the investigation of these forces plays a significant role for researchers and manufacturers. Many authors have recently used the finite element method to analyze electromagnetic forces. In this paper, an analytic model is first developed for magnetic vector potential formulations to compute the electromagnetic forces (i.e., axial and radial forces) acting on the low- and high-voltage windings of an amorphous core transformer. The finite element technique is then presented to validate the results obtained from the analytical model. The developed model is applied to an actual problem

Micromachining of carbon nanofiller reinforced polymern nanocomposites

The modern industry has been observing a growing demand for micromanufacturing of nanocomposites. This is driven by the miniaturisation trend to obtain products with micro features, high accuracy and light weight. From an engineering perspective, a miniaturised system can provide many benefits over its predecessors such as precision operation, mobility, or power consumption. Based on these, many techniques of micro-manufacturing have been applied, and micromilling of nanocomposites has shown a huge potential to be applied in this field due to its high capability in producing high-complexity-3D micro-features in a wide variety of workpiece materials, with high dimensional accuracy. However, micromilling of nanocomposites is deemed to be a complicated process due to the anisotropic, heterogeneous structure and advanced mechanical properties of these materials associated with the size effects in micromachining. Also, applying micromachining of nanocomposites is a principal approach to bridge the knowledge gap between macro and micro/nano cuttings which is identified by the so-called “size effect”. This physical phenomenon exhibits by the association between various factors including cutting edge radius, negative tool rake angle, work-piece material microstructure, and minimum uncut chip thickness (MUCT) (or minimum chip load). These lead to unstable cutting regimes, resulting in corrupted chip formation, tool vibration and subsequently, low machined surface quality as well as high tool wear rate. The enormous potential of applying micromachining of nanocomposites in manufacturing micro-products, as well as the need to fill the knowledge gap of the field of this study, has prompted researchers to uncover the underlying mechanisms and allow appropriate adaption of this technique in industrial applications

Bayesian Learning with Information Gain Provably Bounds Risk for a Robust Adversarial Defense

We present a new algorithm to learn a deep neural network model robust against adversarial attacks. Previous algorithms demonstrate an adversarially trained Bayesian Neural Network (BNN) provides improved robustness. We recognize the adversarial learning approach for approximating the multi-modal posterior distribution of a Bayesian model can lead to mode collapse; consequently, the model's achievements in robustness and performance are sub-optimal. Instead, we first propose preventing mode collapse to better approximate the multi-modal posterior distribution. Second, based on the intuition that a robust model should ignore perturbations and only consider the informative content of the input, we conceptualize and formulate an information gain objective to measure and force the information learned from both benign and adversarial training instances to be similar. Importantly. we prove and demonstrate that minimizing the information gain objective allows the adversarial risk to approach the conventional empirical risk. We believe our efforts provide a step toward a basis for a principled method of adversarially training BNNs. Our model demonstrate significantly improved robustness--up to 20%--compared with adversarial training and Adv-BNN under PGD attacks with 0.035 distortion on both CIFAR-10 and STL-10 datasets.Comment: Published at ICML 2022. Code is available at https://github.com/baogiadoan/IG-BN

Should Vietnamese firm’s stocks be listed in a MSCI Global Equity index? Experience drawn from the sample of 30 countries

This study was conducted to investigate the argument of the increased co‐movement between the return of stocks, which are added to an MSCI Global Equity Index (MSCI Index) and the returns of the market index. It means that inclusion of the newly added stocks in an index leads to increased comovement between these stocks and the rest of the index. The MSCI Index is a broad and investable global equity benchmark and serve as the basis for over 500 exchange traded funds throughout the world. Our sample covers the MSCI Index inclusions from May 2003 to August 2008, corresponding to 16 adjustment quarters. Over this period, we have 1,274 index inclusion events over 46 countries in total. We found that inclusion into the MSCI Index leads to on average a higher beta with the national index. 21 out of the 30 countries in our sample experienced an increase in beta in the post‐inclusion period. Given the two stock exchanges in Vietnam are young in terms of a number of years since establishment and a small size of the market by the international standard, caution is required when evidence from wellestablished and matured markets used in this study is drawn. Nevertheless, the implications for listed firms in Vietnam are that their stocks will be more frequently traded by various groups of investors as long as the stocks are listed with an MSCI index, including the powerful MSCI Frontier Markets Indexes of 26 countries in the world

An Improved MobileNet for Disease Detection on Tomato Leaves

Tomatoes are widely grown vegetables, and farmers face challenges in caring for them, particularly regarding plant diseases. The MobileNet architecture is renowned for its simplicity and compatibility with mobile devices. This study introduces MobileNet as a deep learning model to enhance disease detection efficiency in tomato plants. The model is evaluated on a dataset of 2,064 tomato leaf images, encompassing early blight, leaf spot, yellow curl, and healthy leaves. Results demonstrate promising accuracy, exceeding 0.980 for disease classification and 0.975 for distinguishing between diseases and healthy cases. Moreover, the proposed model outperforms existing approaches in terms of accuracy and training time for plant leaf disease detection