Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks

Transferability captures the ability of an attack against a machine-learning model to be effective against a different, potentially unknown, model. Empirical evidence for transferability has been shown in previous work, but the underlying reasons why an attack transfers or not are not yet well understood. In this paper, we present a comprehensive analysis aimed to investigate the transferability of both test-time evasion and training-time poisoning attacks. We provide a unifying optimization framework for evasion and poisoning attacks, and a formal definition of transferability of such attacks. We highlight two main factors contributing to attack transferability: the intrinsic adversarial vulnerability of the target model, and the complexity of the surrogate model used to optimize the attack. Based on these insights, we define three metrics that impact an attack's transferability. Interestingly, our results derived from theoretical analysis hold for both evasion and poisoning attacks, and are confirmed experimentally using a wide range of linear and non-linear classifiers and datasets

Towards Robust and Reproducible Active Learning Using Neural Networks

Active learning (AL) is a promising ML paradigm that has the potential to parse through large unlabeled data and help reduce annotation cost in domains where labeling entire data can be prohibitive. Recently proposed neural network based AL methods use different heuristics to accomplish this goal. In this study, we show that recent AL methods offer a gain over random baseline under a brittle combination of experimental conditions. We demonstrate that such marginal gains vanish when experimental factors are changed, leading to reproducibility issues and suggesting that AL methods lack robustness. We also observe that with a properly tuned model, which employs recently proposed regularization techniques, the performance significantly improves for all AL methods including the random sampling baseline, and performance differences among the AL methods become negligible. Based on these observations, we suggest a set of experiments that are critical to assess the true effectiveness of an AL method. To facilitate these experiments we also present an open source toolkit. We believe our findings and recommendations will help advance reproducible research in robust AL using neural networks

Towards adversarial robustness with 01 lossmodels, and novel convolutional neural netsystems for ultrasound images

This dissertation investigates adversarial robustness with 01 loss models and a novel convolutional neural net systems for vascular ultrasound images. In the first part, the dissertation presents stochastic coordinate descent for 01 loss and its sensitivity to adversarial attacks. The study here suggests that 01 loss may be more resilient to adversarial attacks than the hinge loss and further work is required. In the second part, this dissertation proposes sign activation network with a novel gradient-free stochastic coordinate descent algorithm and its ensembling model. The study here finds that the ensembling model gives a high minimum distortion (as measured by HopSkipJump) compared to full precision, binary, and convolutional neural networks, and explains this phenomenon by measuring the transferability between networks in an ensemble. In the last part, this dissertation tackles three important segmentation problems for vascular ultrasound images with novel convolutional neural networks. More specifically, these three problems are: (1) vessel segmentation in the internal carotid artery, (2) vessel segmentation in the entire carotid system, and (3) vessel and plaque segmentation in the entire carotid system. The study here represents a first successful step towards the automated segmentation of vessel and plaque in carotid artery ultrasound images and is an important step in creating a system that can independently evaluate carotid ultrasounds