10 research outputs found
Decorrelative Network Architecture for Robust Electrocardiogram Classification
Artificial intelligence has made great progresses in medical data analysis,
but the lack of robustness and interpretability has kept these methods from
being widely deployed. In particular, data-driven models are vulnerable to
adversarial attacks, which are small, targeted perturbations that dramatically
degrade model performance. As a recent example, while deep learning has shown
impressive performance in electrocardiogram (ECG) classification, Han et al.
crafted realistic perturbations that fooled the network 74% of the time [2020].
Current adversarial defense paradigms are computationally intensive and
impractical for many high dimensional problems. Previous research indicates
that a network vulnerability is related to the features learned during
training. We propose a novel approach based on ensemble decorrelation and
Fourier partitioning for training parallel network arms into a decorrelated
architecture to learn complementary features, significantly reducing the chance
of a perturbation fooling all arms of the deep learning model. We test our
approach in ECG classification, demonstrating a much-improved 77.2% chance of
at least one correct network arm on the strongest adversarial attack tested, in
contrast to a 21.7% chance from a comparable ensemble. Our approach does not
require expensive optimization with adversarial samples, and thus can be scaled
to large problems. These methods can easily be applied to other tasks for
improved network robustness.Comment: 12 pages, 6 figure
Physical Passive Patch Adversarial Attacks on Visual Odometry Systems
Deep neural networks are known to be susceptible to adversarial perturbations
-- small perturbations that alter the output of the network and exist under
strict norm limitations. While such perturbations are usually discussed as
tailored to a specific input, a universal perturbation can be constructed to
alter the model's output on a set of inputs. Universal perturbations present a
more realistic case of adversarial attacks, as awareness of the model's exact
input is not required. In addition, the universal attack setting raises the
subject of generalization to unseen data, where given a set of inputs, the
universal perturbations aim to alter the model's output on out-of-sample data.
In this work, we study physical passive patch adversarial attacks on visual
odometry-based autonomous navigation systems. A visual odometry system aims to
infer the relative camera motion between two corresponding viewpoints, and is
frequently used by vision-based autonomous navigation systems to estimate their
state. For such navigation systems, a patch adversarial perturbation poses a
severe security issue, as it can be used to mislead a system onto some
collision course. To the best of our knowledge, we show for the first time that
the error margin of a visual odometry model can be significantly increased by
deploying patch adversarial attacks in the scene. We provide evaluation on
synthetic closed-loop drone navigation data and demonstrate that a comparable
vulnerability exists in real data. A reference implementation of the proposed
method and the reported experiments is provided at
https://github.com/patchadversarialattacks/patchadversarialattacks.Comment: Accepted to ACCV 202
Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR
There has been much discussion of the right to explanation in the EU General
Data Protection Regulation, and its existence, merits, and disadvantages.
Implementing a right to explanation that opens the black box of algorithmic
decision-making faces major legal and technical barriers. Explaining the
functionality of complex algorithmic decision-making systems and their
rationale in specific cases is a technically challenging problem. Some
explanations may offer little meaningful information to data subjects, raising
questions around their value. Explanations of automated decisions need not
hinge on the general public understanding how algorithmic systems function.
Even though such interpretability is of great importance and should be pursued,
explanations can, in principle, be offered without opening the black box.
Looking at explanations as a means to help a data subject act rather than
merely understand, one could gauge the scope and content of explanations
according to the specific goal or action they are intended to support. From the
perspective of individuals affected by automated decision-making, we propose
three aims for explanations: (1) to inform and help the individual understand
why a particular decision was reached, (2) to provide grounds to contest the
decision if the outcome is undesired, and (3) to understand what would need to
change in order to receive a desired result in the future, based on the current
decision-making model. We assess how each of these goals finds support in the
GDPR. We suggest data controllers should offer a particular type of
explanation, unconditional counterfactual explanations, to support these three
aims. These counterfactual explanations describe the smallest change to the
world that can be made to obtain a desirable outcome, or to arrive at the
closest possible world, without needing to explain the internal logic of the
system
Towards robust convolutional neural networks in challenging environments
Image classification is one of the fundamental tasks in the field of computer vision. Although Artificial Neural Network (ANN) showed a lot of promise in this field, the lack of efficient computer hardware subdued its potential to a great extent. In the early 2000s, advances in hardware coupled with better network design saw the dramatic rise of Convolutional Neural Network (CNN). Deep CNNs pushed the State-of-The-Art (SOTA) in a number of vision tasks, including image classification, object detection, and segmentation. Presently, CNNs dominate these tasks. Although CNNs exhibit impressive classification performance on clean images, they are vulnerable to distortions, such as noise and blur. Fine-tuning a pre-trained CNN on mutually exclusive or a union set of distortions is a brute-force solution. This iterative fine-tuning process with all known types of distortion is, however, exhaustive and the network struggles to handle unseen distortions. CNNs are also vulnerable to image translation or shift, partly due to common Down-Sampling (DS) layers, e.g., max-pooling and strided convolution. These operations violate the Nyquist sampling rate and cause aliasing. The textbook solution is low-pass filtering (blurring) before down-sampling, which can benefit deep networks as well. Even so, non-linearity units, such as ReLU, often re-introduce the problem, suggesting that blurring alone may not suffice. Another important but under-explored issue for CNNs is unknown or Open Set Recognition (OSR). CNNs are commonly designed for closed set arrangements, where test instances only belong to some ‘Known Known’ (KK) classes used in training. As such, they predict a class label for a test sample based on the distribution of the KK classes. However, when used under the OSR setup (where an input may belong to an ‘Unknown Unknown’ or UU class), such a network will always classify a test instance as one of the KK classes even if it is from a UU class. Historically, CNNs have struggled with detecting objects in images with large difference in scale, especially small objects. This is because the DS layers inside a CNN often progressively wipe out the signal from small objects. As a result, the final layers are left with no signature from these objects leading to degraded performance. In this work, we propose solutions to the above four problems. First, we improve CNN robustness against distortion by proposing DCT based augmentation, adaptive regularisation, and noise suppressing Activation Functions (AF). Second, to ensure further performance gain and robustness to image transformations, we introduce anti-aliasing properties inside the AF and propose a novel DS method called blurpool. Third, to address the OSR problem, we propose a novel training paradigm that ensures detection of UU classes and accurate classification of the KK classes. Finally, we introduce a novel CNN that enables a deep detector to identify small objects with high precision and recall. We evaluate our methods on a number of benchmark datasets and demonstrate that they outperform contemporary methods in the respective problem set-ups.Doctor of Philosoph