1,248 research outputs found
To Trust or Not To Trust Prediction Scores for Membership Inference Attacks
Membership inference attacks (MIAs) aim to determine whether a specific
sample was used to train a predictive model. Knowing this may indeed lead to a
privacy breach. Most MIAs, however, make use of the model's prediction scores -
the probability of each output given some input - following the intuition that
the trained model tends to behave differently on its training data. We argue
that this is a fallacy for many modern deep network architectures.
Consequently, MIAs will miserably fail since overconfidence leads to high
false-positive rates not only on known domains but also on out-of-distribution
data and implicitly acts as a defense against MIAs. Specifically, using
generative adversarial networks, we are able to produce a potentially infinite
number of samples falsely classified as part of the training data. In other
words, the threat of MIAs is overestimated, and less information is leaked than
previously assumed. Moreover, there is actually a trade-off between the
overconfidence of models and their susceptibility to MIAs: the more classifiers
know when they do not know, making low confidence predictions, the more they
reveal the training data.Comment: 15 pages, 8 figures, 10 table
The Compact Support Neural Network
Neural networks are popular and useful in many fields, but they have the
problem of giving high confidence responses for examples that are away from the
training data. This makes the neural networks very confident in their
prediction while making gross mistakes, thus limiting their reliability for
safety-critical applications such as autonomous driving, space exploration,
etc. This paper introduces a novel neuron generalization that has the standard
dot-product-based neuron and the {\color{black} radial basis function (RBF)}
neuron as two extreme cases of a shape parameter. Using a rectified linear unit
(ReLU) as the activation function results in a novel neuron that has compact
support, which means its output is zero outside a bounded domain. To address
the difficulties in training the proposed neural network, it introduces a novel
training method that takes a pretrained standard neural network that is
fine-tuned while gradually increasing the shape parameter to the desired value.
The theoretical findings of the paper are a bound on the gradient of the
proposed neuron and a proof that a neural network with such neurons has the
universal approximation property. This means that the network can approximate
any continuous and integrable function with an arbitrary degree of accuracy.
The experimental findings on standard benchmark datasets show that the proposed
approach has smaller test errors than state-of-the-art competing methods and
outperforms the competing methods in detecting out-of-distribution samples on
two out of three datasets.Comment: 13 pages, 6 figure
Multidimensional Uncertainty-Aware Evidential Neural Networks
Traditional deep neural networks (NNs) have significantly contributed to the
state-of-the-art performance in the task of classification under various
application domains. However, NNs have not considered inherent uncertainty in
data associated with the class probabilities where misclassification under
uncertainty may easily introduce high risk in decision making in real-world
contexts (e.g., misclassification of objects in roads leads to serious
accidents). Unlike Bayesian NN that indirectly infer uncertainty through weight
uncertainties, evidential NNs (ENNs) have been recently proposed to explicitly
model the uncertainty of class probabilities and use them for classification
tasks. An ENN offers the formulation of the predictions of NNs as subjective
opinions and learns the function by collecting an amount of evidence that can
form the subjective opinions by a deterministic NN from data. However, the ENN
is trained as a black box without explicitly considering inherent uncertainty
in data with their different root causes, such as vacuity (i.e., uncertainty
due to a lack of evidence) or dissonance (i.e., uncertainty due to conflicting
evidence). By considering the multidimensional uncertainty, we proposed a novel
uncertainty-aware evidential NN called WGAN-ENN (WENN) for solving an
out-of-distribution (OOD) detection problem. We took a hybrid approach that
combines Wasserstein Generative Adversarial Network (WGAN) with ENNs to jointly
train a model with prior knowledge of a certain class, which has high vacuity
for OOD samples. Via extensive empirical experiments based on both synthetic
and real-world datasets, we demonstrated that the estimation of uncertainty by
WENN can significantly help distinguish OOD samples from boundary samples. WENN
outperformed in OOD detection when compared with other competitive
counterparts.Comment: AAAI 202
Towards robust convolutional neural networks in challenging environments
Image classification is one of the fundamental tasks in the field of computer vision. Although Artificial Neural Network (ANN) showed a lot of promise in this field, the lack of efficient computer hardware subdued its potential to a great extent. In the early 2000s, advances in hardware coupled with better network design saw the dramatic rise of Convolutional Neural Network (CNN). Deep CNNs pushed the State-of-The-Art (SOTA) in a number of vision tasks, including image classification, object detection, and segmentation. Presently, CNNs dominate these tasks. Although CNNs exhibit impressive classification performance on clean images, they are vulnerable to distortions, such as noise and blur. Fine-tuning a pre-trained CNN on mutually exclusive or a union set of distortions is a brute-force solution. This iterative fine-tuning process with all known types of distortion is, however, exhaustive and the network struggles to handle unseen distortions. CNNs are also vulnerable to image translation or shift, partly due to common Down-Sampling (DS) layers, e.g., max-pooling and strided convolution. These operations violate the Nyquist sampling rate and cause aliasing. The textbook solution is low-pass filtering (blurring) before down-sampling, which can benefit deep networks as well. Even so, non-linearity units, such as ReLU, often re-introduce the problem, suggesting that blurring alone may not suffice. Another important but under-explored issue for CNNs is unknown or Open Set Recognition (OSR). CNNs are commonly designed for closed set arrangements, where test instances only belong to some ‘Known Known’ (KK) classes used in training. As such, they predict a class label for a test sample based on the distribution of the KK classes. However, when used under the OSR setup (where an input may belong to an ‘Unknown Unknown’ or UU class), such a network will always classify a test instance as one of the KK classes even if it is from a UU class. Historically, CNNs have struggled with detecting objects in images with large difference in scale, especially small objects. This is because the DS layers inside a CNN often progressively wipe out the signal from small objects. As a result, the final layers are left with no signature from these objects leading to degraded performance. In this work, we propose solutions to the above four problems. First, we improve CNN robustness against distortion by proposing DCT based augmentation, adaptive regularisation, and noise suppressing Activation Functions (AF). Second, to ensure further performance gain and robustness to image transformations, we introduce anti-aliasing properties inside the AF and propose a novel DS method called blurpool. Third, to address the OSR problem, we propose a novel training paradigm that ensures detection of UU classes and accurate classification of the KK classes. Finally, we introduce a novel CNN that enables a deep detector to identify small objects with high precision and recall. We evaluate our methods on a number of benchmark datasets and demonstrate that they outperform contemporary methods in the respective problem set-ups.Doctor of Philosoph
- …