2,655 research outputs found
Adversarial Feature Stacking for Accurate and Robust Predictions
Deep Neural Networks (DNNs) have achieved remarkable performance on a variety
of applications but are extremely vulnerable to adversarial perturbation. To
address this issue, various defense methods have been proposed to enhance model
robustness. Unfortunately, the most representative and promising methods, such
as adversarial training and its variants, usually degrade model accuracy on
benign samples, limiting practical utility. This indicates that it is difficult
to extract both robust and accurate features using a single network under
certain conditions, such as limited training data, resulting in a trade-off
between accuracy and robustness. To tackle this problem, we propose an
Adversarial Feature Stacking (AFS) model that can jointly take advantage of
features with varied levels of robustness and accuracy, thus significantly
alleviating the aforementioned trade-off. Specifically, we adopt multiple
networks adversarially trained with different perturbation budgets to extract
either more robust features or more accurate features. These features are then
fused by a learnable merger to give final predictions. We evaluate the AFS
model on CIFAR-10 and CIFAR-100 datasets with strong adaptive attack methods,
which significantly advances the state-of-the-art in terms of the trade-off.
Without extra training data, the AFS model achieves a benign accuracy
improvement of 6% on CIFAR-10 and 9% on CIFAR-100 with comparable or even
stronger robustness than the state-of-the-art adversarial training methods.
This work demonstrates the feasibility to obtain both accurate and robust
models under the circumstances of limited training data
Stacked Penalized Logistic Regression for Selecting Views in Multi-View Learning
In biomedical research, many different types of patient data can be
collected, such as various types of omics data and medical imaging modalities.
Applying multi-view learning to these different sources of information can
increase the accuracy of medical classification models compared with
single-view procedures. However, collecting biomedical data can be expensive
and/or burdening for patients, so that it is important to reduce the amount of
required data collection. It is therefore necessary to develop multi-view
learning methods which can accurately identify those views that are most
important for prediction. In recent years, several biomedical studies have used
an approach known as multi-view stacking (MVS), where a model is trained on
each view separately and the resulting predictions are combined through
stacking. In these studies, MVS has been shown to increase classification
accuracy. However, the MVS framework can also be used for selecting a subset of
important views. To study the view selection potential of MVS, we develop a
special case called stacked penalized logistic regression (StaPLR). Compared
with existing view-selection methods, StaPLR can make use of faster
optimization algorithms and is easily parallelized. We show that nonnegativity
constraints on the parameters of the function which combines the views play an
important role in preventing unimportant views from entering the model. We
investigate the performance of StaPLR through simulations, and consider two
real data examples. We compare the performance of StaPLR with an existing view
selection method called the group lasso and observe that, in terms of view
selection, StaPLR is often more conservative and has a consistently lower false
positive rate.Comment: 26 pages, 9 figures. Accepted manuscrip
- …