3,879 research outputs found
Interpreting Adversarially Trained Convolutional Neural Networks
We attempt to interpret how adversarially trained convolutional neural
networks (AT-CNNs) recognize objects. We design systematic approaches to
interpret AT-CNNs in both qualitative and quantitative ways and compare them
with normally trained models. Surprisingly, we find that adversarial training
alleviates the texture bias of standard CNNs when trained on object recognition
tasks, and helps CNNs learn a more shape-biased representation. We validate our
hypothesis from two aspects. First, we compare the salience maps of AT-CNNs and
standard CNNs on clean images and images under different transformations. The
comparison could visually show that the prediction of the two types of CNNs is
sensitive to dramatically different types of features. Second, to achieve
quantitative verification, we construct additional test datasets that destroy
either textures or shapes, such as style-transferred version of clean data,
saturated images and patch-shuffled ones, and then evaluate the classification
accuracy of AT-CNNs and normal CNNs on these datasets. Our findings shed some
light on why AT-CNNs are more robust than those normally trained ones and
contribute to a better understanding of adversarial training over CNNs from an
interpretation perspective.Comment: To apper in ICML1
An Integrated Approach to Produce Robust Models with High Efficiency
Deep Neural Networks (DNNs) needs to be both efficient and robust for
practical uses. Quantization and structure simplification are promising ways to
adapt DNNs to mobile devices, and adversarial training is the most popular
method to make DNNs robust. In this work, we try to obtain both features by
applying a convergent relaxation quantization algorithm, Binary-Relax (BR), to
a robust adversarial-trained model, ResNets Ensemble via Feynman-Kac Formalism
(EnResNet). We also discover that high precision, such as ternary (tnn) and
4-bit, quantization will produce sparse DNNs. However, this sparsity is
unstructured under advarsarial training. To solve the problems that adversarial
training jeopardizes DNNs' accuracy on clean images and the struture of
sparsity, we design a trade-off loss function that helps DNNs preserve their
natural accuracy and improve the channel sparsity. With our trade-off loss
function, we achieve both goals with no reduction of resistance under weak
attacks and very minor reduction of resistance under strong attcks. Together
with quantized EnResNet with trade-off loss function, we provide robust models
that have high efficiency
A Theoretically Guaranteed Deep Optimization Framework for Robust Compressive Sensing MRI
Magnetic Resonance Imaging (MRI) is one of the most dynamic and safe imaging
techniques available for clinical applications. However, the rather slow speed
of MRI acquisitions limits the patient throughput and potential indi cations.
Compressive Sensing (CS) has proven to be an efficient technique for
accelerating MRI acquisition. The most widely used CS-MRI model, founded on the
premise of reconstructing an image from an incompletely filled k-space, leads
to an ill-posed inverse problem. In the past years, lots of efforts have been
made to efficiently optimize the CS-MRI model. Inspired by deep learning
techniques, some preliminary works have tried to incorporate deep architectures
into CS-MRI process. Unfortunately, the convergence issues (due to the
experience-based networks) and the robustness (i.e., lack real-world noise
modeling) of these deeply trained optimization methods are still missing. In
this work, we develop a new paradigm to integrate designed numerical solvers
and the data-driven architectures for CS-MRI. By introducing an optimal
condition checking mechanism, we can successfully prove the convergence of our
established deep CS-MRI optimization scheme. Furthermore, we explicitly
formulate the Rician noise distributions within our framework and obtain an
extended CS-MRI network to handle the real-world nosies in the MRI process.
Extensive experimental results verify that the proposed paradigm outperforms
the existing state-of-the-art techniques both in reconstruction accuracy and
efficiency as well as robustness to noises in real scene
- …