1,547 research outputs found
A Unified Approximation Framework for Compressing and Accelerating Deep Neural Networks
Deep neural networks (DNNs) have achieved significant success in a variety of
real world applications, i.e., image classification. However, tons of
parameters in the networks restrict the efficiency of neural networks due to
the large model size and the intensive computation. To address this issue,
various approximation techniques have been investigated, which seek for a light
weighted network with little performance degradation in exchange of smaller
model size or faster inference. Both low-rankness and sparsity are appealing
properties for the network approximation. In this paper we propose a unified
framework to compress the convolutional neural networks (CNNs) by combining
these two properties, while taking the nonlinear activation into consideration.
Each layer in the network is approximated by the sum of a structured sparse
component and a low-rank component, which is formulated as an optimization
problem. Then, an extended version of alternating direction method of
multipliers (ADMM) with guaranteed convergence is presented to solve the
relaxed optimization problem. Experiments are carried out on VGG-16, AlexNet
and GoogLeNet with large image classification datasets. The results outperform
previous work in terms of accuracy degradation, compression rate and speedup
ratio. The proposed method is able to remarkably compress the model (with up to
4.9x reduction of parameters) at a cost of little loss or without loss on
accuracy.Comment: 8 pages, 5 figures, 6 table
Deep BCD-Net Using Identical Encoding-Decoding CNN Structures for Iterative Image Recovery
In "extreme" computational imaging that collects extremely undersampled or
noisy measurements, obtaining an accurate image within a reasonable computing
time is challenging. Incorporating image mapping convolutional neural networks
(CNN) into iterative image recovery has great potential to resolve this issue.
This paper 1) incorporates image mapping CNN using identical convolutional
kernels in both encoders and decoders into a block coordinate descent (BCD)
signal recovery method and 2) applies alternating direction method of
multipliers to train the aforementioned image mapping CNN. We refer to the
proposed recurrent network as BCD-Net using identical encoding-decoding CNN
structures. Numerical experiments show that, for a) denoising low
signal-to-noise-ratio images and b) extremely undersampled magnetic resonance
imaging, the proposed BCD-Net achieves significantly more accurate image
recovery, compared to BCD-Net using distinct encoding-decoding structures
and/or the conventional image recovery model using both wavelets and total
variation.Comment: 5 pages, 3 figure
- …