1,500 research outputs found
Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation
The advancement of deep convolutional neural networks (DCNNs) has driven
significant improvement in the accuracy of recognition systems for many
computer vision tasks. However, their practical applications are often
restricted in resource-constrained environments. In this paper, we introduce
projection convolutional neural networks (PCNNs) with a discrete back
propagation via projection (DBPP) to improve the performance of binarized
neural networks (BNNs). The contributions of our paper include: 1) for the
first time, the projection function is exploited to efficiently solve the
discrete back propagation problem, which leads to a new highly compressed CNNs
(termed PCNNs); 2) by exploiting multiple projections, we learn a set of
diverse quantized kernels that compress the full-precision kernels in a more
efficient way than those proposed previously; 3) PCNNs achieve the best
classification performance compared to other state-of-the-art BNNs on the
ImageNet and CIFAR datasets
Bayesian Optimized 1-Bit CNNs
Deep convolutional neural networks (DCNNs) have dominated the recent
developments in computer vision through making various record-breaking models.
However, it is still a great challenge to achieve powerful DCNNs in
resource-limited environments, such as on embedded devices and smart phones.
Researchers have realized that 1-bit CNNs can be one feasible solution to
resolve the issue; however, they are baffled by the inferior performance
compared to the full-precision DCNNs. In this paper, we propose a novel
approach, called Bayesian optimized 1-bit CNNs (denoted as BONNs), taking the
advantage of Bayesian learning, a well-established strategy for hard problems,
to significantly improve the performance of extreme 1-bit CNNs. We incorporate
the prior distributions of full-precision kernels and features into the
Bayesian framework to construct 1-bit CNNs in an end-to-end manner, which have
not been considered in any previous related methods. The Bayesian losses are
achieved with a theoretical support to optimize the network simultaneously in
both continuous and discrete spaces, aggregating different losses jointly to
improve the model capacity. Extensive experiments on the ImageNet and CIFAR
datasets show that BONNs achieve the best classification performance compared
to state-of-the-art 1-bit CNNs
- …