6,179 research outputs found
From Hashing to CNNs: Training BinaryWeight Networks via Hashing
Deep convolutional neural networks (CNNs) have shown appealing performance on
various computer vision tasks in recent years. This motivates people to deploy
CNNs to realworld applications. However, most of state-of-art CNNs require
large memory and computational resources, which hinders the deployment on
mobile devices. Recent studies show that low-bit weight representation can
reduce much storage and memory demand, and also can achieve efficient network
inference. To achieve this goal, we propose a novel approach named BWNH to
train Binary Weight Networks via Hashing. In this paper, we first reveal the
strong connection between inner-product preserving hashing and binary weight
networks, and show that training binary weight networks can be intrinsically
regarded as a hashing problem. Based on this perspective, we propose an
alternating optimization method to learn the hash codes instead of directly
learning binary weights. Extensive experiments on CIFAR10, CIFAR100 and
ImageNet demonstrate that our proposed BWNH outperforms current state-of-art by
a large margin
Training a Binary Weight Object Detector by Knowledge Transfer for Autonomous Driving
Autonomous driving has harsh requirements of small model size and energy
efficiency, in order to enable the embedded system to achieve real-time
on-board object detection. Recent deep convolutional neural network based
object detectors have achieved state-of-the-art accuracy. However, such models
are trained with numerous parameters and their high computational costs and
large storage prohibit the deployment to memory and computation resource
limited systems. Low-precision neural networks are popular techniques for
reducing the computation requirements and memory footprint. Among them, binary
weight neural network (BWN) is the extreme case which quantizes the float-point
into just bit. BWNs are difficult to train and suffer from accuracy
deprecation due to the extreme low-bit representation. To address this problem,
we propose a knowledge transfer (KT) method to aid the training of BWN using a
full-precision teacher network. We built DarkNet- and MobileNet-based binary
weight YOLO-v2 detectors and conduct experiments on KITTI benchmark for car,
pedestrian and cyclist detection. The experimental results show that the
proposed method maintains high detection accuracy while reducing the model size
of DarkNet-YOLO from 257 MB to 8.8 MB and MobileNet-YOLO from 193 MB to 7.9 MB.Comment: Accepted by ICRA 201
Towards Effective Low-bitwidth Convolutional Neural Networks
This paper tackles the problem of training a deep convolutional neural
network with both low-precision weights and low-bitwidth activations.
Optimizing a low-precision network is very challenging since the training
process can easily get trapped in a poor local minima, which results in
substantial accuracy loss. To mitigate this problem, we propose three
simple-yet-effective approaches to improve the network training. First, we
propose to use a two-stage optimization strategy to progressively find good
local minima. Specifically, we propose to first optimize a net with quantized
weights and then quantized activations. This is in contrast to the traditional
methods which optimize them simultaneously. Second, following a similar spirit
of the first method, we propose another progressive optimization approach which
progressively decreases the bit-width from high-precision to low-precision
during the course of training. Third, we adopt a novel learning scheme to
jointly train a full-precision model alongside the low-precision one. By doing
so, the full-precision model provides hints to guide the low-precision model
training. Extensive experiments on various datasets ( i.e., CIFAR-100 and
ImageNet) show the effectiveness of the proposed methods. To highlight, using
our methods to train a 4-bit precision network leads to no performance decrease
in comparison with its full-precision counterpart with standard network
architectures ( i.e., AlexNet and ResNet-50).Comment: 11 page
- …