Previous work has shown that it is possible to train deep neural networks
with low precision weights and activations. In the extreme case it is even
possible to constrain the network to binary values. The costly floating point
multiplications are then reduced to fast logical operations. High end smart
phones such as Google's Pixel 2 and Apple's iPhone X are already equipped with
specialised hardware for image processing and it is very likely that other
future consumer hardware will also have dedicated accelerators for deep neural
networks. Binary neural networks are attractive in this case because the
logical operations are very fast and efficient when implemented in hardware. We
propose a transfer learning based architecture where we first train a binary
network on Imagenet and then retrain part of the network for different tasks
while keeping most of the network fixed. The fixed binary part could be
implemented in a hardware accelerator while the last layers of the network are
evaluated in software. We show that a single binary neural network trained on
the Imagenet dataset can indeed be used as a feature extractor for other
datasets.Comment: Machine Learning on the Phone and other Consumer Devices, NIPS2017
Worksho
