32,128 research outputs found
DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks
The field of video compression has developed some of the most sophisticated
and efficient compression algorithms known in the literature, enabling very
high compressibility for little loss of information. Whilst some of these
techniques are domain specific, many of their underlying principles are
universal in that they can be adapted and applied for compressing different
types of data. In this work we present DeepCABAC, a compression algorithm for
deep neural networks that is based on one of the state-of-the-art video coding
techniques. Concretely, it applies a Context-based Adaptive Binary Arithmetic
Coder (CABAC) to the network's parameters, which was originally designed for
the H.264/AVC video coding standard and became the state-of-the-art for
lossless compression. Moreover, DeepCABAC employs a novel quantization scheme
that minimizes the rate-distortion function while simultaneously taking the
impact of quantization onto the accuracy of the network into account.
Experimental results show that DeepCABAC consistently attains higher
compression rates than previously proposed coding techniques for neural network
compression. For instance, it is able to compress the VGG16 ImageNet model by
x63.6 with no loss of accuracy, thus being able to represent the entire network
with merely 8.7MB. The source code for encoding and decoding can be found at
https://github.com/fraunhoferhhi/DeepCABAC
Domain Adaptive Neural Networks for Object Recognition
We propose a simple neural network model to deal with the domain adaptation
problem in object recognition. Our model incorporates the Maximum Mean
Discrepancy (MMD) measure as a regularization in the supervised learning to
reduce the distribution mismatch between the source and target domains in the
latent space. From experiments, we demonstrate that the MMD regularization is
an effective tool to provide good domain adaptation models on both SURF
features and raw image pixels of a particular image data set. We also show that
our proposed model, preceded by the denoising auto-encoder pretraining,
achieves better performance than recent benchmark models on the same data sets.
This work represents the first study of MMD measure in the context of neural
networks
Recent advances in coding theory for near error-free communications
Channel and source coding theories are discussed. The following subject areas are covered: large constraint length convolutional codes (the Galileo code); decoder design (the big Viterbi decoder); Voyager's and Galileo's data compression scheme; current research in data compression for images; neural networks for soft decoding; neural networks for source decoding; finite-state codes; and fractals for data compression
- …