17,502 research outputs found
Network Sketching: Exploiting Binary Structure in Deep CNNs
Convolutional neural networks (CNNs) with deep architectures have
substantially advanced the state-of-the-art in computer vision tasks. However,
deep networks are typically resource-intensive and thus difficult to be
deployed on mobile devices. Recently, CNNs with binary weights have shown
compelling efficiency to the community, whereas the accuracy of such models is
usually unsatisfactory in practice. In this paper, we introduce network
sketching as a novel technique of pursuing binary-weight CNNs, targeting at
more faithful inference and better trade-off for practical applications. Our
basic idea is to exploit binary structure directly in pre-trained filter banks
and produce binary-weight models via tensor expansion. The whole process can be
treated as a coarse-to-fine model approximation, akin to the pencil drawing
steps of outlining and shading. To further speedup the generated models, namely
the sketches, we also propose an associative implementation of binary tensor
convolutions. Experimental results demonstrate that a proper sketch of AlexNet
(or ResNet) outperforms the existing binary-weight models by large margins on
the ImageNet large scale classification task, while the committed memory for
network parameters only exceeds a little.Comment: To appear in CVPR201
Dilated Spatial Generative Adversarial Networks for Ergodic Image Generation
Generative models have recently received renewed attention as a result of
adversarial learning. Generative adversarial networks consist of samples
generation model and a discrimination model able to distinguish between genuine
and synthetic samples. In combination with convolutional (for the
discriminator) and de-convolutional (for the generator) layers, they are
particularly suitable for image generation, especially of natural scenes.
However, the presence of fully connected layers adds global dependencies in the
generated images. This may lead to high and global variations in the generated
sample for small local variations in the input noise. In this work we propose
to use architec-tures based on fully convolutional networks (including among
others dilated layers), architectures specifically designed to generate
globally ergodic images, that is images without global dependencies. Conducted
experiments reveal that these architectures are well suited for generating
natural textures such as geologic structures
- …