3,502 research outputs found
Learning Efficient Convolutional Networks through Irregular Convolutional Kernels
As deep neural networks are increasingly used in applications suited for
low-power devices, a fundamental dilemma becomes apparent: the trend is to grow
models to absorb increasing data that gives rise to memory intensive; however
low-power devices are designed with very limited memory that can not store
large models. Parameters pruning is critical for deep model deployment on
low-power devices. Existing efforts mainly focus on designing highly efficient
structures or pruning redundant connections for networks. They are usually
sensitive to the tasks or relay on dedicated and expensive hashing storage
strategies. In this work, we introduce a novel approach for achieving a
lightweight model from the views of reconstructing the structure of
convolutional kernels and efficient storage. Our approach transforms a
traditional square convolution kernel to line segments, and automatically learn
a proper strategy for equipping these line segments to model diverse features.
The experimental results indicate that our approach can massively reduce the
number of parameters (pruned 69% on DenseNet-40) and calculations (pruned 59%
on DenseNet-40) while maintaining acceptable performance (only lose less than
2% accuracy)
- …