381,714 research outputs found
WaveNets: Wavelet Channel Attention Networks
Channel Attention reigns supreme as an effective technique in the field of
computer vision. However, the proposed channel attention by SENet suffers from
information loss in feature learning caused by the use of Global Average
Pooling (GAP) to represent channels as scalars. Thus, designing effective
channel attention mechanisms requires finding a solution to enhance features
preservation in modeling channel inter-dependencies. In this work, we utilize
Wavelet transform compression as a solution to the channel representation
problem. We first test wavelet transform as an Auto-Encoder model equipped with
conventional channel attention module. Next, we test wavelet transform as a
standalone channel compression method. We prove that global average pooling is
equivalent to the recursive approximate Haar wavelet transform. With this
proof, we generalize channel attention using Wavelet compression and name it
WaveNet. Implementation of our method can be embedded within existing channel
attention methods with a couple of lines of code. We test our proposed method
using ImageNet dataset for image classification task. Our method outperforms
the baseline SENet, and achieves the state-of-the-art results. Our code
implementation is publicly available at https://github.com/hady1011/WaveNet-C.Comment: IEEE BigData2022 conferenc
- …