Weather recognition is an essential support for many practical life
applications, including traffic safety, environment, and meteorology. However,
many existing related works cannot comprehensively describe weather conditions
due to their complex co-occurrence dependencies. This paper proposes a novel
multi-label weather recognition model considering these dependencies. The
proposed model called MASK-Convolutional Neural Network-Transformer (MASK-CT)
is based on the Transformer, the convolutional process, and the MASK mechanism.
The model employs multiple convolutional layers to extract features from
weather images and a Transformer encoder to calculate the probability of each
weather condition based on the extracted features. To improve the
generalization ability of MASK-CT, a MASK mechanism is used during the training
phase. The effect of the MASK mechanism is explored and discussed. The Mask
mechanism randomly withholds some information from one-pair training instances
(one image and its corresponding label). There are two types of MASK methods.
Specifically, MASK-I is designed and deployed on the image before feeding it
into the weather feature extractor and MASK-II is applied to the image label.
The Transformer encoder is then utilized on the randomly masked image features
and labels. The experimental results from various real-world weather
recognition datasets demonstrate that the proposed MASK-CT model outperforms
state-of-the-art methods. Furthermore, the high-speed dynamic real-time weather
recognition capability of the MASK-CT is evaluated.Comment: Under Revie