Cloud Image Retrieval for Sea Fog Recognition (CIR-SFR) Using Double Branch Residual Neural Network

Sea fog is a common weather phenomenon at sea, which reduces visibility and causes tremendous hazards to marine transportation, marine fishing, and other maritime operations. Traditional sea fog monitoring methods have enormous difficulties in characterizing the diversity of sea fog and distinguishing sea fog from low-level clouds. Thus, we propose a cloud image retrieval method for sea fog recognition (CIR-SFR) in a deep learning (DL) framework by combining the advantages of metric learning. CIR-SFR includes the feature extraction module and the retrieval-based SFR module. The feature extraction module adopts the double branch residual neural network (DBRNN) to comprehensively extract the global and local features of cloud images. By introducing local branches and using activation masks, DBRNN can focus on regions of interest in cloud images. Moreover, cloud image features are projected into the semantic space by introducing multisimilarity loss, which effectively improves the discrimination ability of sea fog and low-level clouds. For the retrieval-based SFR module, similar cloud images are retrieved from the cloud image dataset according to the distance in the feature space, and accurate SFR results are obtained by counting the percentage of various cloud image types in the retrieval results. To evaluate the SFR system, we establish a dataset of 2544 cloud images including clear sky, low-level cloud, medium high cloud, and sea fog. Experimental results show that the proposed method outperforms the traditional methods in SFR, which provides a new way for SFR