1st Place Solution to MultiEarth 2023 Challenge on Multimodal SAR-to-EO Image Translation

The Multimodal Learning for Earth and Environment Workshop (MultiEarth 2023) aims to harness the substantial amount of remote sensing data gathered over extensive periods for the monitoring and analysis of Earth's ecosystems'health. The subtask, Multimodal SAR-to-EO Image Translation, involves the use of robust SAR data, even under adverse weather and lighting conditions, transforming it into high-quality, clear, and visually appealing EO data. In the context of the SAR2EO task, the presence of clouds or obstructions in EO data can potentially pose a challenge. To address this issue, we propose the Clean Collector Algorithm (CCA), designed to take full advantage of this cloudless SAR data and eliminate factors that may hinder the data learning process. Subsequently, we applied pix2pixHD for the SAR-to-EO translation and Restormer for image enhancement. In the final evaluation, the team 'CDRL' achieved an MAE of 0.07313, securing the top rank on the leaderboard

Class-Wise Adaptive Strategy for Semi Supervised Semantic Segmentation

Semi-supervised semantic segmentation learns a model for classifying pixels into specific classes using a few labeled samples and numerous unlabeled images. The recent leading approach is consistency regularization by self-training with pseudo-labeling pixels having high confidences for unlabeled images. However, using only high-confidence pixels for self-training may result in losing much of the information in the unlabeled sets due to poor confidence calibration of modern deep learning networks. In this paper, we propose a class-wise adaptive strategy for semi supervised semantic segmentation (CASS) to cope with the loss of most information that occurs in existing high-confidence-based pseudo-labeling methods. Unlike existing semi-supervised semantic segmentation frameworks, CASS constructs a validation set on a labeled set, to leverage the calibration performance for each class. On this basis, we propose class-wise adaptive thresholds and class-wise adaptive over-sampling using the analysis results from the validation set. Our proposed CASS achieves state-of-the-art performance on the full data partition of the base PASCAL VOC 2012 dataset and on the 1/4 data partition of the Cityscapes dataset with significant margins of 83.0 and 80.4 mIoU, respectively