Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models

Zero-shot referring image segmentation is a challenging task because it aims to find an instance segmentation mask based on the given referring descriptions, without training on this type of paired data. Current zero-shot methods mainly focus on using pre-trained discriminative models (e.g., CLIP). However, we have observed that generative models (e.g., Stable Diffusion) have potentially understood the relationships between various visual elements and text descriptions, which are rarely investigated in this task. In this work, we introduce a novel Referring Diffusional segmentor (Ref-Diff) for this task, which leverages the fine-grained multi-modal information from generative models. We demonstrate that without a proposal generator, a generative model alone can achieve comparable performance to existing SOTA weakly-supervised models. When we combine both generative and discriminative models, our Ref-Diff outperforms these competing methods by a significant margin. This indicates that generative models are also beneficial for this task and can complement discriminative models for better referring segmentation. Our code is publicly available at https://github.com/kodenii/Ref-Diff

Hierarchically Structured Matrix Recovery-Based Channel Estimation for RIS-Aided Communications

Reconfigurable intelligent surface (RIS) has emerged as a promising technology for improving capacity and extending coverage of wireless networks. In this work, we consider RIS-aided millimeter wave (mmWave) multiple-input and multiple-output (MIMO) communications, where acquiring accurate channel state information is challenging due to the high dimensionality of channels. To achieve efficient channel estimation, fully exploiting the structures of the channels is crucial. To this end, we formulate the channel estimation as a hierarchically structured matrix recovery problem, and design a low-complexity message passing algorithm to solve it, leveraging the unitary approximate message passing. Simulation results demonstrate the superiority of the proposed algorithm with performance close to the oracle bound

Water-Body Segmentation for SAR Images: Past, Current, and Future

Synthetic Aperture Radar (SAR), as a microwave sensor that can sense a target all day or night under all-weather conditions, is of great significance for detecting water resources, such as coastlines, lakes and rivers. This paper reviews literature published in the past 30 years in the field of water body extraction in SAR images, and makes some proposals that the community working with SAR image waterbody extraction should consider. Firstly, this review focuses on the main ideas and characteristics of traditional water body extraction on SAR images, mainly focusing on traditional Machine Learning (ML) methods. Secondly, how Deep Learning (DL) methods are applied and optimized in the task of water-body segmentation for SAR images is summarized from the two levels of pixel and image. We also pay more attention to the most popular networks, such as U-Net and its modified models, and novel networks, such as the Cascaded Fully-Convolutional Network (CFCN) and River-Net. In the end, an in-depth discussion is presented, along with conclusions and future trends, on the limitations and challenges of DL for water-body segmentation

Water-Body Segmentation for SAR Images: Past, Current, and Future

Synthetic Aperture Radar (SAR), as a microwave sensor that can sense a target all day or night under all-weather conditions, is of great significance for detecting water resources, such as coastlines, lakes and rivers. This paper reviews literature published in the past 30 years in the field of water body extraction in SAR images, and makes some proposals that the community working with SAR image waterbody extraction should consider. Firstly, this review focuses on the main ideas and characteristics of traditional water body extraction on SAR images, mainly focusing on traditional Machine Learning (ML) methods. Secondly, how Deep Learning (DL) methods are applied and optimized in the task of water-body segmentation for SAR images is summarized from the two levels of pixel and image. We also pay more attention to the most popular networks, such as U-Net and its modified models, and novel networks, such as the Cascaded Fully-Convolutional Network (CFCN) and River-Net. In the end, an in-depth discussion is presented, along with conclusions and future trends, on the limitations and challenges of DL for water-body segmentation

Raw Data-Based Motion Compensation for High-Resolution Sliding Spotlight Synthetic Aperture Radar

For accurate motion compensation (MOCO) in airborne synthetic aperture radar (SAR) imaging, a high-precision inertial navigation system (INS) is required. However, an INS is not always precise enough or is sometimes not even included in airborne SAR systems. In this paper, a new, raw, data-based range-invariant motion compensation approach, which can effectively extract the displacements in the line-of-sight (LOS) direction, is proposed for high-resolution sliding spotlight SAR mode. In this approach, the sub-aperture radial accelerations of the airborne platform are estimated via a well-developed weighted total least square (WTLS) method considering the time-varying beam direction. The effectiveness of the proposed approach is validated by two airborne sliding spotlight C band SAR raw datasets containing different types of terrain, with a high spatial resolution of about 0.15 m in azimuth

Classification of Land Cover in Complex Terrain Using Gaofen-3 SAR Ascending and Descending Orbit Data

Synthetic aperture radar (SAR) image is an effective remote sensing data source for geographic surveys. However, accurate land cover mapping based on SAR image in areas of complex terrain has become a challenge due to serious geometric distortions and the inadequate separation ability of dual-polarization data. To address these issues, a new land cover mapping framework which is suitable for complex terrain is proposed based on Gaofen-3 data of ascending and descending orbits. Firstly, the geometric distortion area is determined according to the local incident angle, based on analysis of the SAR imaging mechanism, and the correct polarization information of the opposite track is used to compensate for the geometric distortion area, including layovers and shadows. Then, the dual orbital polarization characteristics (DOPC) and dual polarization radar vegetation index (DpRVI) of dual-pol SAR data are extracted, and the optimal feature combination is found by means of Jeffries–Matusita (J-M) distance analysis. Finally, the deep learning method 2D convolutional neural network (2D-CNN) is applied to classify the compensated images. The proposed method was applied to a mountainous region of the Danjiangkou ecological protection area in China. The accuracy and reliability of the method were experimentally compared using the uncompensated images and the images without DpRVI. Quantitative evaluation revealed that the proposed method achieved better performance in complex terrain areas, with an overall accuracy (OA) score of 0.93, and a Kappa coefficient score of 0.92. Compared with the uncompensated image, OA increased by 5% and Kappa increased by 6%. Compared with the images without DpRVI, OA increased by 4% and Kappa increased by 5%. In summary, the results demonstrate the importance of ascending and descending orbit data to compensate geometric distortion and reveal the effectiveness of optimal feature combination including DpRVI. Its simple and effective polarization information compensation capability can broaden the promising application prospects of SAR images

Water-Body Type Classification in Dual PolSAR Imagery Using a Two-Step Deep-Learning Method

Water-body type problems classification plays a vital role in ecological conservation, water resource management, and urban planning. Accurate classification can aid decision-makers in understanding the functions of different water-body types, providing key information for urban planning and promoting harmony between human activities and the natural environment. Despite extensive research in the field of water-body segmentation, exploration in the water-body type classification community is not as widespread. Therefore, this article proposes a novel water-body type classification method based on a two-step deep-learning model, decomposing water-body type classification into water-body segmentation and water-body type identification. Especially, this method constructs a unique data strategy by organically integrating backscatter features, polarimetric features, and DEM features, providing the model with rich and comprehensive information. In the first step, the segmentation network uses the fused feature to extract all water-body from synthetic aperture radar images. Subsequently, the extracted water-body are combined with the input data, forming a multifeature input for the identification network to distinguish between natural and artificial water-body. During this process, a swarm intelligence optimization algorithm is employed to explore the optimal hyperparameters of the network, including those of the segmentation and identification networks. Finally, the proposed method is assessed using extensive experiments on water-body segmentation tasks, water-body type identification tasks, and joint water-body type classification tasks. This article not only provides a new perspective in the field of water-body type classification but also demonstrates the immense potential of deep-learning network hyperparameter optimization and feature fusion in solving such

Effect of exhaust thermal parameters on optimal circuit layouts and optimal thermoelectric generator structure used in internal combustion engine

To improve the thermoelectric generator (TEG) power conversion efficiency when recovering automobile exhaust waste heat, it is important to design an optimal structure and circuit layouts on TEG responding to actual changed exhaust parameters. However, full-series-current thermoelectric models were commonly used without considering on other circuit layouts in previous studies. In order to explore the effect of circuit layout on thermoelectric generator’s performance, this research introduced two kinds of thermoelectric models with full series-current mode and multi-series current mode taken. By taking finite element analysis, it is mainly focused on revealing the TEG power output performance and the optimal structure scales under different circuit layouts. Besides, the effect of exhaust thermal parameters were considered in the optimization process. Finally, by comparing the effects of structure scales, circuit layout mode, exhaust temperature, and mass flow rate on the net power, the maximal power conversion was achievable which was helpful to complete a more comprehensively optimal design on thermoelectric generator. It is found that when the multi-series current stage increases from 2 to 5, the net power can obviously increase from 13% to 27% compared to a full-series current layout. As the number of stage increases, the power increase becomes less and less obvious. Whatever the exhaust parameters, the same optimal height scale and optimal width scale of the series current mode can be used in the multi-series current mode. The optimal length of a multi-stage should be designed according to exhaust parameters. It is strongly recommended to use multi-stage series current mode for its obvious improvement on power for large-scale TEG systems