Trans-Inpainter: Wireless Channel Information- Guided Image Restoration via Multimodal Transformer

Abstract

Image inpainting is a critical computer vision task to restore missing or damaged image regions. In this paper, we propose Trans-Inpainter, a novel multimodal image inpainting method guided by Channel State Information (CSI) data. Leveraging the power of transformer architectures, Trans-Inpainter effectively extracts visual information from CSI time sequences, enabling high-quality and realistic image inpainting. To evaluate its performance, we compare Trans-Inpainter with RF-Inpainter, the state-of-the-art radio frequency (RF) signal-based image inpainting technique. Through comprehensive experiments, Trans-Inpainter consistently demonstrates superior performance in various scenarios. Additionally, we investigate the impact of CSI data variations on Trans-Inpainter's imaging ability, analyzing individual sensor data, fused data from multiple sensors, and altered CSI matrix dimensions. These insights provide valuable references for future wireless sensing and computer vision studies