Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model

Image restoration algorithms for atmospheric turbulence are known to be much more challenging to design than traditional ones such as blur or noise because the distortion caused by the turbulence is an entanglement of spatially varying blur, geometric distortion, and sensor noise. Existing CNN-based restoration methods built upon convolutional kernels with static weights are insufficient to handle the spatially dynamical atmospheric turbulence effect. To address this problem, in this paper, we propose a physics-inspired transformer model for imaging through atmospheric turbulence. The proposed network utilizes the power of transformer blocks to jointly extract a dynamical turbulence distortion map and restore a turbulence-free image. In addition, recognizing the lack of a comprehensive dataset, we collect and present two new real-world turbulence datasets that allow for evaluation with both classical objective metrics (e.g., PSNR and SSIM) and a new task-driven metric using text recognition accuracy. Both real testing sets and all related code will be made publicly available.Comment: This paper is accepted as a poster at ECCV 202

Airborne forward pointing UV Rayleigh lidar for remote clear air turbulence (CAT) detection: system design and performance

A high-performance airborne UV Rayleigh lidar system was developed within the European project DELICAT. With its forward-pointing architecture it aims at demonstrating a novel detection scheme for clear air turbulence (CAT) for an aeronautics safety application. Due to its occurrence in clear and clean air at high altitudes (aviation cruise flight level), this type of turbulence evades microwave radar techniques and in most cases coherent Doppler lidar techniques. The present lidar detection technique relies on air density fluctuations measurement and is thus independent of backscatter from hydrometeors and aerosol particles. The subtle air density fluctuations caused by the turbulent air flow demand exceptionally high stability of the setup and in particular of the detection system. This paper describes an airborne test system for the purpose of demonstrating this technology and turbulence detection method: a high-power UV Rayleigh lidar system is installed on a research aircraft in a forward-looking configuration for use in cruise flight altitudes. Flight test measurements demonstrate this unique lidar system being able to resolve air density fluctuations occurring in light-to-moderate CAT at 5 km or moderate CAT at 10 km distance. A scaling of the determined stability and noise characteristics shows that such performance is adequate for an application in commercial air transport.Comment: 17 pages, 19 figures. Pre-publish to Applied Optics (OSA

Physics-Driven Turbulence Image Restoration with Stochastic Refinement

Image distortion by atmospheric turbulence is a stochastic degradation, which is a critical problem in long-range optical imaging systems. A number of research has been conducted during the past decades, including model-based and emerging deep-learning solutions with the help of synthetic data. Although fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions recently, the training of such models only relies on the synthetic data and ground truth pairs. This paper proposes the Physics-integrated Restoration Network (PiRN) to bring the physics-based simulator directly into the training process to help the network to disentangle the stochasticity from the degradation and the underlying image. Furthermore, to overcome the ``average effect" introduced by deterministic models and the domain gap between the synthetic and real-world degradation, we further introduce PiRN with Stochastic Refinement (PiRN-SR) to boost its perceptual quality. Overall, our PiRN and PiRN-SR improve the generalization to real-world unknown turbulence conditions and provide a state-of-the-art restoration in both pixel-wise accuracy and perceptual quality. Our codes are available at \url{https://github.com/VITA-Group/PiRN}.Comment: Accepted by ICCV 202

Analysis of deep learning architectures for turbulence mitigation in long-range imagery

In long range imagery, the atmosphere along the line of sight can result in unwanted visual effects. Random variations in the refractive index of the air causes light to shift and distort. When captured by a camera, this randomly induced variation results in blurred and spatially distorted images. The removal of such effects is greatly desired. Many traditional methods are able to reduce the effects of turbulence within images, however they require complex optimisation procedures or have large computational complexity. The use of deep learning for image processing has now become commonplace, with neural networks being able to outperform traditional methods in many fields. This paper presents an evaluation of various deep learning architectures on the task of turbulence mitigation. The core disadvantage of deep learning is the dependence on a large quantity of relevant data. For the task of turbulence mitigation, real life data is difficult to obtain, as a clean undistorted image is not always obtainable. Turbulent images were therefore generated with the use of a turbulence simulator. This was able to accurately represent atmospheric conditions and apply the resulting spatial distortions onto clean images. This paper provides a comparison between current state of the art image reconstruction convolutional neural networks. Each network is trained on simulated turbulence data. They are then assessed on a series of test images. It is shown that the networks are unable to provide high quality output images. However, they are shown to be able to reduce the effects of spatial warping within the test images. This paper provides critical analysis into the effectiveness of the application of deep learning. It is shown that deep learning has potential in this field, and can be used to make further improvements in the future

Experimental characterization and mitigation of turbulence induced signal fades within an ad hoc FSO network

Optical beams propagating through the turbulent atmospheric channel suffer from both the attenuation and phase distortion. Since future wireless networks are envisaged to be deployed in the ad hoc mesh topology, this paper presents the experimental laboratory characterization of mitigation of turbulence induced signal fades for two ad hoc scenarios. Results from measurements of the thermal structure constant along the propagation channels, changes of the coherence lengths for different turbulence regimes and the eye diagrams for partially correlated turbulences in free space optical channels are discussed. Based on these results future deployment of optical ad hoc networks can be more straightforwardly planned

Challenges and Opportunities of Optical Wireless Communication Technologies

In this chapter, we present various opportunities of using optical wireless communication (OWC) technologies in each sector of optical communication networks. Moreover, challenges of optical wireless network implementations are investigated. We characterized the optical wireless communication channel through the channel measurements and present different models for the OWC link performance evaluations. In addition, we present some technologies for the OWC performance enhancement in order to address the last-mile transmission bottleneck of the system efficiently. The technologies can be of great help in alleviating the stringent requirement by the cloud radio access network (C-RAN) backhaul/fronthaul as well as in the evolution toward an efficient backhaul/fronthaul for the 5G network. Furthermore, we present a proof-of-concept experiment in order to demonstrate and evaluate high capacity/flexible coherent PON and OWC links for different network configurations in the terrestrial links. To achieve this, we employ advanced modulation format and digital signal processing (DSP) techniques in the offline and real-time mode of the operation. The proposed configuration has the capability to support different applications, services, and multiple operators over a shared optical fiber infrastructure

Foreground segmentation in atmospheric turbulence degraded video sequences to aid in background stabilization

Abstract: Video sequences captured over a long range through the turbulent atmosphere contain some degree of atmospheric turbulence degradation (ATD). Stabilization of the geometric distortions present in video sequences containing ATD and containing objects undergoing real motion is a challenging task. This is due to the difficulty of discriminating what visible motion is real motion and what is caused by ATD warping. Due to this, most stabilization techniques applied to ATD sequences distort real motion in the sequence. In this study we propose a new method to classify foreground regions in ATD video sequences. This classification is used to stabilize the background of the scene while preserving objects undergoing real motion by compositing them back into the sequence. A hand annotated dataset of three ATD sequences is produced with which the performance of this approach can be quantitatively measured and compared against the current state-of-the-art