Foreground segmentation in atmospheric turbulence degraded video sequences to aid in background stabilization

Abstract: Video sequences captured over a long range through the turbulent atmosphere contain some degree of atmospheric turbulence degradation (ATD). Stabilization of the geometric distortions present in video sequences containing ATD and containing objects undergoing real motion is a challenging task. This is due to the difficulty of discriminating what visible motion is real motion and what is caused by ATD warping. Due to this, most stabilization techniques applied to ATD sequences distort real motion in the sequence. In this study we propose a new method to classify foreground regions in ATD video sequences. This classification is used to stabilize the background of the scene while preserving objects undergoing real motion by compositing them back into the sequence. A hand annotated dataset of three ATD sequences is produced with which the performance of this approach can be quantitatively measured and compared against the current state-of-the-art

AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models

Although many long-range imaging systems are designed to support extended vision applications, a natural obstacle to their operation is degradation due to atmospheric turbulence. Atmospheric turbulence causes significant degradation to image quality by introducing blur and geometric distortion. In recent years, various deep learning-based single image atmospheric turbulence mitigation methods, including CNN-based and GAN inversion-based, have been proposed in the literature which attempt to remove the distortion in the image. However, some of these methods are difficult to train and often fail to reconstruct facial features and produce unrealistic results especially in the case of high turbulence. Denoising Diffusion Probabilistic Models (DDPMs) have recently gained some traction because of their stable training process and their ability to generate high quality images. In this paper, we propose the first DDPM-based solution for the problem of atmospheric turbulence mitigation. We also propose a fast sampling technique for reducing the inference times for conditional DDPMs. Extensive experiments are conducted on synthetic and real-world data to show the significance of our model. To facilitate further research, all codes and pretrained models are publically available at http://github.com/Nithin-GK/AT-DDPMComment: Accepted to IEEE WACV 202

Blind Deconvolution of Anisoplanatic Images Collected by a Partially Coherent Imaging System

Coherent imaging systems offer unique benefits to system operators in terms of resolving power, range gating, selective illumination and utility for applications where passively illuminated targets have limited emissivity or reflectivity. This research proposes a novel blind deconvolution algorithm that is based on a maximum a posteriori Bayesian estimator constructed upon a physically based statistical model for the intensity of the partially coherent light at the imaging detector. The estimator is initially constructed using a shift-invariant system model, and is later extended to the case of a shift-variant optical system by the addition of a transfer function term that quantifies optical blur for wide fields-of-view and atmospheric conditions. The estimators are evaluated using both synthetically generated imagery, as well as experimentally collected image data from an outdoor optical range. The research is extended to consider the effects of weighted frame averaging for the individual short-exposure frames collected by the imaging system. It was found that binary weighting of ensemble frames significantly increases spatial resolution

Image enhancement techniques applied to solar feature detection

This dissertation presents the development of automatic image enhancement techniques for solar feature detection. The new method allows for detection and tracking of the evolution of filaments in solar images. Series of H-alpha full-disk images are taken in regular time intervals to observe the changes of the solar disk features. In each picture, the solar chromosphere filaments are identified for further evolution examination. The initial preprocessing step involves local thresholding to convert grayscale images into black-and-white pictures with chromosphere granularity enhanced. An alternative preprocessing method, based on image normalization and global thresholding is presented. The next step employs morphological closing operations with multi-directional linear structuring elements to extract elongated shapes in the image. After logical union of directional filtering results, the remaining noise is removed from the final outcome using morphological dilation and erosion with a circular structuring element. Experimental results show that the developed techniques can achieve excellent results in detecting large filaments and good detection rates for small filaments. The final chapter discusses proposed directions of the future research and applications to other areas of solar image processing, in particular to detection of solar flares, plages and sunspots

Image Restoration

This book represents a sample of recent contributions of researchers all around the world in the field of image restoration. The book consists of 15 chapters organized in three main sections (Theory, Applications, Interdisciplinarity). Topics cover some different aspects of the theory of image restoration, but this book is also an occasion to highlight some new topics of research related to the emergence of some original imaging devices. From this arise some real challenging problems related to image reconstruction/restoration that open the way to some new fundamental scientific questions closely related with the world we interact with

Restoration of Images Taken Through a Turbulent Medium

This thesis investigates the problem of how information contained in multiple, short exposure images of the same scene taken through (and distorted by) a turbulent medium (turbulent atmosphere or moving water surface) may be extracted and combined to produce a single image with superior quality and higher resolution. This problem is generally termed image restoration. It has many applications in fields as diverse as remote sensing, military intelligence, surveillance and recognition at a long distance, and other imaging problems which suffer from turbulent media, including e.g. the atmosphere and moving water surface. Wide-area/near-to-ground imaging (through atmosphere) and water imaging are the two main focuses of this thesis. The central technique used to solve these problems is speckle imaging, which is used to process a large number of images of the object with short exposure times such that the turbulent effect is frozen in each frame. A robust and efficient method using the bispectrum is developed to recover an almost diffraction-limited sharp image using the information contained in the captured short exposure images. Both the accuracy and the potential of these new algorithms have been investigated. Motivated by the lucky imaging technique which was used to select superior frames for astronomical imaging application, a new and more efficient technique is proposed. This technique is called lucky region, and it is aimed at selecting image regions with high quality as opposed to selecting a whole image as a lucky image. A new algorithm using bicoherence is proposed for lucky region selection. Its performance, as well as practical factors that may affect the performance, are investigated both theoretically and empirically. To further improve the quality of the recovered clean image after the speckle bispectrum processing, we also investigate blind deconvolution. One of the original contributions is to use natural image sparsity as a prior knowledge for the turbulence image restoration problem. A new algorithm is proposed and its performance is validated experimentally. The new methods are extended to the case of water imaging: restoration of images distorted by moving water waves. It is shown that this problem can be effectively solved by techniques developed in this thesis. Possible practical applications include various forms of ocean observation

Sensory information processing (1 July 1976 - 31 March 1977)

technical reportThe student of human visual perception is often overwhelmed by the vast amount of data that has been accumulated from experiments performed within the last century or so. It is often difficult to understand why a certain experiment has been performed.Results from similar experiments sometimes seem to conflict. Further confusion results when the student encounters raging controversies, the resolution of which would seem to minimally advance our knowledge of how we see. The reason for all this trouble stems from the fact that s suitable superstructure providing organization and support of this accumulation of data does not exist, i.e. an adequate theory of perception is not in hand

REMOTE SENSING DATA ANALYSIS FOR ENVIRONMENTAL AND HUMANITARIAN PURPOSES. The automation of information extraction from free satellite data.

This work is aimed at investigating technical possibilities to provide information on environmental parameters that can be used for risk management. The World food Program (WFP) is the United Nations Agency which is involved in risk management for fighting hunger in least-developed and low-income countries, where victims of natural and manmade disasters, refugees, displaced people and the hungry poor suffer from severe food shortages. Risk management includes three different phases (pre-disaster, response and post disaster) to be managed through different activities and actions. Pre disaster activities are meant to develop and deliver risk assessment, establish prevention actions and prepare the operative structures for managing an eventual emergency or disaster. In response and post disaster phase actions planned in the pre-disaster phase are executed focusing on saving lives and secondly, on social economic recovery. In order to optimally manage its operations in the response and post disaster phases, WFP needs to know, in order to estimate the impact an event will have on future food security as soon as possible, the areas affected by the natural disaster, the number of affected people, and the effects that the event can cause to vegetation. For this, providing easy-to-consult thematic maps about the affected areas and population, with adequate spatial resolution, time frequency and regular updating can result determining. Satellite remote sensed data have increasingly been used in the last decades in order to provide updated information about land surface with an acceptable time frequency. Furthermore, satellite images can be managed by automatic procedures in order to extract synthetic information about the ground condition in a very short time and can be easily shared in the web. The work of thesis, focused on the analysis and processing of satellite data, was carried out in cooperation with the association ITHACA (Information Technology for Humanitarian Assistance, Cooperation and Action), a center of research which works in cooperation with the WFP in order to provide IT products and tools for the management of food emergencies caused by natural disasters. These products should be able to facilitate the forecasting of the effects of catastrophic events, the estimation of the extension and location of the areas hit by the event, of the affected population and thereby the planning of interventions on the area that could be affected by food insecurity. The requested features of the instruments are: • Regular updating • Spatial resolution suitable for a synoptic analysis • Low cost • Easy consultation Ithaca is developing different activities to provide georeferenced thematic data to WFP users, such a spatial data infrastructure for storing, querying and manipulating large amounts of global geographic information, and for sharing it between a large and differentiated community; a system of early warning for floods, a drought monitoring tool, procedures for rapid mapping in the response phase in a case of natural disaster, web GIS tools to distribute and share georeferenced information, that can be consulted only by means of a web browser. The work of thesis is aimed at providing applications for the automatic production of base georeferenced thematic data, by using free global satellite data, which have characteristics suitable for analysis at a regional scale. In particular the main themes of the applications are water bodies and vegetation phenology. The first application aims at providing procedures for the automatic extraction of water bodies and will lead to the creation and update of an historical archive, which can be analyzed in order to catch the seasonality of water bodies and delineate scenarios of historical flooded areas. The automatic extraction of phenological parameters from satellite data will allow to integrate the existing drought monitoring system with information on vegetation seasonality and to provide further information for the evaluation of food insecurity in the post disaster phase. In the thesis are described the activities carried on for the development of procedures for the automatic processing of free satellite data in order to produce customized layers according to the exigencies in format and distribution of the final users. The main activities, which focused on the development of an automated procedure for the extraction of flooded areas, include the research of an algorithm for the classification of water bodies from satellite data, an important theme in the field of management of the emergencies due to flood events. Two main technologies are generally used: active sensors (radar) and passive sensors (optical data). Advantages for active sensors include the ability to obtain measurements anytime, regardless of the time of day or season, while passive sensors can only be used in the daytime cloud free conditions. Even if with radar technologies is possible to get information on the ground in all weather conditions, it is not possible to use radar data to obtain a continuous archive of flooded areas, because of the lack of a predetermined frequency in the acquisition of the images. For this reason the choice of the dataset went in favor of MODIS (Moderate Resolution Imaging Spectroradiometer), optical data with a daily frequency, a spatial resolution of 250 meters and an historical archive of 10 years. The presence of cloud coverage prevents from the acquisition of the earth surface, and the shadows due to clouds can be wrongly classified as water bodies because of the spectral response very similar to the one of water. After an analysis of the state of the art of the algorithms of automated classification of water bodies in images derived from optical sensors, the author developed an algorithm that allows to classify the data of reflectivity and to temporally composite them in order to obtain flooded areas scenarios for each event. This procedure was tested in the Bangladesh areas, providing encouraging classification accuracies. For the vegetation theme, the main activities performed, here described, include the review of the existing methodologies for phenological studies and the automation of the data flow between inputs and outputs with the use of different global free satellite datasets. In literature, many studies demonstrated the utility of the NDVI (Normalized Difference Vegetation Index) indices for the monitoring of vegetation dynamics, in the study of cultivations, and for the survey of the vegetation water stress. The author developed a procedure for creating layers of phenological parameters which integrates the TIMESAT software, produced by Lars Eklundh and Per Jönsson, for processing NDVI indices derived from different satellite sensors: MODIS (Moderate Resolution Imaging Spectroradiometer), AVHRR (Advanced Very High Resolution Radiometer) AND SPOT (Système Pour l'Observation de la Terre) VEGETATION. The automated procedure starts from data downloading, calls in a batch mode the software and provides customized layers of phenological parameters such as the starting of the season or length of the season and many others