Image Diversification via Deep Learning based Generative Models

Machine learning driven pattern recognition from imagery such as object detection has been prevalenting among society due to the high demand for autonomy and the recent remarkable advances in such technology. The machine learning technologies acquire the abstraction of the existing data and enable inference of the pattern of the future inputs. However, such technologies require a sheer amount of images as a training dataset which well covers the distribution of the future inputs in order to predict the proper patterns whereas it is impracticable to prepare enough variety of images in many cases. To address this problem, this thesis pursues to discover the method to diversify image datasets for fully enabling the capability of machine learning driven applications. Focusing on the plausible image synthesis ability of generative models, we investigate a number of approaches to expand the variety of the output images using image-to-image translation, mixup and diffusion models along with the technique to enable a computation and training dataset efficient diffusion approach. First, we propose the combined use of unpaired image-to-image translation and mixup for data augmentation on limited non-visible imagery. Second, we propose diffusion image-to-image translation that generates greater quality images than other previous adversarial training based translation methods. Third, we propose a patch-wise and discrete conditional training of diffusion method enabling the reduction of the computation and the robustness on small training datasets. Subsequently, we discuss a remaining open challenge about evaluation and the direction of future work. Lastly, we make an overall conclusion after stating social impact of this research field

Statistical and Machine Learning Models for Remote Sensing Data Mining - Recent Advancements

This book is a reprint of the Special Issue entitled "Statistical and Machine Learning Models for Remote Sensing Data Mining - Recent Advancements" that was published in Remote Sensing, MDPI. It provides insights into both core technical challenges and some selected critical applications of satellite remote sensing image analytics

The InflateSAR Campaign: Testing SAR Vessel Detection Systems for Refugee Rubber Inflatables

Countless numbers of people lost their lives at Europe’s southern borders in recent years in the attempt to cross to Europe in small rubber inflatables. This work examines satellite-based approaches to build up future systems that can automatically detect those boats. We compare the performance of several automatic vessel detectors using real synthetic aperture radar (SAR) data from X-band and C-band sensors on TerraSAR-X and Sentinel-1. The data was collected in an experimental campaign where an empty boat lies on a lake’s surface to analyse the influence of main sensor parameters (incidence angle, polarization mode, spatial resolution) on the detectability of our inflatable. All detectors are implemented with a moving window and use local clutter statistics from the adjacent water surface. Among tested detectors are well-known intensity-based (CA-CFAR), sublook-based (sublook correlation) and polarimetric-based (PWF, PMF, PNF, entropy, symmetry and iDPolRAD) approaches. Additionally, we introduced a new version of the volume detecting iDPolRAD aimed at detecting surface anomalies and compare two approaches to combine the volume and the surface in one algorithm, producing two new highly performing detectors. The results are compared with receiver operating characteristic (ROC) curves, enabling us to compare detectors independently of threshold selection

Robust CFAR Detector Based on Truncated Statistics for Polarimetric Synthetic Aperture Radar

Constant false alarm rate (CFAR) algorithms using a local training window are widely used for ship detection with synthetic aperture radar (SAR) imagery. However, when the density of the targets is high, such as in busy shipping lines and crowded harbors, the background statistics may be contaminated by the presence of nearby targets in the training window. Recently, a robust CFAR detector based on truncated statistics (TS) was proposed. However, the truncation of data in the format of polarimetric covariance matrices is much more complicated with respect to the truncation of intensity (single polarization) data. In this article, a polarimetric whitening filter TS CFAR (PWF-TS-CFAR) is proposed to estimate the background parameters accurately in the contaminated sea clutter for PolSAR imagery. The CFAR detector uses a polarimetric whitening filter (PWF) to turn the multidimensional problem to a 1-D case. It uses truncation to exclude possible statistically interfering outliers and uses TS to model the remaining background samples. The algorithm does not require prior knowledge of the interfering targets, and it is performed iteratively and adaptively to derive better estimates of the polarimetric covariance matrix (although this is computationally expensive). The PWF-TS-CFAR detector provides accurate background clutter modeling, a stable false alarm property, and improves the detection performance in high-target-density situations. RadarSat2 data are used to verify our derivations, and the results are in line with the theory

Scalable computing for earth observation - Application on Sea Ice analysis

In recent years, Deep learning (DL) networks have shown considerable improvements and have become a preferred methodology in many different applications. These networks have outperformed other classical techniques, particularly in large data settings. In earth observation from the satellite field, for example, DL algorithms have demonstrated the ability to learn complicated nonlinear relationships in input data accurately. Thus, it contributed to advancement in this field. However, the training process of these networks has heavy computational overheads. The reason is two-fold: The sizable complexity of these networks and the high number of training samples needed to learn all parameters comprising these architectures. Although the quantity of training data enhances the accuracy of the trained models in general, the computational cost may restrict the amount of analysis that can be done. This issue is particularly critical in satellite remote sensing, where a myriad of satellites generate an enormous amount of data daily, and acquiring in-situ ground truth for building a large training dataset is a fundamental prerequisite. This dissertation considers various aspects of deep learning based sea ice monitoring from SAR data. In this application, labeling data is very costly and time-consuming. Also, in some cases, it is not even achievable due to challenges in establishing the required domain knowledge, specifically when it comes to monitoring Arctic Sea ice with Synthetic Aperture Radar (SAR), which is the application domain of this thesis. Because the Arctic is remote, has long dark seasons, and has a very dynamic weather system, the collection of reliable in-situ data is very demanding. In addition to the challenges of interpreting SAR data of sea ice, this issue makes SAR-based sea ice analysis with DL networks a complicated process. We propose novel DL methods to cope with the problems of scarce training data and address the computational cost of the training process. We analyze DL network capabilities based on self-designed architectures and learn strategies, such as transfer learning for sea ice classification. We also address the scarcity of training data by proposing a novel deep semi-supervised learning method based on SAR data for incorporating unlabeled data information into the training process. Finally, a new distributed DL method that can be used in a semi-supervised manner is proposed to address the computational complexity of deep neural network training

A unified model for context-based behavioural modelling and classification

A unified Bayesian model that simultaneously performs behavioural modelling, information fusion and classification is presented. The model is expressed in the form of a dynamic Bayesian network (DBN). Behavioural modelling is performed by tracking the continuous dynamics of a entity and incorporating various contextual elements that influence behaviour. The entity is classified according to its behaviour. Classification is expressed as a conditional probability of the entity class given its tracked trajectory and the contextual elements. Inference in the DBN is performed using a derived Gaussian sum filter. The model is applied to classify vessels, according to their behaviour, in a maritime piracy situation. The novel aspects of this work include the unified approach to behaviour modelling and classification, the way in which contextual information is fused, the unique approach to classification according to behaviour and the associated derived Gaussian sum filter inference algorithm.South African National Research Foundation (NRF) and the the Advanced Sensors and Electronics Defence (ASED) Centre of KACST through the Council for Scientific and Industrial Research (CSIR).http://www.elsevier.com/locate/eswa2016-11-30hb201

Frugal Satellite Image Change Detection with Deep-Net Inversion

Change detection in satellite imagery seeks to find occurrences of targeted changes in a given scene taken at different instants. This task has several applications ranging from land-cover mapping, to anthropogenic activity monitory as well as climate change and natural hazard damage assessment. However, change detection is highly challenging due to the acquisition conditions and also to the subjectivity of changes. In this paper, we devise a novel algorithm for change detection based on active learning. The proposed method is based on a question and answer model that probes an oracle (user) about the relevance of changes only on a small set of critical images (referred to as virtual exemplars), and according to oracle's responses updates deep neural network (DNN) classifiers. The main contribution resides in a novel adversarial model that allows learning the most representative, diverse and uncertain virtual exemplars (as inverted preimages of the trained DNNs) that challenge (the most) the trained DNNs, and this leads to a better re-estimate of these networks in the subsequent iterations of active learning. Experiments show the out-performance of our proposed deep-net inversion against the related work.Comment: arXiv admin note: text overlap with arXiv:2212.1397