Machine Learning for Nonlinear Inverse Problems

There have been multiple mathematical models presented by scientists that allow cor- roborating the behavior of complex systems. However, these models estimate values that can be measured but are unable to determine the parameters that caused such behaviors. Inverse Problems aim at finding the parameters of a model, given by systems of equa- tions, from noisy observations/measurements. These are typically ill-posed problems that may have no exact solutions, multiple solutions or unstable solutions. In this the- sis, we will be restringing our work to nonlinear inverse problems that have an exact single solution but due to their complexity can not be computed analytically and a small uncertainty in the measurements may induce a large uncertainty in the solutions. Our approach resorts to deep learning techniques in order to support reasoning for this family of non-linear inverse problems. In this thesis, we will employ the forward model to generate the dataset used to train the neural network which is going to be used as a regression model to approximate the desired inverse function. This work will be applied to a research area widely used in climate change studies with potential applications in water quality monitoring, denominated Ocean Color. We aim to obtain a model that is capable of accurately estimating the concentration of active seawater compounds, from remote sensing measurements of the sea surface reflectance taking into consideration the impact of uncertainty on the sensor observations and the model approximations.Houve múltiplos modelos matemáticos propostos por cientistas que permitem cor- roborar o comportamento de sistemas complexos. No entanto, estes modelos estimam valores que podem ser medidos porém são incapazes de determinar os parâmetros que causaram tais comportamentos. Os problemas inversos visam encontrar os parâmetros de um modelo, dados por sis- temas de equações, a partir de observações/medições ruidosas. Estes são tipicamente problemas ill-posed que podem não ter soluções exactas, ter múltiplas soluções ou solu- ções instáveis. Nesta tese, iremos restringir o nosso trabalho a problemas inversos não lineares que devido à sua complexidade não podem ser computados analiticamente e uma pequena incerteza nas medições pode induzir uma grande incerteza nas soluções. A nossa abordagem recorre a técnicas de aprendizagem profunda com o intuito de arranjar soluções para resolver esta familia de problemas inversos não lineares. Nesta tese vamos empregar o modelo direto para gerar o conjunto de dados utilizado para treinar a rede neural que vai ser utilizado como modelo de regressão para aproximar a função inversa desejada. Este trabalho será aplicado a uma área de investigação amplamente utilizada em estu- dos sobre alterações climáticas com potenciais aplicações na monitorização da qualidade da água, denominada Ocean Color. O nosso objectivo é obter um modelo capaz de estimar com precisão a concentração de compostos activos da água do mar, a partir de medições de detecção remota da reflectância da superfície marítima, tendo em consideração o impacto da incerteza nas observações do sensor e nas aproximações do modelo

Probabilistic constraint reasoning

Dissertação apresentada para obtenção do Grau de Doutor em Engenharia Informática, pela Universidade Nova de Lisboa, Faculdade de Ciências e TecnologiaThe continuous constraint paradigm has been often used to model safe reasoning in applications where uncertainty arises. Constraint propagation propagates intervals of uncertainty among the variables of the problem, eliminating values that do not belong to any solution. However, constraint programming is very conservative: if initial intervals are wide (reflecting large uncertainty), the obtained safe enclosure of all consistent scenarios may be inadequately wide for decision support. Since all scenarios are considered equally likely, insufficient pruning leads to great inefficiency if some costly decisions may be justified by very unlikely scenarios. Even when probabilistic information is available for the variables of the problem, the continuous constraint paradigm is unable to incorporate and reason with such information. Therefore, it is incapable of distinguishing between different scenarios, based on their likelihoods. This thesis presents a probabilistic continuous constraint paradigm that associates a probabilistic space to the variables of the problem, enabling probabilistic reasoning to complement the underlying constraint reasoning. Such reasoning is used to address probabilistic queries and requires the computation of multi-dimensional integrals on possibly non linear integration regions. Suitable algorithms for such queries are developed, using safe or approximate integration techniques and relying on methods from continuous constraint programming in order to compute safe covers of the integration region. The thesis illustrates the adequacy of the probabilistic continuous constraint framework for decision support in nonlinear continuous problems with uncertain information, namely on inverse and reliability problems, two different types of engineering problems where the developed framework is particularly adequate to support decision makers

RVM Classification of Hyperspectral Images Based on Wavelet Kernel Non-negative Matrix Fractorization

A novel kernel framework for hyperspectral image classification based on relevance vector machine (RVM) is presented in this paper. The new feature extraction algorithm based on Mexican hat wavelet kernel non-negative matrix factorization (WKNMF) for hyperspectral remote sensing images is proposed. By using the feature of multi-resolution analysis, the new method of nonlinear mapping capability based on kernel NMF can be improved. The new classification framework of hyperspectral image data combined with the novel WKNMF and RVM. The simulation experimental results on HYDICE and AVIRIS data sets are both show that the classification accuracy of proposed method compared with other experiment methods even can be improved over 10% in some cases and the classification precision of small sample data area can be improved effectively

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Bayesian Methodology for Ocean Color Remote Sensing

66 pagesThe inverse ocean color problem, i.e., the retrieval of marine reflectance from top-of-atmosphere (TOA) reflectance, is examined in a Bayesian context. The solution is expressed as a probability distribution that measures the likelihood of encountering specific values of the marine reflectance given the observed TOA reflectance. This conditional distribution, the posterior distribution, allows the construction of reliable multi-dimensional confidence domains of the retrieved marine reflectance. The expectation and covariance of the posterior distribution are computed, which gives for each pixel an estimate of the marine reflectance and a measure of its uncertainty. Situations for which forward model and observation are incompatible are also identified. Prior distributions of the forward model parameters that are suitable for use at the global scale, as well as a noise model, are determined. Partition-based models are defined and implemented for SeaWiFS, to approximate numerically the expectation and covariance. The ill-posed nature of the inverse problem is illustrated, indicating that a large set of ocean and atmospheric states, or pre-images, may correspond to very close values of the satellite signal. Theoretical performance is good globally, i.e., on average over all the geometric and geophysical situations considered, with negligible biases and standard deviation decreasing from 0.004 at 412 nm to 0.001 at 670 nm. Errors are smaller for geometries that avoid Sun glint and minimize air mass and aerosol influence, and for small aerosol optical thickness and maritime aerosols. The estimated uncertainty is consistent with the inversion error. The theoretical concepts and inverse models are applied to actual SeaWiFS imagery, and comparisons are made with estimates from the SeaDAS standard atmospheric correction algorithm and in situ measurements. The Bayesian and SeaDAS marine reflectance fields exhibit resemblance in patterns of variability, but the Bayesian imagery is less noisy and characterized by different spatial de-correlation scales, with more realistic values in the presence of absorbing aerosols. Experimental errors obtained from match-up data are similar to the theoretical errors determined from simulated data. Regionalization of the inverse models is a natural development to improve retrieval accuracy, for example by including explicit knowledge of the space and time variability of atmospheric variables

Watershed rainfall forecasting using neuro-fuzzy networks with the assimilation of multi-sensor information

The complex temporal heterogeneity of rainfall coupled with mountainous physiographic context makes a great challenge in the development of accurate short-term rainfall forecasts. This study aims to explore the effectiveness of multiple rainfall sources (gauge measurement, and radar and satellite products) for assimilation-based multi-sensor precipitation estimates and make multi-step-ahead rainfall forecasts based on the assimilated precipitation. Bias correction procedures for both radar and satellite precipitation products were first built, and the radar and satellite precipitation products were generated through the Quantitative Precipitation Estimation and Segregation Using Multiple Sensors (QPESUMS) and the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks-Cloud Classification System (PERSIANN-CCS), respectively. Next, the synthesized assimilated precipitation was obtained by merging three precipitation sources (gauges, radars and satellites) according to their individual weighting factors optimized by nonlinear search methods. Finally, the multi-step-ahead rainfall forecasting was carried out by using the adaptive network-based fuzzy inference system (ANFIS). The Shihmen Reservoir watershed in northern Taiwan was the study area, where 641 hourly data sets of thirteen historical typhoon events were collected. Results revealed that the bias adjustments in QPESUMS and PERSIANN-CCS products did improve the accuracy of these precipitation products (in particular, 30-60% improvement rates for the QPESUMS, in terms of RMSE), and the adjusted PERSIANN-CCS and QPESUMS individually provided about 10% and 24% contribution accordingly to the assimilated precipitation. As far as rainfall forecasting is concerned, the results demonstrated that the ANFIS fed with the assimilated precipitation provided reliable and stable forecasts with the correlation coefficients higher than 0.85 and 0.72 for one- and two-hour-ahead rainfall forecasting, respectively. The obtained forecasting results are very valuable information for the flood warning in the study watershed during typhoon periods. © 2013 Elsevier B.V

Spatial-Temporal Data Mining for Ocean Science: Data, Methodologies, and Opportunities

With the increasing amount of spatial-temporal~(ST) ocean data, numerous spatial-temporal data mining (STDM) studies have been conducted to address various oceanic issues, e.g., climate forecasting and disaster warning. Compared with typical ST data (e.g., traffic data), ST ocean data is more complicated with some unique characteristics, e.g., diverse regionality and high sparsity. These characteristics make it difficult to design and train STDM models. Unfortunately, an overview of these studies is still missing, hindering computer scientists to identify the research issues in ocean while discouraging researchers in ocean science from applying advanced STDM techniques. To remedy this situation, we provide a comprehensive survey to summarize existing STDM studies in ocean. Concretely, we first summarize the widely-used ST ocean datasets and identify their unique characteristics. Then, typical ST ocean data quality enhancement techniques are discussed. Next, we classify existing STDM studies for ocean into four types of tasks, i.e., prediction, event detection, pattern mining, and anomaly detection, and elaborate the techniques for these tasks. Finally, promising research opportunities are highlighted. This survey will help scientists from the fields of both computer science and ocean science have a better understanding of the fundamental concepts, key techniques, and open challenges of STDM in ocean