Approximate Inference for Nonstationary Heteroscedastic Gaussian process Regression

This paper presents a novel approach for approximate integration over the uncertainty of noise and signal variances in Gaussian process (GP) regression. Our efficient and straightforward approach can also be applied to integration over input dependent noise variance (heteroscedasticity) and input dependent signal variance (nonstationarity) by setting independent GP priors for the noise and signal variances. We use expectation propagation (EP) for inference and compare results to Markov chain Monte Carlo in two simulated data sets and three empirical examples. The results show that EP produces comparable results with less computational burden

Agent behavior monitoring using optimal action selection and twin gaussian processes

The increasing trend towards delegating complex tasks to autonomous artificial agents in safety-critical socio-technical systems makes agent behavior monitoring of paramount importance. In this work, a probabilistic approach for on-line monitoring using optimal action selection and twin Gaussian processes (TGP) is proposed. A Kullback-Leibler (KL) based metric is proposed to characterize the deviation of an agent behavior (modeled as a controlled stochastic process) to its specification. The optimal behavior specification is obtained using Linearly Solvable Markov Decision Processes (LSMDP) whereby the Bellman equation is made linear through an exponential transformation such that the optimal control policy is obtained in an explicit form.Sociedad Argentina de Informática e Investigación Operativa (SADIO

Agent behavior monitoring using optimal action selection and twin gaussian processes

The increasing trend towards delegating complex tasks to autonomous artificial agents in safety-critical socio-technical systems makes agent behavior monitoring of paramount importance. In this work, a probabilistic approach for on-line monitoring using optimal action selection and twin Gaussian processes (TGP) is proposed. A Kullback-Leibler (KL) based metric is proposed to characterize the deviation of an agent behavior (modeled as a controlled stochastic process) to its specification. The optimal behavior specification is obtained using Linearly Solvable Markov Decision Processes (LSMDP) whereby the Bellman equation is made linear through an exponential transformation such that the optimal control policy is obtained in an explicit form.Sociedad Argentina de Informática e Investigación Operativa (SADIO

Modulating Surrogates for Bayesian Optimization

Bayesian optimization (BO) methods often rely on the assumption that the objective function is well-behaved, but in practice, this is seldom true for real-world objectives even if noise-free observations can be collected. Common approaches, which try to model the objective as precisely as possible, often fail to make progress by spending too many evaluations modeling irrelevant details. We address this issue by proposing surrogate models that focus on the well-behaved structure in the objective function, which is informative for search, while ignoring detrimental structure that is challenging to model from few observations. First, we demonstrate that surrogate models with appropriate noise distributions can absorb challenging structures in the objective function by treating them as irreducible uncertainty. Secondly, we show that a latent Gaussian process is an excellent surrogate for this purpose, comparing with Gaussian processes with standard noise distributions. We perform numerous experiments on a range of BO benchmarks and find that our approach improves reliability and performance when faced with challenging objective functions

Speeding up the inference in Gaussian process models

In this dissertation Gaussian processes are used to define prior distributions over latent functions in hierarchical Bayesian models. Gaussian process is a non-parametric model with which one does not need to fix the functional form of the latent function, but its properties can be defined implicitly. These implicit statements are encoded in the mean and covariance function, which determine, for example, the smoothness and variability of the function. This non-parametric nature of the Gaussian process gives rise to a flexible and diverse class of probabilistic models. There are two main challenges with using Gaussian processes. Their main complication is the computational time which increases rapidly as a function of a number of data points. Other challenge is the analytically intractable inference, which exacerbates the slow computational time. This dissertation considers methods to alleviate these problems. The inference problem is attacked with approximative methods. The Laplace approximation and expectation propagation algorithm are utilized to give Gaussian approximation to the conditional posterior distribution of the latent function given the hyperparameters. The integration over hyperparameters is performed using a Monte Carlo, a grid based, or a central composite design integration. Markov chain Monte Carlo methods over all unknown parameters are used as a golden standard to which the other methods are compared. The rapidly increasing computational time is cured with sparse approximations to Gaussian process and compactly supported covariance functions. These are both analyzed in detail and tested in experiments. Practical details on their implementation with the approximative inference techniques are discussed. The techniques for speeding up the inference are tested in three modeling problems. The problems considered are disease mapping, regression and classification. The disease mapping and regression problems are tackled with standard and robust observation models. The results show that the techniques presented speed up the inference considerably without compromising the accuracy severely

A Novel Black Box Process Quality Optimization Approach based on Hit Rate

Hit rate is a key performance metric in predicting process product quality in integrated industrial processes. It represents the percentage of products accepted by downstream processes within a controlled range of quality. However, optimizing hit rate is a non-convex and challenging problem. To address this issue, we propose a data-driven quasi-convex approach that combines factorial hidden Markov models, multitask elastic net, and quasi-convex optimization. Our approach converts the original non-convex problem into a set of convex feasible problems, achieving an optimal hit rate. We verify the convex optimization property and quasi-convex frontier through Monte Carlo simulations and real-world experiments in steel production. Results demonstrate that our approach outperforms classical models, improving hit rates by at least 41.11% and 31.01% on two real datasets. Furthermore, the quasi-convex frontier provides a reference explanation and visualization for the deterioration of solutions obtained by conventional models

Structure-property relationships in laser assisted and mechanically deformed advanced materials

This research concentrates on the structural crystallographic properties in laser assisted and mechanically deformed shapes within hexagonal closed packed (HCP) material. This thesis explores a new ‘challenge and opportunity’ for lasers in the field of manufacturing processing, i.e. using a laser as an innovative tool for materials forming purposes. Indeed the laser have been used before as a sophisticated heat source for manufacturing products but the processing-structure-property relationships have been largely neglected. Here, we present a detailed and in-depth study of the structure-performance relationships in laser assisted and mechanically deformed advanced materials

Crowd-Based Learning of Spatial Fields for the Internet of Things: From Harvesting of Data to Inference

open4siThe knowledge of spatial distributions of physical quantities, such as radio-frequency (RF) interference, pollution, geomagnetic field magnitude, temperature, humidity, audio, and light intensity, will foster the development of new context-aware applications. For example, knowing the distribution of RF interference might significantly improve cognitive radio systems [1], [2]. Similarly, knowing the spatial variations of the geomagnetic field could support autonomous navigation of robots (including drones) in factories and/or hazardous scenarios [3]. Other examples are related to the estimation of temperature gradients, detection of sources of RF signals, or percentages of certain chemical components. As a result, people could get personalized health-related information based on their exposure to sources of risks (e.g., chemical or pollution). We refer to these spatial distributions of physical quantities as spatial fields. All of the aforementioned examples have in common that learning the spatial fields requires a large number of sensors (agents) surveying the area [4], [5].embargoed_20190303Arias-De-Reyna, Eva; Closas, Pau; Dardari, Davide; Djuric, Petar M.Arias-De-Reyna, Eva; Closas, Pau; Dardari, Davide; Djuric, Petar M