Cross-Fertilizing Strategies for Better EM Mountain Climbing and DA Field Exploration: A Graphical Guide Book

In recent years, a variety of extensions and refinements have been developed for data augmentation based model fitting routines. These developments aim to extend the application, improve the speed and/or simplify the implementation of data augmentation methods, such as the deterministic EM algorithm for mode finding and stochastic Gibbs sampler and other auxiliary-variable based methods for posterior sampling. In this overview article we graphically illustrate and compare a number of these extensions, all of which aim to maintain the simplicity and computation stability of their predecessors. We particularly emphasize the usefulness of identifying similarities between the deterministic and stochastic counterparts as we seek more efficient computational strategies. We also demonstrate the applicability of data augmentation methods for handling complex models with highly hierarchical structure, using a high-energy high-resolution spectral imaging model for data from satellite telescopes, such as the Chandra X-ray Observatory.Comment: Published in at http://dx.doi.org/10.1214/09-STS309 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Sample efficiency, transfer learning and interpretability for deep reinforcement learning

Deep learning has revolutionised artificial intelligence, where the application of increased compute to train neural networks on large datasets has resulted in improvements in real-world applications such as object detection, text-to-speech synthesis and machine translation. Deep reinforcement learning (DRL) has similarly shown impressive results in board and video games, but less so in real-world applications such as robotic control. To address this, I have investigated three factors prohibiting further deployment of DRL: sample efficiency, transfer learning, and interpretability. To decrease the amount of data needed to train DRL systems, I have explored various storage strategies and exploration policies for episodic control (EC) algorithms, resulting in the application of online clustering to improve the memory efficiency of EC algorithms, and the maximum entropy mellowmax policy for improving the sample efficiency and final performance of the same EC algorithms. To improve performance during transfer learning, I have shown that a multi-headed neural network architecture trained using hierarchical reinforcement learning can retain the benefits of positive transfer between tasks while mitigating the interference effects of negative transfer. I additionally investigated the use of multi-headed architectures to reduce catastrophic forgetting under the continual learning setting. While the use of multiple heads worked well within a simple environment, it was of limited use within a more complex domain, indicating that this strategy does not scale well. Finally, I applied a wide range of quantitative and qualitative techniques to better interpret trained DRL agents. In particular, I compared the effects of training DRL agents both with and without visual domain randomisation (DR), a popular technique to achieve simulation-to-real transfer, providing a series of tests that can be applied before real-world deployment. One of the major findings is that DR produces more entangled representations within trained DRL agents, indicating quantitatively that they are invariant to nuisance factors associated with the DR process. Additionally, while my environment allowed agents trained without DR to succeed without requiring complex recurrent processing, all agents trained with DR appear to integrate information over time, as evidenced through ablations on the recurrent state.Open Acces

Towards Active Event Recognition

Directing robot attention to recognise activities and to anticipate events like goal-directed actions is a crucial skill for human-robot interaction. Unfortunately, issues like intrinsic time constraints, the spatially distributed nature of the entailed information sources, and the existence of a multitude of unobservable states affecting the system, like latent intentions, have long rendered achievement of such skills a rather elusive goal. The problem tests the limits of current attention control systems. It requires an integrated solution for tracking, exploration and recognition, which traditionally have been seen as separate problems in active vision.We propose a probabilistic generative framework based on a mixture of Kalman filters and information gain maximisation that uses predictions in both recognition and attention-control. This framework can efficiently use the observations of one element in a dynamic environment to provide information on other elements, and consequently enables guided exploration.Interestingly, the sensors-control policy, directly derived from first principles, represents the intuitive trade-off between finding the most discriminative clues and maintaining overall awareness.Experiments on a simulated humanoid robot observing a human executing goal-oriented actions demonstrated improvement on recognition time and precision over baseline systems

Strategy Selection in Structured Populations

Evolutionary game theory studies frequency dependent selection. The fitness of a strategy is not constant, but depends on the relative frequencies of strategies in the population. This type of evolutionary dynamics occurs in many settings of ecology, infectious disease dynamics, animal behavior and social interactions of humans. Traditionally evolutionary game dynamics are studied in well-mixed populations, where the interaction between any two individuals is equally likely. There have also been several approaches to study evolutionary games in structured populations. In this paper we present a simple result that holds for a large variety of population structures. We consider the game between two strategies, A and B, described by the payoff matrix View the MathML source. We study a mutation and selection process. For weak selection strategy A is favored over B if and only if σa+b>c+σd. This means the effect of population structure on strategy selection can be described by a single parameter, σ. We present the values of σ for various examples including the well-mixed population, games on graphs, games in phenotype space and games on sets. We give a proof for the existence of such a σ, which holds for all population structures and update rules that have certain (natural) properties. We assume weak selection, but allow any mutation rate. We discuss the relationship between σ and the critical benefit to cost ratio for the evolution of cooperation. The single parameter, σ, allows us to quantify the ability of a population structure to promote the evolution of cooperation or to choose efficient equilibria in coordination games.MathematicsOrganismic and Evolutionary Biolog

Efficient Wasserstein Natural Gradients for Reinforcement Learning

A novel optimization approach is proposed for application to policy gradient methods and evolution strategies for reinforcement learning (RL). The procedure uses a computationally efficient Wasserstein natural gradient (WNG) descent that takes advantage of the geometry induced by a Wasserstein penalty to speed optimization. This method follows the recent theme in RL of including a divergence penalty in the objective to establish a trust region. Experiments on challenging tasks demonstrate improvements in both computational cost and performance over advanced baseline

Genetic mapping of metabolic biomarkers of cardiometabolic diseases

Cardiometabolic disorders (CMDs) are a major public health problem worldwide. The main goal of this thesis is to characterize the genetic architecture of CMD-related metabolites in a Lebanese cohort. In order to maximise the extraction of meaningful biological information from this dataset, an important part of this thesis focuses on the evaluation and subsequent improvement of the standard methods currently used for molecular epidemiology studies. First, I describe MetaboSignal, a novel network-based approach to explore the genetic regulation of the metabolome. Second, I comprehensively compare the recovery of metabolic information in the different 1H NMR strategies routinely used for metabolic profiling of plasma (standard 1D, spin-echo and JRES). Third, I describe a new method for dimensionality reduction of 1H NMR datasets prior to statistical modelling. Finally, I use all this methodological knowledge to search for molecular biomarkers of CMDs in a Lebanese population. Metabolome-wide association analyses identified a number of metabolites associated with CMDs, as well as several associations involving N-glycan units from acute-phase glycoproteins. Genetic mapping of these metabolites validated previously reported gene-metabolite associations, and revealed two novel loci associated with CMD-related metabolites. Collectively, this work contributes to the ongoing efforts to characterize the molecular mechanisms underlying complex human diseases.Open Acces

A Low-Complexity Double EP-based Detector for Iterative Detection and Decoding in MIMO

We propose a new iterative detection and decoding (IDD) algorithm for multiple-input multiple-output (MIMO) based on expectation propagation (EP) with application to massive MIMO scenarios. Two main results are presented. We first introduce EP to iteratively improve the Gaussian approximations of both the estimation of the posterior by the MIMO detector and the soft output of the channel decoder. With this novel approach, denoted by double-EP (DEP), the convergence is very much improved with a computational complexity just two times the one of the linear minimum mean square error (LMMSE) based IDD, as illustrated by the included experiments. Besides, as in the LMMSE MIMO detector, when the number of antennas increases, the computational cost of the matrix inversion operation required by the DEP becomes unaffordable. In this work we also develop approaches of DEP where the mean and the covariance matrix of the posterior are approximated by using the Gauss-Seidel and Neumann series methods, respectively. This low-complexity DEP detector has quadratic complexity in the number of antennas, as the low-complexity LMMSE techniques. Experimental results show that the new low-complexity DEP achieves the performance of the DEP as the ratio between the number of transmitting and receiving antennas decreasesProyectos Nacionales Españoles del Gobierno de España TEC2017-90093-C3-2-

A Bias-Aware EnKF Estimator for Aerodynamic Flows

Ensemble methods can integrate measurement data and CFD-based models to estimate the state of fluid systems in a robust and cost-efficient way. However, discretization errors can render numerical solutions a biased representation of reality. Left unaccounted for, biased forecast and observation models can lead to poor estimator performance. In this work, we propose a low-rank representation for the bias whose dynamics is represented by a colorednoise process. System state and bias parameters are simultaneously corrected on-line with the Ensemble Kalman Filter (EnKF) algorithm. The proposed methodology is demonstrated to achieve a 70% error reduction for the problem of estimating the state of the two-dimensional low-Re flow past a flat plate at high angle of attack using an ensemble of coarse-mesh simulations and pressure measurements at the surface of the body, compared to a bias-blind estimator. Strategies to determine the bias statistics and to deal with nonlinear observation functions in the context of ensemble methods are discussed

Bayesian optimisation for likelihood-free cosmological inference

Many cosmological models have only a finite number of parameters of interest, but a very expensive data-generating process and an intractable likelihood function. We address the problem of performing likelihood-free Bayesian inference from such black-box simulation-based models, under the constraint of a very limited simulation budget (typically a few thousand). To do so, we adopt an approach based on the likelihood of an alternative parametric model. Conventional approaches to approximate Bayesian computation such as likelihood-free rejection sampling are impractical for the considered problem, due to the lack of knowledge about how the parameters affect the discrepancy between observed and simulated data. As a response, we make use of a strategy previously developed in the machine learning literature (Bayesian optimisation for likelihood-free inference, BOLFI), which combines Gaussian process regression of the discrepancy to build a surrogate surface with Bayesian optimisation to actively acquire training data. We extend the method by deriving an acquisition function tailored for the purpose of minimising the expected uncertainty in the approximate posterior density, in the parametric approach. The resulting algorithm is applied to the problems of summarising Gaussian signals and inferring cosmological parameters from the Joint Lightcurve Analysis supernovae data. We show that the number of required simulations is reduced by several orders of magnitude, and that the proposed acquisition function produces more accurate posterior approximations, as compared to common strategies.Comment: 16+9 pages, 12 figures. Matches PRD published version after minor modification