High-order Moment Closure Models with Random Batch Method for Efficient Computation of Multiscale Turbulent Systems

We propose a high-order stochastic-statistical moment closure model for efficient ensemble prediction of leading-order statistical moments and probability density functions in multiscale complex turbulent systems. The statistical moment equations are closed by a precise calibration of the high-order feedbacks using ensemble solutions of the consistent stochastic equations, suitable for modeling complex phenomena including non-Gaussian statistics and extreme events. To address challenges associated with closely coupled spatio-temporal scales in turbulent states and expensive large ensemble simulation for high-dimensional systems, we introduce efficient computational strategies using the random batch method (RBM). This approach significantly reduces the required ensemble size while accurately capturing essential high-order structures. Only a small batch of small-scale fluctuation modes is used for each time update of the samples, and exact convergence to the full model statistics is ensured through frequent resampling of the batches during time evolution. Furthermore, we develop a reduced-order model to handle systems with really high dimension by linking the large number of small-scale fluctuation modes to ensemble samples of dominant leading modes. The effectiveness of the proposed models is validated by numerical experiments on the one-layer and two-layer Lorenz '96 systems, which exhibit representative chaotic features and various statistical regimes. The full and reduced-order RBM models demonstrate uniformly high skill in capturing the time evolution of crucial leading-order statistics, non-Gaussian probability distributions, while achieving significantly lower computational cost compared to direct Monte-Carlo approaches.Comment: 31 pages, 11 figure

A 3D maximum likelihood analysis for studying highly extended sources in VERITAS data

Imaging Atmospheric Cherenkov Telescopes have provided insights into many astrophysical phenomena including particle acceleration in SNR, studies of the extragalactic background light, and high energy emission from pulsars. However, current techniques for analyzing IACT data are poorly suited to studying emission greater than a few tenths of a degree. This thesis outlines a new analysis technique, known as the 3D MLM, designed to enhance the sensitivity of the VERITAS IACT array to sources from 0.5◦ to greater than 2◦ in radius. This analysis employs a maximum likelihood technique which models the expected distribution of events in the VERITAS data in two dimensions analogous to sky coordinates. Additionally a third dimension based on mean scaled width, a known gamma/hadron discrimination parameter, im- proves the sensitivity of the 3D MLM to such highly extended sources. Simulated VERITAS observations of 2FHL J0431.2+5553e, a 1.27◦ radius Fermi-LAT source, demonstrate the potential superior performance of this new technique over the standard ring background model analysis

Visual scene recognition with biologically relevant generative models

This research focuses on developing visual object categorization methodologies that are based on machine learning techniques and biologically inspired generative models of visual scene recognition. Modelling the statistical variability in visual patterns, in the space of features extracted from them by an appropriate low level signal processing technique, is an important matter of investigation for both humans and machines. To study this problem, we have examined in detail two recent probabilistic models of vision: a simple multivariate Gaussian model as suggested by (Karklin & Lewicki, 2009) and a restricted Boltzmann machine (RBM) proposed by (Hinton, 2002). Both the models have been widely used for visual object classification and scene analysis tasks before. This research highlights that these models on their own are not plausible enough to perform the classification task, and suggests Fisher kernel as a means of inducing discrimination into these models for classification power. Our empirical results on standard benchmark data sets reveal that the classification performance of these generative models could be significantly boosted near to the state of the art performance, by drawing a Fisher kernel from compact generative models that computes the data labels in a fraction of total computation time. We compare the proposed technique with other distance based and kernel based classifiers to show how computationally efficient the Fisher kernels are. To the best of our knowledge, Fisher kernel has not been drawn from the RBM before, so the work presented in the thesis is novel in terms of its idea and application to vision problem

Weighted reduced basis methods for parabolic PDEs with random input data

This work focuses on model order reduction for parabolic partial differential equations with parametrized random input data. The input data enter the model via model coefficients, external sources or boundary conditions, for instance. The outcome of the model problem is not only the solution, but also a quantity of interest (or output). The output is determined by a functional which maps the solution to a real number. Random data cause randomness of the outcomes of the model problem and, hence, statistical quantities are of interest. In particular, this work studies the expected value of the solution and the output. In order to approximate the expectation, a Monte Carlo estimator is utilized. For high accuracy Monte Carlo requires many evaluations of the underlying problem and, hence, it can become computationally infeasible. In order to overcome this computational issue, a reduced basis method (RBM) is considered. The RBM is a Galerkin projection onto a low-dimensional space (reduced basis space). The construction of the reduced basis space combines a proper orthogonal decomposition (POD) with a greedy approach, called POD-greedy algorithm, which is state of the art for the RBM for parabolic problems. The POD-greedy uses computationally cheap error estimators in order to build a reduced basis. This thesis proposes efficient reduced order models regarding the expected value of the errors resulting from the model order reduction. To this end, the probability density function of the random input data is used as a weight for the reduced space construction of the RBM, called weighted RBM. In the past, a weighted RBM has been successfully applied to elliptic partial differential equations with parametrized random input data. This work combines the ideas of a RBM for parabolic partial differential equations Grepl and Patera (2005) and a weighted RBM for elliptic problems Chen et al. (2013) in order to extend the weighted approach also for the RBM for parabolic problems. The performance of a non-weighted and a weighted approach are compared numerically with respect to the expected solution error and the expected output error. Furthermore, this work provides a numerical comparison of a non-weighted RBM and a weighted RBM regarding an optimum reference. The reference is obtained by an orthogonal projection onto a POD space, which minimizes the expected squared solution error

Learning a Restricted Boltzmann Machine using biased Monte Carlo sampling

International audienceRestricted Boltzmann Machines are simple and powerful generative models that can encode any complex dataset. Despite all their advantages, in practice the trainings are often unstable and it is difficult to assess their quality because the dynamics are affected by extremely slow time dependencies. This situation becomes critical when dealing with low-dimensional clustered datasets, where the time required to sample ergodically the trained models becomes computationally prohibitive. In this work, we show that this divergence of Monte Carlo mixing times is related to a phenomenon of phase coexistence, similar to that which occurs in physics near a first-order phase transition. We show that sampling the equilibrium distribution using the Markov chain Monte Carlo method can be dramatically accelerated when using biased sampling techniques, in particular the Tethered Monte Carlo (TMC) method. This sampling technique efficiently solves the problem of evaluating the quality of a given trained model and generating new samples in a reasonable amount of time. Moreover, we show that this sampling technique can also be used to improve the computation of the log-likelihood gradient during training, leading to dramatic improvements in training RBMs with artificial clustered datasets. On real low-dimensional datasets, this new training method fits RBM models with significantly faster relaxation dynamics than those obtained with standard PCD recipes. We also show that TMC sampling can be used to recover the free-energy profile of the RBM. This proves to be extremely useful to compute the probability distribution of a given model and to improve the generation of new decorrelated samples in slow PCD-trained models. The main limitations of this method are, first, the restriction to effective low-dimensional datasets and, second, the fact that the Tethered MC method breaks the possibility of performing parallel alternative Monte Carlo updates, which limits the size of the systems we can consider in practice