Models and Fréchet kernels for frequency-(in)dependent Q

We present a new method for the modelling of frequency-dependent and frequency-independent Q in time-domain seismic wave propagation. Unlike previous approaches, attenuation models are constructed such that Q as a function of position in the Earth appears explicitly as a parameter in the equations of motion. This feature facilitates the derivation of Fréchet kernels for Q using adjoint techniques. Being simple products of the forward strain field and the adjoint memory variables, these kernels can be computed with no additional cost, compared to Fréchet kernels for elastic properties. The same holds for Fréchet kernels for the power-law exponent of frequency-dependent Q, that we derive as well. To illustrate our developments, we present examples from regional- and global-scale time-domain wave propagatio

Tensor Decomposition in Multiple Kernel Learning

Modern data processing and analytic tasks often deal with high dimensional matrices or tensors; for example: environmental sensors monitor (time, location, temperature, light) data. For large scale tensors, efficient data representation plays a major role in reducing computational time and finding patterns. The thesis firstly studies about fundamental matrix, tensor decomposition algorithms and applications, in connection with Tensor Train decomposition algorithm. The second objective is applying the tensor perspective in Multiple Kernel Learning problems, where the stacking of kernels can be seen as a tensor. Decomposition this kind of tensor leads to an efficient factorization approach in finding the best linear combination of kernels through the similarity alignment. Interestingly, thanks to the symmetry of the kernel matrix, a novel decomposition algorithm for multiple kernels is derived for reducing the computational complexity. In term of applications, this new approach allows the manipulation of large scale multiple kernels problems. For example, with P kernels and n samples, it reduces the memory complexity of O(P^2n^2) to O(P^2r^2+ 2rn) where r < n is the number of low-rank components. This compression is also valuable in pair-wise multiple kernel learning problem which models the relation among pairs of objects and its complexity is in the double scale. This study proposes AlignF_TT, a kernel alignment algorithm which is based on the novel decomposition algorithm for the tensor of kernels. Regarding the predictive performance, the proposed algorithm can gain an improvement in 18 artificially constructed datasets and achieve comparable performance in 13 real-world datasets in comparison with other multiple kernel learning algorithms. It also reveals that the small number of low-rank components is sufficient for approximating the tensor of kernels

Spontaneous stochasticity and renormalization group in discrete multi-scale dynamics

We introduce a class of multi-scale systems with discrete time, motivated by the problem of inviscid limit in fluid dynamics in the presence of small-scale noise. These systems are infinite-dimensional and defined on a scale-invariant space-time lattice. We propose a qualitative theory describing the vanishing regularization (inviscid) limit as an attractor of the renormalization group operator acting in the space of flow maps or respective probability kernels. If the attractor is a nontrivial probability kernel, we say that the inviscid limit is spontaneously stochastic: it defines a stochastic (Markov) process solving deterministic equations with deterministic initial and boundary conditions. The results are illustrated with solvable models: symbolic systems leading to digital turbulence and systems of expanding interacting phases.Comment: 34 pages, 10 figure

High-performance Kernel Machines with Implicit Distributed Optimization and Randomization

In order to fully utilize "big data", it is often required to use "big models". Such models tend to grow with the complexity and size of the training data, and do not make strong parametric assumptions upfront on the nature of the underlying statistical dependencies. Kernel methods fit this need well, as they constitute a versatile and principled statistical methodology for solving a wide range of non-parametric modelling problems. However, their high computational costs (in storage and time) pose a significant barrier to their widespread adoption in big data applications. We propose an algorithmic framework and high-performance implementation for massive-scale training of kernel-based statistical models, based on combining two key technical ingredients: (i) distributed general purpose convex optimization, and (ii) the use of randomization to improve the scalability of kernel methods. Our approach is based on a block-splitting variant of the Alternating Directions Method of Multipliers, carefully reconfigured to handle very large random feature matrices, while exploiting hybrid parallelism typically found in modern clusters of multicore machines. Our implementation supports a variety of statistical learning tasks by enabling several loss functions, regularization schemes, kernels, and layers of randomized approximations for both dense and sparse datasets, in a highly extensible framework. We evaluate the ability of our framework to learn models on data from applications, and provide a comparison against existing sequential and parallel libraries.Comment: Work presented at MMDS 2014 (June 2014) and JSM 201

Finite mixture models: visualisation, localised regression, and prediction

Initially, this thesis introduces a new graphical tool, that can be used to summarise data possessing a mixture structure. Computation of the required summary statistics makes use of posterior probabilities of class membership obtained from a fitted mixture model. In this context, both real and simulated data are used to highlight the usefulness of the tool for the visualisation of mixture data in comparison to the use of a traditional boxplot. This thesis uses localised mixture models to produce predictions from time series data. Estimation method used in these models is achieved using a kernel-weighted version of an EM-algorithm: exponential kernels with different bandwidths are used as weight functions. By modelling a mixture of local regressions at a target time point, but using different bandwidths, an informative estimated mixture probabilities can be gained relating to the amount of information available in the data set. This information is given a scale of resolution, that corresponds to each bandwidth. Nadaraya-Watson and local linear estimators are used to carry out localised estimation. For prediction at a future time point, a new methodology of bandwidth selection and adequate methods are proposed for each local method, and then compared to competing forecasting routines. A simulation study is executed to assess the performance of this model for prediction. Finally, double-localised mixture models are presented, that can be used to improve predictions for a variable time series using additional information provided by other time series. Estimation for these models is achieved using a double-kernel-weighted version of the EM-algorithm, employing exponential kernels with different horizontal bandwidths and normal kernels with different vertical bandwidths, that are focused around a target observation at a given time point. Nadaraya-Watson and local linear estimators are used to carry out the double-localised estimation. For prediction at a future time point, different approaches are considered for each local method, and are compared to competing forecasting routines. Real data is used to investigate the performance of the localised and double-localised mixture models for prediction. The data used predominately in this thesis is taken from the International Energy Agency (IEA)