Spectral kernels for Gaussian processes

Gaussian processes are flexible distributions over functions, which provide a nonparametric nonlinear Bayesian regression framework. The covariance kernel, an operator determining the similarity between two points, is central to every Gaussian process model, for it encodes prior knowledge of the function being inferred. In this work, we extend the expressive power of Gaussian process models by proposing two families of kernels, over which an efficient search of expressive hidden representations of data is possible. The harmonizable mixture kernel (HMK) is a theoretically sound approach, to derive a parametric kernel by taking the Fourier transform of a generalized spectral density, modeled by a Gaussian mixture model. The convolutional spectral kernel (CSK) is a nonparametric kernel generalizing HMK, derived from taking the convolution of two spectral mixture kernel feature maps. We show that the two classes of kernels theoretically exhibit high levels of expressiveness, and we introduce Wigner distribution functions as a useful tool to interpret kernels. We also study efficient inference specially designed for the two new kernel families. We propose variational Fourier features (VFF), an inter-domain sparse inference approach utilizing the generalized spectral density.\par Experiments are extensively conducted for the two kernels and one new inference methods. We demonstrate experimentally that HMK interpolates between local patterns, and VFF offers a robust framework for learning kernel hyperparameters. We show that CSK can extract complex patterns using a nonparametric approach, with the added advantage of adapting spectral frequencies for each pair of data points

Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey

Data-driven paradigms are well-known and salient demands of future wireless communication. Empowered by big data and machine learning, next-generation data-driven communication systems will be intelligent with the characteristics of expressiveness, scalability, interpretability, and especially uncertainty modeling, which can confidently involve diversified latent demands and personalized services in the foreseeable future. In this paper, we review a promising family of nonparametric Bayesian machine learning methods, i.e., Gaussian processes (GPs), and their applications in wireless communication. Since GPs achieve the expressive and interpretable learning ability with uncertainty, it is particularly suitable for wireless communication. Moreover, it provides a natural framework for collaborating data and empirical models (DEM). Specifically, we first envision three-level motivations of data-driven wireless communication using GPs. Then, we present the background of the GPs in terms of covariance structure and model inference. The expressiveness of the GP model using various interpretable kernel designs is surveyed, namely, stationary, non-stationary, deep, and multi-task kernels. Furthermore, we review the distributed GPs with promising scalability, which is suitable for applications in wireless networks with a large number of distributed edge devices. Finally, we list representative solutions and promising techniques that adopt GPs in wireless communication systems

Principled methods for mixtures processing

This document is my thesis for getting the habilitation à diriger des recherches, which is the french diploma that is required to fully supervise Ph.D. students. It summarizes the research I did in the last 15 years and also provides the short­term research directions and applications I want to investigate. Regarding my past research, I first describe the work I did on probabilistic audio modeling, including the separation of Gaussian and α­stable stochastic processes. Then, I mention my work on deep learning applied to audio, which rapidly turned into a large effort for community service. Finally, I present my contributions in machine learning, with some works on hardware compressed sensing and probabilistic generative models.My research programme involves a theoretical part that revolves around probabilistic machine learning, and an applied part that concerns the processing of time series arising in both audio and life sciences