Probabilistic Machine Learning in the Age of Deep Learning: New Perspectives for Gaussian Processes, Bayesian Optimisation and Beyond

Advances in artificial intelligence (AI) are rapidly transforming our world, with systems now surpassing human capabilities in numerous domains. Much of this progress traces back to machine learning (ML), particularly deep learning, and its ability to uncover meaningful patterns in data. However, true intelligence in AI demands more than raw predictive power; it requires a principled approach to making decisions under uncertainty. Probabilistic ML offers a framework for reasoning about the unknown in ML models through probability theory and Bayesian inference. Gaussian processes (GPs) are a quintessential probabilistic model known for their flexibility, data efficiency, and well-calibrated uncertainty estimates. GPs are integral to sequential decision-making algorithms like Bayesian optimisation (BO), which optimises expensive black-box objective functions. Despite efforts to improve GP scalability, performance gaps persist compared to neural networks (NNs) due to their lack of representation learning capabilities. This thesis aims to integrate deep learning with probabilistic methods and lend probabilistic perspectives to deep learning. Key contributions include: (1) Extending orthogonally-decoupled sparse GP approximations to incorporate nonlinear NN activations as inter-domain features, bringing predictive performance closer to NNs. (2) Framing cycle-consistent adversarial networks (CYCLEGANs) for unpaired image-to-image translation as variational inference (VI) in an implicit latent variable model, providing a Bayesian perspective on these deep generative models. (3) Introducing a model-agnostic reformulation of BO based on binary classification, enabling the integration of powerful modelling approaches like deep learning for complex optimisation tasks. By enriching the interplay between deep learning and probabilistic ML, this thesis advances the foundations of AI, facilitating the development of more capable and dependable automated decision-making systems

Advances in knowledge discovery and data mining Part II

19th Pacific-Asia Conference, PAKDD 2015, Ho Chi Minh City, Vietnam, May 19-22, 2015, Proceedings, Part II</p

Nyström landmark sampling and regularized Christoffel functions

More details in the proofs. Typos correctedSelecting diverse and important items, called landmarks, from a large set is a problem of interest in machine learning. As a specific example, in order to deal with large training sets, kernel methods often rely on low rank matrix Nyström approximations based on the selection or sampling of landmarks. In this context, we propose a deterministic and a randomized adaptive algorithm for selecting landmark points within a training data set. These landmarks are related to the minima of a sequence of kernelized Christoffel functions. Beyond the known connection between Christoffel functions and leverage scores, a connection of our method with finite determinantal point processes (DPPs) is also explained. Namely, our construction promotes diversity among important landmark points in a way similar to DPPs. Also, we explain how our randomized adaptive algorithm can influence the accuracy of Kernel Ridge Regression

Generalized averaged Gaussian quadrature and applications

A simple numerical method for constructing the optimal generalized averaged Gaussian quadrature formulas will be presented. These formulas exist in many cases in which real positive GaussKronrod formulas do not exist, and can be used as an adequate alternative in order to estimate the error of a Gaussian rule. We also investigate the conditions under which the optimal averaged Gaussian quadrature formulas and their truncated variants are internal

MS FT-2-2 7 Orthogonal polynomials and quadrature: Theory, computation, and applications

Quadrature rules find many applications in science and engineering. Their analysis is a classical area of applied mathematics and continues to attract considerable attention. This seminar brings together speakers with expertise in a large variety of quadrature rules. It is the aim of the seminar to provide an overview of recent developments in the analysis of quadrature rules. The computation of error estimates and novel applications also are described