5 research outputs found
PAC-Bayes unleashed: generalisation bounds with unbounded losses
We present new PAC-Bayesian generalisation bounds for learning problems with
unbounded loss functions. This extends the relevance and applicability of the
PAC-Bayes learning framework, where most of the existing literature focuses on
supervised learning problems with a bounded loss function (typically assumed to
take values in the interval [0;1]). In order to relax this assumption, we
propose a new notion called HYPE (standing for \emph{HYPothesis-dependent
rangE}), which effectively allows the range of the loss to depend on each
predictor. Based on this new notion we derive a novel PAC-Bayesian
generalisation bound for unbounded loss functions, and we instantiate it on a
linear regression problem. To make our theory usable by the largest audience
possible, we include discussions on actual computation, practicality and
limitations of our assumptions.Comment: 24 page
Upper and Lower Bounds on the Performance of Kernel PCA
27 pagesPrincipal Component Analysis (PCA) is a popular method for dimension reduction and has attracted an unfailing interest for decades. Recently, kernel PCA has emerged as an extension of PCA but, despite its use in practice, a sound theoretical understanding of kernel PCA is missing. In this paper, we contribute lower and upper bounds on the efficiency of kernel PCA, involving the empirical eigenvalues of the kernel Gram matrix. Two bounds are for fixed estimators, and two are for randomized estimators through the PAC-Bayes theory. We control how much information is captured by kernel PCA on average, and we dissect the bounds to highlight strengths and limitations of the kernel PCA algorithm. Therefore, we contribute to the better understanding of kernel PCA. Our bounds are briefly illustrated on a toy numerical example