Transfer Learning is an emerging framework for learning from data that aims at intelligently
transferring information between tasks. This is achieved by developing algorithms
that can perform multiple tasks simultaneously, as well as translating previously
acquired knowledge to novel learning problems.
In this thesis, we investigate the application of Gaussian Processes to various forms
of transfer learning with a focus on classification problems. This process initiates
with a thorough introduction to the framework of Transfer learning, providing a clear
taxonomy of the areas of research. Following that, we continue by reviewing the recent
advances on Multi-task learning for regression with Gaussian processes, and compare
the performance of some of these methods on a real data set. This review gives insights
about the strengths and weaknesses of each method, which acts as a point of reference
to apply these methods to other forms of transfer learning.
The main contributions of this thesis are reported in the three following chapters.
The third chapter investigates the application of Multi-task Gaussian processes to classification
problems. We extend a previously proposed model to the classification scenario,
providing three inference methods due to the non-Gaussian likelihood the classification
paradigm imposes. The forth chapter extends the multi-task scenario to the
semi-supervised case. Using labeled and unlabeled data, we construct a novel covariance
function that is able to capture the geometry of the distribution of each task. This
setup allows unlabeled data to be utilised to infer the level of correlation between the
tasks. Moreover, we also discuss the potential use of this model to situations where no
labeled data are available for certain tasks. The fifth chapter investigates a novel form
of transfer learning called meta-generalising. The question at hand is if, after training
on a sufficient number of tasks, it is possible to make predictions on a novel task. In
this situation, the predictor is embedded in an environment of multiple tasks but has no
information about the origins of the test task. This elevates the concept of generalising
from the level of data to the level of tasks. We employ a model based on a hierarchy
of Gaussian processes, in a mixtures of expert sense, to make predictions based on the
relation between the distributions of the novel and the training tasks. Each chapter is
accompanied with a thorough experimental part giving insights about the potentials
and the limits of the proposed methods