This paper presents a Bayesian framework to construct non-linear,
parsimonious, shallow models for multitask regression. The proposed framework
relies on the fact that Random Fourier Features (RFFs) enables the
approximation of an RBF kernel by an extreme learning machine whose hidden
layer is formed by RFFs. The main idea is to combine both dual views of a same
model under a single Bayesian formulation that extends the Sparse Bayesian
Extreme Learning Machines to multitask problems. From the kernel methods point
of view, the proposed formulation facilitates the introduction of prior domain
knowledge through the RBF kernel parameter. From the extreme learning machines
perspective, the new formulation helps control overfitting and enables a
parsimonious overall model (the models that serve each task share a same set of
RFFs selected within the joint Bayesian optimisation). The experimental results
show that combining advantages from kernel methods and extreme learning
machines within the same framework can lead to significant improvements in the
performance achieved by each of these two paradigms independently