3 research outputs found
When is there a Representer Theorem? Reflexive Banach spaces
We consider a general regularised interpolation problem for learning a
parameter vector from data. The well known representer theorem says that under
certain conditions on the regulariser there exists a solution in the linear
span of the data points. This is at the core of kernel methods in machine
learning as it makes the problem computationally tractable. Most literature
deals only with sufficient conditions for representer theorems in Hilbert
spaces. We prove necessary and sufficient conditions for the existence of
representer theorems in reflexive Banach spaces and illustrate why in a sense
reflexivity is the minimal requirement on the function space. We further show
that if the learning relies on the linear representer theorem, then the
solution is independent of the regulariser and in fact determined by the
function space alone. This in particular shows the value of generalising
Hilbert space learning theory to Banach spaces.Comment: 25 pages, 2 figures. arXiv admin note: text overlap with
arXiv:1709.00084, arXiv:1804.0960
When is there a Representer Theorem? Nondifferentiable Regularisers and Banach spaces
We consider a general regularised interpolation problem for learning a
parameter vector from data. The well known representer theorem says that under
certain conditions on the regulariser there exists a solution in the linear
span of the data points. This is the core of kernel methods in machine learning
as it makes the problem computationally tractable. Necessary and sufficient
conditions for differentiable regularisers on Hilbert spaces to admit a
representer theorem have been proved. We extend those results to
nondifferentiable regularisers on uniformly convex and uniformly smooth Banach
spaces. This gives a (more) complete answer to the question when there is a
representer theorem. We then note that for regularised interpolation in fact
the solution is determined by the function space alone and independent of the
regulariser, making the extension to Banach spaces even more valuable.Comment: 20 pages, 3 figure
Approximate Representer Theorems in Non-reflexive Banach Spaces
The representer theorem is one of the most important mathematical foundations
for regularised learning and kernel methods. Classical formulations of the
theorem state sufficient conditions under which a regularisation problem on a
Hilbert space admits a solution in the subspace spanned by the representers of
the data points. This turns the problem into an equivalent optimisation problem
in a finite dimensional space, making it computationally tractable. Moreover,
Banach space methods for learning have been receiving more and more attention.
Considering the representer theorem in Banach spaces is hence of increasing
importance. Recently the question of the necessary condition for a representer
theorem to hold in Hilbert spaces and certain Banach spaces has been
considered. It has been shown that a classical representer theorem cannot exist
in general in non-reflexive Banach spaces. In this paper we propose a notion of
approximate solutions and approximate representer theorem to overcome this
problem. We show that for these notions we can indeed extend the previous
results to obtain a unified theory for the existence of representer theorems in
any general Banach spaces, in particular including -type spaces. We give a
precise characterisation when a regulariser admits a classical representer
theorem and when only an approximate representer theorem is possible.Comment: 18 pages, 1 figur