420 research outputs found
Pointwise Convergence in Probability of General Smoothing Splines
Establishing the convergence of splines can be cast as a variational problem
which is amenable to a -convergence approach. We consider the case in
which the regularization coefficient scales with the number of observations,
, as . Using standard theorems from the
-convergence literature, we prove that the general spline model is
consistent in that estimators converge in a sense slightly weaker than weak
convergence in probability for . Without further assumptions
we show this rate is sharp. This differs from rates for strong convergence
using Hilbert scales where one can often choose
Deep Limits of Residual Neural Networks
Neural networks have been very successful in many applications; we often,
however, lack a theoretical understanding of what the neural networks are
actually learning. This problem emerges when trying to generalise to new data
sets. The contribution of this paper is to show that, for the residual neural
network model, the deep layer limit coincides with a parameter estimation
problem for a nonlinear ordinary differential equation. In particular, whilst
it is known that the residual neural network model is a discretisation of an
ordinary differential equation, we show convergence in a variational sense.
This implies that optimal parameters converge in the deep layer limit. This is
a stronger statement than saying for a fixed parameter the residual neural
network model converges (the latter does not in general imply the former). Our
variational analysis provides a discrete-to-continuum -convergence
result for the objective function of the residual neural network training step
to a variational problem constrained by a system of ordinary differential
equations; this rigorously connects the discrete setting to a continuum
problem
Convergence and Rates for Fixed-Interval Multiple-Track Smoothing Using -Means Type Optimization
We address the task of estimating multiple trajectories from unlabeled data.
This problem arises in many settings, one could think of the construction of
maps of transport networks from passive observation of travellers, or the
reconstruction of the behaviour of uncooperative vehicles from external
observations, for example. There are two coupled problems. The first is a data
association problem: how to map data points onto individual trajectories. The
second is, given a solution to the data association problem, to estimate those
trajectories. We construct estimators as a solution to a regularized
variational problem (to which approximate solutions can be obtained via the
simple, efficient and widespread -means method) and show that, as the number
of data points, , increases, these estimators exhibit stable behaviour. More
precisely, we show that they converge in an appropriate Sobolev space in
probability and with rate
Convergence of the -Means Minimization Problem using -Convergence
The -means method is an iterative clustering algorithm which associates
each observation with one of clusters. It traditionally employs cluster
centers in the same space as the observed data. By relaxing this requirement,
it is possible to apply the -means method to infinite dimensional problems,
for example multiple target tracking and smoothing problems in the presence of
unknown data association. Via a -convergence argument, the associated
optimization problem is shown to converge in the sense that both the -means
minimum and minimizers converge in the large data limit to quantities which
depend upon the observed data only through its distribution. The theory is
supplemented with two examples to demonstrate the range of problems now
accessible by the -means method. The first example combines a non-parametric
smoothing problem with unknown data association. The second addresses tracking
using sparse data from a network of passive sensors
Consistency of Fractional Graph-Laplacian Regularization in Semi-Supervised Learning with Finite Labels
Laplace learning is a popular machine learning algorithm for finding missing
labels from a small number of labelled feature vectors using the geometry of a
graph. More precisely, Laplace learning is based on minimising a
graph-Dirichlet energy, equivalently a discrete Sobolev
semi-norm, constrained to taking the values of known labels on a given subset.
The variational problem is asymptotically ill-posed as the number of unlabeled
feature vectors goes to infinity for finite given labels due to a lack of
regularity in minimisers of the continuum Dirichlet energy in any dimension
higher than one. In particular, continuum minimisers are not continuous. One
solution is to consider higher-order regularisation, which is the analogue of
minimising Sobolev semi-norms. In this paper we consider the
asymptotics of minimising a graph variant of the Sobolev
semi-norm with pointwise constraints. We show that, as expected, one needs
where is the dimension of the data manifold. We also show that
there must be a upper bound on the connectivity of the graph; that is, highly
connected graphs lead to degenerate behaviour of the minimiser even when
.Comment: 37 pages, 4 figure
Large data limit for a phase transition model with the p-Laplacian on point clouds
The consistency of a nonlocal anisotropic Ginzburg-Landau type functional for
data classification and clustering is studied. The Ginzburg-Landau objective
functional combines a double well potential, that favours indicator valued
function, and the -Laplacian, that enforces regularity. Under appropriate
scaling between the two terms minimisers exhibit a phase transition on the
order of where is the number of data points. We study
the large data asymptotics, i.e. as , in the regime where
. The mathematical tool used to address this question is
-convergence. In particular, it is proved that the discrete model
converges to a weighted anisotropic perimeter
- …