19,448 research outputs found
Analysis of error propagation in particle filters with approximation
This paper examines the impact of approximation steps that become necessary
when particle filters are implemented on resource-constrained platforms. We
consider particle filters that perform intermittent approximation, either by
subsampling the particles or by generating a parametric approximation. For such
algorithms, we derive time-uniform bounds on the weak-sense error and
present associated exponential inequalities. We motivate the theoretical
analysis by considering the leader node particle filter and present numerical
experiments exploring its performance and the relationship to the error bounds.Comment: Published in at http://dx.doi.org/10.1214/11-AAP760 the Annals of
Applied Probability (http://www.imstat.org/aap/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods
Training neural networks is a challenging non-convex optimization problem,
and backpropagation or gradient descent can get stuck in spurious local optima.
We propose a novel algorithm based on tensor decomposition for guaranteed
training of two-layer neural networks. We provide risk bounds for our proposed
method, with a polynomial sample complexity in the relevant parameters, such as
input dimension and number of neurons. While learning arbitrary target
functions is NP-hard, we provide transparent conditions on the function and the
input for learnability. Our training method is based on tensor decomposition,
which provably converges to the global optimum, under a set of mild
non-degeneracy conditions. It consists of simple embarrassingly parallel linear
and multi-linear operations, and is competitive with standard stochastic
gradient descent (SGD), in terms of computational complexity. Thus, we propose
a computationally efficient method with guaranteed risk bounds for training
neural networks with one hidden layer.Comment: The tensor decomposition analysis is expanded, and the analysis of
ridge regression is added for recovering the parameters of last layer of
neural networ
- …