2,966 research outputs found
The geometry of nonlinear least squares with applications to sloppy models and optimization
Parameter estimation by nonlinear least squares minimization is a common
problem with an elegant geometric interpretation: the possible parameter values
of a model induce a manifold in the space of data predictions. The minimization
problem is then to find the point on the manifold closest to the data. We show
that the model manifolds of a large class of models, known as sloppy models,
have many universal features; they are characterized by a geometric series of
widths, extrinsic curvatures, and parameter-effects curvatures. A number of
common difficulties in optimizing least squares problems are due to this common
structure. First, algorithms tend to run into the boundaries of the model
manifold, causing parameters to diverge or become unphysical. We introduce the
model graph as an extension of the model manifold to remedy this problem. We
argue that appropriate priors can remove the boundaries and improve convergence
rates. We show that typical fits will have many evaporated parameters. Second,
bare model parameters are usually ill-suited to describing model behavior; cost
contours in parameter space tend to form hierarchies of plateaus and canyons.
Geometrically, we understand this inconvenient parametrization as an extremely
skewed coordinate basis and show that it induces a large parameter-effects
curvature on the manifold. Using coordinates based on geodesic motion, these
narrow canyons are transformed in many cases into a single quadratic, isotropic
basin. We interpret the modified Gauss-Newton and Levenberg-Marquardt fitting
algorithms as an Euler approximation to geodesic motion in these natural
coordinates on the model manifold and the model graph respectively. By adding a
geodesic acceleration adjustment to these algorithms, we alleviate the
difficulties from parameter-effects curvature, improving both efficiency and
success rates at finding good fits.Comment: 40 pages, 29 Figure
Compressive Parameter Estimation for Sparse Translation-Invariant Signals Using Polar Interpolation
We propose new compressive parameter estimation algorithms that make use of
polar interpolation to improve the estimator precision. Our work extends
previous approaches involving polar interpolation for compressive parameter
estimation in two aspects: (i) we extend the formulation from real non-negative
amplitude parameters to arbitrary complex ones, and (ii) we allow for mismatch
between the manifold described by the parameters and its polar approximation.
To quantify the improvements afforded by the proposed extensions, we evaluate
six algorithms for estimation of parameters in sparse translation-invariant
signals, exemplified with the time delay estimation problem. The evaluation is
based on three performance metrics: estimator precision, sampling rate and
computational complexity. We use compressive sensing with all the algorithms to
lower the necessary sampling rate and show that it is still possible to attain
good estimation precision and keep the computational complexity low. Our
numerical experiments show that the proposed algorithms outperform existing
approaches that either leverage polynomial interpolation or are based on a
conversion to a frequency-estimation problem followed by a super-resolution
algorithm. The algorithms studied here provide various tradeoffs between
computational complexity, estimation precision, and necessary sampling rate.
The work shows that compressive sensing for the class of sparse
translation-invariant signals allows for a decrease in sampling rate and that
the use of polar interpolation increases the estimation precision.Comment: 13 pages, 5 figures, to appear in IEEE Transactions on Signal
Processing; minor edits and correction
- …