62,002 research outputs found
Second-Order Stochastic Optimization for Machine Learning in Linear Time
First-order stochastic methods are the state-of-the-art in large-scale
machine learning optimization owing to efficient per-iteration complexity.
Second-order methods, while able to provide faster convergence, have been much
less explored due to the high cost of computing the second-order information.
In this paper we develop second-order stochastic methods for optimization
problems in machine learning that match the per-iteration cost of gradient
based methods, and in certain settings improve upon the overall running time
over popular first-order methods. Furthermore, our algorithm has the desirable
property of being implementable in time linear in the sparsity of the input
data
Self-similar prior and wavelet bases for hidden incompressible turbulent motion
This work is concerned with the ill-posed inverse problem of estimating
turbulent flows from the observation of an image sequence. From a Bayesian
perspective, a divergence-free isotropic fractional Brownian motion (fBm) is
chosen as a prior model for instantaneous turbulent velocity fields. This
self-similar prior characterizes accurately second-order statistics of velocity
fields in incompressible isotropic turbulence. Nevertheless, the associated
maximum a posteriori involves a fractional Laplacian operator which is delicate
to implement in practice. To deal with this issue, we propose to decompose the
divergent-free fBm on well-chosen wavelet bases. As a first alternative, we
propose to design wavelets as whitening filters. We show that these filters are
fractional Laplacian wavelets composed with the Leray projector. As a second
alternative, we use a divergence-free wavelet basis, which takes implicitly
into account the incompressibility constraint arising from physics. Although
the latter decomposition involves correlated wavelet coefficients, we are able
to handle this dependence in practice. Based on these two wavelet
decompositions, we finally provide effective and efficient algorithms to
approach the maximum a posteriori. An intensive numerical evaluation proves the
relevance of the proposed wavelet-based self-similar priors.Comment: SIAM Journal on Imaging Sciences, 201
- …