21 research outputs found
Accurate Optimization of Weighted Nuclear Norm for Non-Rigid Structure from Motion
Fitting a matrix of a given rank to data in a least squares sense can be done
very effectively using 2nd order methods such as Levenberg-Marquardt by
explicitly optimizing over a bilinear parameterization of the matrix. In
contrast, when applying more general singular value penalties, such as weighted
nuclear norm priors, direct optimization over the elements of the matrix is
typically used. Due to non-differentiability of the resulting objective
function, first order sub-gradient or splitting methods are predominantly used.
While these offer rapid iterations it is well known that they become inefficent
near the minimum due to zig-zagging and in practice one is therefore often
forced to settle for an approximate solution.
In this paper we show that more accurate results can in many cases be
achieved with 2nd order methods. Our main result shows how to construct
bilinear formulations, for a general class of regularizers including weighted
nuclear norm penalties, that are provably equivalent to the original problems.
With these formulations the regularizing function becomes twice differentiable
and 2nd order methods can be applied. We show experimentally, on a number of
structure from motion problems, that our approach outperforms state-of-the-art
methods
Wave Physics-informed Matrix Factorizations
With the recent success of representation learning methods, which includes
deep learning as a special case, there has been considerable interest in
developing techniques that incorporate known physical constraints into the
learned representation. As one example, in many applications that involve a
signal propagating through physical media (e.g., optics, acoustics, fluid
dynamics, etc), it is known that the dynamics of the signal must satisfy
constraints imposed by the wave equation. Here we propose a matrix
factorization technique that decomposes such signals into a sum of components,
where each component is regularized to ensure that it {nearly} satisfies wave
equation constraints. Although our proposed formulation is non-convex, we prove
that our model can be efficiently solved to global optimality. Through this
line of work we establish theoretical connections between wave-informed
learning and filtering theory in signal processing. We further demonstrate the
application of this work on modal analysis problems commonly arising in
structural diagnostics and prognostics.Comment: arXiv admin note: text overlap with arXiv:2107.0914