2,147 research outputs found
Separable approximations and decomposition methods for the augmented Lagrangian
In this paper we study decomposition methods based on separable approximations for minimizing the augmented Lagrangian. In particular, we study and compare the Diagonal Quadratic Approximation Method (DQAM) of Mulvey and Ruszczyński [13] and the Parallel Coordinate Descent Method (PCDM) of Richtárik and Takáč [23]. We show that the two methods are equivalent for feasibility problems up to the selection of a single step-size parameter. Furthermore, we prove an improved complexity bound for PCDM under strong convexity, and show that this bound is at least 8(L ′ / ¯ L)(ω − 1) 2 times better than the best known bound for DQAM, where ω is the degree of partial separability and L ′ and ¯ L are the maximum and average of the block Lipschitz constants of the gradient of the quadratic penalty appearing in the augmented Lagrangian.
A decomposition procedure based on approximate newton directions
The efficient solution of large-scale linear and nonlinear optimization problems may require exploiting any special structure in them in an efficient manner. We describe and analyze some cases in which this special structure can be used with very little cost to obtain search directions from decomposed subproblems. We also study how to correct these directions using (decomposable) preconditioned conjugate gradient methods to ensure local convergence in all cases. The choice of appropriate preconditioners results in a natural manner from the structure in the problem. Finally, we conduct computational experiments to compare the resulting procedures with direct methods, as well as to study the impact of different preconditioner choices
Consistent Dynamic Mode Decomposition
We propose a new method for computing Dynamic Mode Decomposition (DMD)
evolution matrices, which we use to analyze dynamical systems. Unlike the
majority of existing methods, our approach is based on a variational
formulation consisting of data alignment penalty terms and constitutive
orthogonality constraints. Our method does not make any assumptions on the
structure of the data or their size, and thus it is applicable to a wide range
of problems including non-linear scenarios or extremely small observation sets.
In addition, our technique is robust to noise that is independent of the
dynamics and it does not require input data to be sequential. Our key idea is
to introduce a regularization term for the forward and backward dynamics. The
obtained minimization problem is solved efficiently using the Alternating
Method of Multipliers (ADMM) which requires two Sylvester equation solves per
iteration. Our numerical scheme converges empirically and is similar to a
provably convergent ADMM scheme. We compare our approach to various
state-of-the-art methods on several benchmark dynamical systems
A DECOMPOSITION PROCEDURE BASED ON APPROXIMATE NEWTON DIRECTIONS
The efficient solution of large-scale linear and nonlinear optimization problems may require exploiting any special structure in them in an efficient manner. We describe and analyze some cases in which this special structure can be used with very little cost to obtain search directions from decomposed subproblems. We also study how to correct these directions using (decomposable) preconditioned conjugate gradient methods to ensure local convergence in all cases. The choice of appropriate preconditioners results in a natural manner from the structure in the problem. Finally, we conduct computational experiments to compare the resulting procedures with direct methods, as well as to study the impact of different preconditioner choices.
- …