We consider the problem of minimizing block-separable convex functions
subject to linear constraints. While the Alternating Direction Method of
Multipliers (ADMM) for two-block linear constraints has been intensively
studied both theoretically and empirically, in spite of some preliminary work,
effective generalizations of ADMM to multiple blocks is still unclear. In this
paper, we propose a randomized block coordinate method named Parallel Direction
Method of Multipliers (PDMM) to solve the optimization problems with
multi-block linear constraints. PDMM randomly updates some primal and dual
blocks in parallel, behaving like parallel randomized block coordinate descent.
We establish the global convergence and the iteration complexity for PDMM with
constant step size. We also show that PDMM can do randomized block coordinate
descent on overlapping blocks. Experimental results show that PDMM performs
better than state-of-the-arts methods in two applications, robust principal
component analysis and overlapping group lasso.Comment: This paper has been withdrawn by the authors. There are errors in
Equations from 139-19
