53 research outputs found
Discrete-Continuous ADMM for Transductive Inference in Higher-Order MRFs
This paper introduces a novel algorithm for transductive inference in
higher-order MRFs, where the unary energies are parameterized by a variable
classifier. The considered task is posed as a joint optimization problem in the
continuous classifier parameters and the discrete label variables. In contrast
to prior approaches such as convex relaxations, we propose an advantageous
decoupling of the objective function into discrete and continuous subproblems
and a novel, efficient optimization method related to ADMM. This approach
preserves integrality of the discrete label variables and guarantees global
convergence to a critical point. We demonstrate the advantages of our approach
in several experiments including video object segmentation on the DAVIS data
set and interactive image segmentation
Generalized Forward-Backward Splitting
This paper introduces the generalized forward-backward splitting algorithm
for minimizing convex functions of the form , where
has a Lipschitz-continuous gradient and the 's are simple in the sense
that their Moreau proximity operators are easy to compute. While the
forward-backward algorithm cannot deal with more than non-smooth
function, our method generalizes it to the case of arbitrary . Our method
makes an explicit use of the regularity of in the forward step, and the
proximity operators of the 's are applied in parallel in the backward
step. This allows the generalized forward backward to efficiently address an
important class of convex problems. We prove its convergence in infinite
dimension, and its robustness to errors on the computation of the proximity
operators and of the gradient of . Examples on inverse problems in imaging
demonstrate the advantage of the proposed methods in comparison to other
splitting algorithms.Comment: 24 pages, 4 figure
Communication-Efficient Algorithms For Distributed Optimization
This thesis is concerned with the design of distributed algorithms for
solving optimization problems. We consider networks where each node has
exclusive access to a cost function, and design algorithms that make all nodes
cooperate to find the minimum of the sum of all the cost functions. Several
problems in signal processing, control, and machine learning can be posed as
such optimization problems. Given that communication is often the most
energy-consuming operation in networks, it is important to design
communication-efficient algorithms. The main contributions of this thesis are a
classification scheme for distributed optimization and a set of corresponding
communication-efficient algorithms.
The class of optimization problems we consider is quite general, since each
function may depend on arbitrary components of the optimization variable, and
not necessarily on all of them. In doing so, we go beyond the common assumption
in distributed optimization and create additional structure that can be used to
reduce the number of communications. This structure is captured by our
classification scheme, which identifies easier instances of the problem, for
example the standard distributed optimization problem, where all functions
depend on all the components of the variable.
In our algorithms, no central node coordinates the network, all the
communications occur between neighboring nodes, and the data associated with
each node is processed locally. We show several applications including average
consensus, support vector machines, network flows, and several distributed
scenarios for compressed sensing. We also propose a new framework for
distributed model predictive control. Through extensive numerical experiments,
we show that our algorithms outperform prior distributed algorithms in terms of
communication-efficiency, even some that were specifically designed for a
particular application.Comment: Thesis defended on October 10, 2013. Dual PhD degree from Carnegie
Mellon University, PA, and Instituto Superior T\'ecnico, Lisbon, Portuga
- …