Playing with Duality: An Overview of Recent Primal-Dual Approaches for Solving Large-Scale Optimization Problems

Optimization methods are at the core of many problems in signal/image processing, computer vision, and machine learning. For a long time, it has been recognized that looking at the dual of an optimization problem may drastically simplify its solution. Deriving efficient strategies which jointly brings into play the primal and the dual problems is however a more recent idea which has generated many important new contributions in the last years. These novel developments are grounded on recent advances in convex analysis, discrete optimization, parallel processing, and non-smooth optimization with emphasis on sparsity issues. In this paper, we aim at presenting the principles of primal-dual approaches, while giving an overview of numerical methods which have been proposed in different contexts. We show the benefits which can be drawn from primal-dual algorithms both for solving large-scale convex optimization problems and discrete ones, and we provide various application examples to illustrate their usefulness

A Computational Comparison of Optimization Methods for the Golomb Ruler Problem

The Golomb ruler problem is defined as follows: Given a positive integer n, locate n marks on a ruler such that the distance between any two distinct pair of marks are different from each other and the total length of the ruler is minimized. The Golomb ruler problem has applications in information theory, astronomy and communications, and it can be seen as a challenge for combinatorial optimization algorithms. Although constructing high quality rulers is well-studied, proving optimality is a far more challenging task. In this paper, we provide a computational comparison of different optimization paradigms, each using a different model (linear integer, constraint programming and quadratic integer) to certify that a given Golomb ruler is optimal. We propose several enhancements to improve the computational performance of each method by exploring bound tightening, valid inequalities, cutting planes and branching strategies. We conclude that a certain quadratic integer programming model solved through a Benders decomposition and strengthened by two types of valid inequalities performs the best in terms of solution time for small-sized Golomb ruler problem instances. On the other hand, a constraint programming model improved by range reduction and a particular branching strategy could have more potential to solve larger size instances due to its promising parallelization features

A note on the primal-dual method for the semi-metric labeling problem

Recently, Komodakis et al. [6] developed the FastPD algorithm for the semi-metric labeling problem, which extends the expansion move algorithm of Boykov et al. [2]. We present a slightly different derivation of the FastPD method

On Minimal Valid Inequalities for Mixed Integer Conic Programs

We study disjunctive conic sets involving a general regular (closed, convex, full dimensional, and pointed) cone K such as the nonnegative orthant, the Lorentz cone or the positive semidefinite cone. In a unified framework, we introduce K-minimal inequalities and show that under mild assumptions, these inequalities together with the trivial cone-implied inequalities are sufficient to describe the convex hull. We study the properties of K-minimal inequalities by establishing algebraic necessary conditions for an inequality to be K-minimal. This characterization leads to a broader algebraically defined class of K- sublinear inequalities. We establish a close connection between K-sublinear inequalities and the support functions of sets with a particular structure. This connection results in practical ways of showing that a given inequality is K-sublinear and K-minimal. Our framework generalizes some of the results from the mixed integer linear case. It is well known that the minimal inequalities for mixed integer linear programs are generated by sublinear (positively homogeneous, subadditive and convex) functions that are also piecewise linear. This result is easily recovered by our analysis. Whenever possible we highlight the connections to the existing literature. However, our study unveils that such a cut generating function view treating the data associated with each individual variable independently is not possible in the case of general cones other than nonnegative orthant, even when the cone involved is the Lorentz cone

Maximum Persistency in Energy Minimization

We consider discrete pairwise energy minimization problem (weighted constraint satisfaction, max-sum labeling) and methods that identify a globally optimal partial assignment of variables. When finding a complete optimal assignment is intractable, determining optimal values for a part of variables is an interesting possibility. Existing methods are based on different sufficient conditions. We propose a new sufficient condition for partial optimality which is: (1) verifiable in polynomial time (2) invariant to reparametrization of the problem and permutation of labels and (3) includes many existing sufficient conditions as special cases. We pose the problem of finding the maximum optimal partial assignment identifiable by the new sufficient condition. A polynomial method is proposed which is guaranteed to assign same or larger part of variables than several existing approaches. The core of the method is a specially constructed linear program that identifies persistent assignments in an arbitrary multi-label setting.Comment: Extended technical report for the CVPR 2014 paper. Update: correction to the proof of characterization theore