A max-flow approach to improved lower bounds for quadratic unconstrained binary optimization (QUBO)

AbstractThe “roof dual” of a QUBO (Quadratic Unconstrained Binary Optimization) problem has been introduced in [P.L. Hammer, P. Hansen, B. Simeone, Roof duality, complementation and persistency in quadratic 0–1 optimization, Mathematical Programming 28 (1984) 121–155]; it provides a bound to the optimum value, along with a polynomial test of the sharpness of this bound, and (due to a “persistency” result) it also determines the values of some of the variables at the optimum. In this paper we provide a graph-theoretic approach to provide bounds, which includes as a special case the roof dual bound, and show that these bounds can be computed in O(n3) time by using network flow techniques. We also obtain a decomposition theorem for quadratic pseudo-Boolean functions, improving the persistency result of [P.L. Hammer, P. Hansen, B. Simeone, Roof duality, complementation and persistency in quadratic 0–1 optimization, Mathematical Programming 28 (1984) 121–155]. Finally, we show that the proposed bounds (including roof duality) can be applied in an iterated way to obtain significantly better bounds. Computational experiments on problems up to thousands of variables are presented

An interior point algorithm for minimum sum-of-squares clustering

Copyright @ 2000 SIAM PublicationsAn exact algorithm is proposed for minimum sum-of-squares nonhierarchical clustering, i.e., for partitioning a given set of points from a Euclidean m-space into a given number of clusters in order to minimize the sum of squared distances from all points to the centroid of the cluster to which they belong. This problem is expressed as a constrained hyperbolic program in 0-1 variables. The resolution method combines an interior point algorithm, i.e., a weighted analytic center column generation method, with branch-and-bound. The auxiliary problem of determining the entering column (i.e., the oracle) is an unconstrained hyperbolic program in 0-1 variables with a quadratic numerator and linear denominator. It is solved through a sequence of unconstrained quadratic programs in 0-1 variables. To accelerate resolution, variable neighborhood search heuristics are used both to get a good initial solution and to solve quickly the auxiliary problem as long as global optimality is not reached. Estimated bounds for the dual variables are deduced from the heuristic solution and used in the resolution process as a trust region. Proved minimum sum-of-squares partitions are determined for the rst time for several fairly large data sets from the literature, including Fisher's 150 iris.This research was supported by the Fonds National de la Recherche Scientifique Suisse, NSERC-Canada, and FCAR-Quebec

Block linear majorants in quadratic 0–1 optimization

AbstractA usual technique to generate upper bounds on the optimum of a quadratic 0–1 maximization problem is to consider a linear majorant (LM) of the quadratic objective function f and then solve the corresponding linear relaxation. Several papers have considered LMs obtained by termwise bounding, but the possibility of bounding groups of terms simultaneously does not appear to have been given much attention so far. In the present paper a broad and flexible computational framework is developed for implementing such a strategy. Here is a brief overview of our approach: in the first place, a suitable collection of “elementary” quadratic functions of few variables (typically, 3 or 4) is generated. All the coefficients of any such function (block) are either 1 or −1, and agree in sign with the corresponding coefficients of the given quadratic function. Next, for each block, a tightest LM (i.e., one having the same value as the block in as many points as possible), or a closest LM (i.e., one minimizing the sum of slacks) is computed. This can be accomplished through the solution of a small mixed-integer program, or a small linear program, respectively. Finally, the objective function is written as a weighted sum of blocks, with non-negative weights. Replacing in this expression each block by the corresponding LM yields an LM of f. We shall choose the weights in this process so that the maximum value of the resulting linear function is as small as possible. This amounts to a large-scale (but still polynomial-size) linear program, which may be solved exactly or, for larger instances, approximately by truncated column generation. The results of a set of 480 numerical tests with up to 200 variables are presented: the upper bounds on the quadratic optimum obtained by the above procedure are (provably) never worse, and often turn out to be substantially sharper, than those resulting from termwise bounding. Moreover, our bounds turn out to be close to the optimum in many (although not all) instances of some well-known benchmarks

Canonical Duality Theory for Global Optimization problems and applications

The canonical duality theory is studied, through a discussion on a general global optimization problem and applications on fundamentally important problems. This general problem is a formulation of the minimization problem with inequality constraints, where the objective function and constraints are any convex or nonconvex functions satisfying certain decomposition conditions. It covers convex problems, mixed integer programming problems and many other nonlinear programming problems. The three main parts of the canonical duality theory are canonical dual transformation, complementary-dual principle and triality theory. The complementary-dual principle is further developed, which conventionally states that each critical point of the canonical dual problem is corresponding to a KKT point of the primal problem with their sharing the same function value. The new result emphasizes that there exists a one-to-one correspondence between KKT points of the dual problem and of the primal problem and each pair of the corresponding KKT points share the same function value, which implies that there is truly no duality gap between the canonical dual problem and the primal problem. The triality theory reveals insightful information about global and local solutions. It is shown that as long as the global optimality condition holds true, the primal problem is equivalent to a convex problem in the dual space, which can be solved efficiently by existing convex methods; even if the condition does not hold, the convex problem still provides a lower bound that is at least as good as that by the Lagrangian relaxation method. It is also shown that through examining the canonical dual problem, the hidden convexity of the primal problem is easily observable. The canonical duality theory is then applied to dealing with three fundamentally important problems. The first one is the spherically constrained quadratic problem, also referred to as the trust region subproblem. The canonical dual problem is onedimensional and it is proved that the primal problem, no matter with convex or nonconvex objective function, is equivalent to a convex problem in the dual space. Moreover, conditions are found which comprise the boundary that separates instances into “hard case” and “easy case”. A canonical primal-dual algorithm is developed, which is able to efficiently solve the problem, including the “hard case”, and can be used as a unified method for similar problems. The second one is the binary quadratic problem, a fundamental problem in discrete optimization. The discussion is focused on lower bounds and analytically solvable cases, which are obtained by analyzing the canonical dual problem with perturbation techniques. The third one is a general nonconvex problem with log-sum-exp functions and quartic polynomials. It arises widely in engineering science and it can be used to approximate nonsmooth optimization problems. The work shows that problems can still be efficiently solved, via the canonical duality approach, even if they are nonconvex and nonsmooth.Doctor of Philosoph

Discrete Optimization in Early Vision - Model Tractability Versus Fidelity

Early vision is the process occurring before any semantic interpretation of an image takes place. Motion estimation, object segmentation and detection are all parts of early vision, but recognition is not. Some models in early vision are easy to perform inference with---they are tractable. Others describe the reality well---they have high fidelity. This thesis improves the tractability-fidelity trade-off of the current state of the art by introducing new discrete methods for image segmentation and other problems of early vision. The first part studies pseudo-boolean optimization, both from a theoretical perspective as well as a practical one by introducing new algorithms. The main result is the generalization of the roof duality concept to polynomials of higher degree than two. Another focus is parallelization; discrete optimization methods for multi-core processors, computer clusters, and graphical processing units are presented. Remaining in an image segmentation context, the second part studies parametric problems where a set of model parameters and a segmentation are estimated simultaneously. For a small number of parameters these problems can still be optimally solved. One application is an optimal method for solving the two-phase Mumford-Shah functional. The third part shifts the focus to curvature regularization---where the commonly used length and area penalization is replaced by curvature in two and three dimensions. These problems can be discretized over a mesh and special attention is given to the mesh geometry. Specifically, hexagonal meshes in the plane are compared to square ones and a method for generating adaptive meshes is introduced and evaluated. The framework is then extended to curvature regularization of surfaces. Finally, the thesis is concluded by three applications to early vision problems: cardiac MRI segmentation, image registration, and cell classification