Image labeling and grouping by minimizing linear functionals over cones

We consider energy minimization problems related to image labeling, partitioning, and grouping, which typically show up at mid-level stages of computer vision systems. A common feature of these problems is their intrinsic combinatorial complexity from an optimization pointof-view. Rather than trying to compute the global minimum - a goal we consider as elusive in these cases - we wish to design optimization approaches which exhibit two relevant properties: First, in each application a solution with guaranteed degree of suboptimality can be computed. Secondly, the computations are based on clearly defined algorithms which do not comprise any (hidden) tuning parameters. In this paper, we focus on the second property and introduce a novel and general optimization technique to the field of computer vision which amounts to compute a sub optimal solution by just solving a convex optimization problem. As representative examples, we consider two binary quadratic energy functionals related to image labeling and perceptual grouping. Both problems can be considered as instances of a general quadratic functional in binary variables, which is embedded into a higher-dimensional space such that sub optimal solutions can be computed as minima of linear functionals over cones in that space (semidefinite programs). Extensive numerical results reveal that, on the average, sub optimal solutions can be computed which yield a gap below 5% with respect to the global optimum in case where this is known

Image Partitioning based on Semidefinite Programming

Many tasks in computer vision lead to combinatorial optimization problems. Automatic image partitioning is one of the most important examples in this context: whether based on some prior knowledge or completely unsupervised, we wish to find coherent parts of the image. However, the inherent combinatorial complexity of such problems often prevents to find the global optimum in polynomial time. For this reason, various approaches have been proposed to find good approximative solutions for image partitioning problems. As an important example, we will first consider different spectral relaxation techniques: based on straightforward eigenvector calculations, these methods compute suboptimal solutions in short time. However, the main contribution of this thesis is to introduce a novel optimization technique for discrete image partitioning problems which is based on a semidefinite programming relaxation. In contrast to approximation methods employing annealing algorithms, this approach involves solving a convex optimization problem, which does not suffer from possible local minima. Using interior point techniques, the solution of the relaxation can be found in polynomial time, and without elaborate parameter tuning. High quality solutions to the original combinatorial problem are then obtained with a randomized rounding technique. The only potential drawback of the semidefinite relaxation approach is that the number of variables of the optimization problem is squared. Nevertheless, it can still be applied to problems with up to a few thousand variables, as is demonstrated for various computer vision tasks including unsupervised segmentation, perceptual grouping and image restoration. Concerning problems of higher dimensionality, we study two different approaches to effectively reduce the number of variables. The first one is based on probabilistic sampling: by considering only a small random fraction of the pixels in the image, our semidefinite relaxation method can be applied in an efficient way while maintaining a reliable quality of the resulting segmentations. The second approach reduces the problem size by computing an over-segmentation of the image in a preprocessing step. After that, the image is partitioned based on the resulting "superpixels" instead of the original pixels. Since the real world does not consist of pixels, it can even be argued that this is the more natural image representation. Initially, our semidefinite relaxation method is defined only for binary partitioning problems. To derive image segmentations into multiple parts, one possibility is to apply the binary approach in a hierarchical way. Besides this natural extension, we also discuss how multiclass partitioning problems can be solved in a direct way based on semidefinite relaxation techniques

A signomial programming approach for binary image restoration by penalized least squares

The authors present a novel optimization approach, using signomial programming (SP), to restore noise-corrupted binary and grayscale images. The approach requires the minimization of a penalized least squares functional over binary variables, which has led to the design of various approximation methods in the past. In this brief, we minimize the functional as a SP problem which is then converted into a reversed geometric programming (GP) problem and solved using standard GP solvers. Numerical experiments show that the proposed approach restores both degraded binary and grayscale images with good accuracy, and is over 20 times faster than the positive semidefinite programming approach. © 2007 IEEE.published_or_final_versio

Efficient graph cuts for unsupervised image segmentation using probabilistic sampling and SVD-based approximation

The application of graph theoretic methods to unsupervised image partitioning has been a very active field of research recently. For weighted graphs encoding the (dis)similarity structure of locally extracted image features, unsupervised segmentations of images into coherent structures can be computed in terms of extremal cuts of the underlying graphs. In this context, we focus on the normalized cut criterion and a related recent convex approach based on semidefinite programming. As both methods soon become computationally demanding with increasing graph size, an important question is how the computations can be accelerated. To this end, we study an SVD approximation method in this paper which has been introduced in a different clustering context. We apply this method, which is based on probabilistic sampling, to both segmentation approaches and compare it with the Nyström extension suggested for the normalized cut. Numerical results confirm that by means of the sampling-based SVD approximation technique, reliable segmentations can be computed with a fraction (less than 5%) of the original computational cost

Large-scale Binary Quadratic Optimization Using Semidefinite Relaxation and Applications

In computer vision, many problems such as image segmentation, pixel labelling, and scene parsing can be formulated as binary quadratic programs (BQPs). For submodular problems, cuts based methods can be employed to efficiently solve large-scale problems. However, general nonsubmodular problems are significantly more challenging to solve. Finding a solution when the problem is of large size to be of practical interest, however, typically requires relaxation. Two standard relaxation methods are widely used for solving general BQPs--spectral methods and semidefinite programming (SDP), each with their own advantages and disadvantages. Spectral relaxation is simple and easy to implement, but its bound is loose. Semidefinite relaxation has a tighter bound, but its computational complexity is high, especially for large scale problems. In this work, we present a new SDP formulation for BQPs, with two desirable properties. First, it has a similar relaxation bound to conventional SDP formulations. Second, compared with conventional SDP methods, the new SDP formulation leads to a significantly more efficient and scalable dual optimization approach, which has the same degree of complexity as spectral methods. We then propose two solvers, namely, quasi-Newton and smoothing Newton methods, for the dual problem. Both of them are significantly more efficiently than standard interior-point methods. In practice, the smoothing Newton solver is faster than the quasi-Newton solver for dense or medium-sized problems, while the quasi-Newton solver is preferable for large sparse/structured problems. Our experiments on a few computer vision applications including clustering, image segmentation, co-segmentation and registration show the potential of our SDP formulation for solving large-scale BQPs.Comment: Fixed some typos. 18 pages. Accepted to IEEE Transactions on Pattern Analysis and Machine Intelligenc

Semidefinite Relaxations for Robust Multiview Triangulation

We propose an approach based on convex relaxations for certifiably optimal robust multiview triangulation. To this end, we extend existing relaxation approaches to non-robust multiview triangulation by incorporating a least squares cost function. We propose two formulations, one based on epipolar constraints and one based on fractional reprojection constraints. The first is lower dimensional and remains tight under moderate noise and outlier levels, while the second is higher dimensional and therefore slower but remains tight even under extreme noise and outlier levels. We demonstrate through extensive experiments that the proposed approaches allow us to compute provably optimal reconstructions even under significant noise and a large percentage of outliers

Using the Eigenvalue Relaxation for Binary Least-Squares Estimation Problems

The goal of this paper is to survey the properties of the eigenvalue relaxation for least squares binary problems. This relaxation is a convex program which is obtained as the Lagrangian dual of the original problem with an implicit compact constraint and as such, is a convex problem with polynomial time complexity. Moreover, as a main pratical advantage of this relaxation over the standard Semi-Definite Programming approach, several efficient bundle methods are available for this problem allowing to address problems of very large dimension. The necessary tools from convex analysis are recalled and shown at work for handling the problem of exactness of this relaxation. Two applications are described. The first one is the problem of binary image reconstruction and the second is the problem of multiuser detection in CDMA systems