Scalable Semidefinite Relaxation for Maximum A Posterior Estimation

Maximum a posteriori (MAP) inference over discrete Markov random fields is a fundamental task spanning a wide spectrum of real-world applications, which is known to be NP-hard for general graphs. In this paper, we propose a novel semidefinite relaxation formulation (referred to as SDR) to estimate the MAP assignment. Algorithmically, we develop an accelerated variant of the alternating direction method of multipliers (referred to as SDPAD-LR) that can effectively exploit the special structure of the new relaxation. Encouragingly, the proposed procedure allows solving SDR for large-scale problems, e.g., problems on a grid graph comprising hundreds of thousands of variables with multiple states per node. Compared with prior SDP solvers, SDPAD-LR is capable of attaining comparable accuracy while exhibiting remarkably improved scalability, in contrast to the commonly held belief that semidefinite relaxation can only been applied on small-scale MRF problems. We have evaluated the performance of SDR on various benchmark datasets including OPENGM2 and PIC in terms of both the quality of the solutions and computation time. Experimental results demonstrate that for a broad class of problems, SDPAD-LR outperforms state-of-the-art algorithms in producing better MAP assignment in an efficient manner.Comment: accepted to International Conference on Machine Learning (ICML 2014

An infeasible interior-point arc-search method with Nesterov's restarting strategy for linear programming problems

An arc-search interior-point method is a type of interior-point methods that approximates the central path by an ellipsoidal arc, and it can often reduce the number of iterations. In this work, to further reduce the number of iterations and computation time for solving linear programming problems, we propose two arc-search interior-point methods using Nesterov's restarting strategy that is well-known method to accelerate the gradient method with a momentum term. The first one generates a sequence of iterations in the neighborhood, and we prove that the convergence of the generated sequence to an optimal solution and the computation complexity is polynomial time. The second one incorporates the concept of the Mehrotra-type interior-point method to improve numerical performance. The numerical experiments demonstrate that the second one reduced the number of iterations and computational time. In particular, the average number of iterations was reduced compared to existing interior-point methods due to the momentum term.Comment: 33 pages, 6 figures, 2 table

Large-scale Binary Quadratic Optimization Using Semidefinite Relaxation and Applications

In computer vision, many problems such as image segmentation, pixel labelling, and scene parsing can be formulated as binary quadratic programs (BQPs). For submodular problems, cuts based methods can be employed to efficiently solve large-scale problems. However, general nonsubmodular problems are significantly more challenging to solve. Finding a solution when the problem is of large size to be of practical interest, however, typically requires relaxation. Two standard relaxation methods are widely used for solving general BQPs--spectral methods and semidefinite programming (SDP), each with their own advantages and disadvantages. Spectral relaxation is simple and easy to implement, but its bound is loose. Semidefinite relaxation has a tighter bound, but its computational complexity is high, especially for large scale problems. In this work, we present a new SDP formulation for BQPs, with two desirable properties. First, it has a similar relaxation bound to conventional SDP formulations. Second, compared with conventional SDP methods, the new SDP formulation leads to a significantly more efficient and scalable dual optimization approach, which has the same degree of complexity as spectral methods. We then propose two solvers, namely, quasi-Newton and smoothing Newton methods, for the dual problem. Both of them are significantly more efficiently than standard interior-point methods. In practice, the smoothing Newton solver is faster than the quasi-Newton solver for dense or medium-sized problems, while the quasi-Newton solver is preferable for large sparse/structured problems. Our experiments on a few computer vision applications including clustering, image segmentation, co-segmentation and registration show the potential of our SDP formulation for solving large-scale BQPs.Comment: Fixed some typos. 18 pages. Accepted to IEEE Transactions on Pattern Analysis and Machine Intelligenc

Monte Carlo algorithms for the detection of necessary linear matrix inequality constraints

We reduce the size of large semidefinite programming problems by identifying necessary linear matrix inequalities (LMI's) using Monte Carlo techniques. We describe three algorithms for detecting necessary LMI constraints that extend algorithms used in linear programming to semidefinite programming. We demonstrate that they are beneficial and could serve as tools for a semidefinite programming preprocessor. A necessary LMI is one whose removal changes the feasible region defined by all the LMI constraints. The general problem of checking whether or not a particular LMI is necessary is NP-complete. However, the methods we describe are polynomial in each iteration, and the number of iterations can be limited by stopping rules. This provides a practical method for reducing the size of some large Semidefinite Programming problems before one attempts to solve them. We demonstrate the applicability of this approach to solving instances of the Lowner ellipsoid problem. We also consider the problem of classification of all the constraints of a semidefinite programming problem as redundant or necessary

Towards Dual-functional Radar-Communication Systems: Optimal Waveform Design

We focus on a dual-functional multi-input-multi-output (MIMO) radar-communication (RadCom) system, where a single transmitter communicates with downlink cellular users and detects radar targets simultaneously. Several design criteria are considered for minimizing the downlink multi-user interference. First, we consider both the omnidirectional and directional beampattern design problems, where the closed-form globally optimal solutions are obtained. Based on these waveforms, we further consider a weighted optimization to enable a flexible trade-off between radar and communications performance and introduce a low-complexity algorithm. The computational costs of the above three designs are shown to be similar to the conventional zero-forcing (ZF) precoding. Moreover, to address the more practical constant modulus waveform design problem, we propose a branch-and-bound algorithm that obtains a globally optimal solution and derive its worst-case complexity as a function of the maximum iteration number. Finally, we assess the effectiveness of the proposed waveform design approaches by numerical results.Comment: 13 pages, 10 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Euclidean distance geometry and applications

Euclidean distance geometry is the study of Euclidean geometry based on the concept of distance. This is useful in several applications where the input data consists of an incomplete set of distances, and the output is a set of points in Euclidean space that realizes the given distances. We survey some of the theory of Euclidean distance geometry and some of the most important applications: molecular conformation, localization of sensor networks and statics.Comment: 64 pages, 21 figure