Exhaustive and Efficient Constraint Propagation: A Semi-Supervised Learning Perspective and Its Applications

This paper presents a novel pairwise constraint propagation approach by decomposing the challenging constraint propagation problem into a set of independent semi-supervised learning subproblems which can be solved in quadratic time using label propagation based on k-nearest neighbor graphs. Considering that this time cost is proportional to the number of all possible pairwise constraints, our approach actually provides an efficient solution for exhaustively propagating pairwise constraints throughout the entire dataset. The resulting exhaustive set of propagated pairwise constraints are further used to adjust the similarity matrix for constrained spectral clustering. Other than the traditional constraint propagation on single-source data, our approach is also extended to more challenging constraint propagation on multi-source data where each pairwise constraint is defined over a pair of data points from different sources. This multi-source constraint propagation has an important application to cross-modal multimedia retrieval. Extensive results have shown the superior performance of our approach.Comment: The short version of this paper appears as oral paper in ECCV 201

An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems

We propose a new fast algorithm for solving one of the standard approaches to ill-posed linear inverse problems (IPLIP), where a (possibly non-smooth) regularizer is minimized under the constraint that the solution explains the observations sufficiently well. Although the regularizer and constraint are usually convex, several particular features of these problems (huge dimensionality, non-smoothness) preclude the use of off-the-shelf optimization tools and have stimulated a considerable amount of research. In this paper, we propose a new efficient algorithm to handle one class of constrained problems (often known as basis pursuit denoising) tailored to image recovery applications. The proposed algorithm, which belongs to the family of augmented Lagrangian methods, can be used to deal with a variety of imaging IPLIP, including deconvolution and reconstruction from compressive observations (such as MRI), using either total-variation or wavelet-based (or, more generally, frame-based) regularization. The proposed algorithm is an instance of the so-called "alternating direction method of multipliers", for which convergence sufficient conditions are known; we show that these conditions are satisfied by the proposed algorithm. Experiments on a set of image restoration and reconstruction benchmark problems show that the proposed algorithm is a strong contender for the state-of-the-art.Comment: 13 pages, 8 figure, 8 tables. Submitted to the IEEE Transactions on Image Processin

Conditional Gradient Algorithms for Rank-One Matrix Approximations with a Sparsity Constraint

The sparsity constrained rank-one matrix approximation problem is a difficult mathematical optimization problem which arises in a wide array of useful applications in engineering, machine learning and statistics, and the design of algorithms for this problem has attracted intensive research activities. We introduce an algorithmic framework, called ConGradU, that unifies a variety of seemingly different algorithms that have been derived from disparate approaches, and allows for deriving new schemes. Building on the old and well-known conditional gradient algorithm, ConGradU is a simplified version with unit step size and yields a generic algorithm which either is given by an analytic formula or requires a very low computational complexity. Mathematical properties are systematically developed and numerical experiments are given.Comment: Minor changes. Final version. To appear in SIAM Revie

Resource dedication problem in a multi-project environment

There can be different approaches to the management of resources within the context of multi-project scheduling problems. In general, approaches to multiproject scheduling problems consider the resources as a pool shared by all projects. On the other hand, when projects are distributed geographically or sharing resources between projects is not preferred, then this resource sharing policy may not be feasible. In such cases, the resources must be dedicated to individual projects throughout the project durations. This multi-project problem environment is defined here as the resource dedication problem (RDP). RDP is defined as the optimal dedication of resource capacities to different projects within the overall limits of the resources and with the objective of minimizing a predetermined objective function. The projects involved are multi-mode resource constrained project scheduling problems with finish to start zero time lag and non-preemptive activities and limited renewable and nonrenewable resources. Here, the characterization of RDP, its mathematical formulation and two different solution methodologies are presented. The first solution approach is a genetic algorithm employing a new improvement move called combinatorial auction for RDP, which is based on preferences of projects for resources. Two different methods for calculating the projects’ preferences based on linear and Lagrangian relaxation are proposed. The second solution approach is a Lagrangian relaxation based heuristic employing subgradient optimization. Numerical studies demonstrate that the proposed approaches are powerful methods for solving this problem

Synchronization-Aware and Algorithm-Efficient Chance Constrained Optimal Power Flow

One of the most common control decisions faced by power system operators is the question of how to dispatch generation to meet demand for power. This is a complex optimization problem that includes many nonlinear, non convex constraints as well as inherent uncertainties about future demand for power and available generation. In this paper we develop convex formulations to appropriately model crucial classes of nonlinearities and stochastic effects. We focus on solving a nonlinear optimal power flow (OPF) problem that includes loss of synchrony constraints and models wind-farm caused fluctuations. In particular, we develop (a) a convex formulation of the deterministic phase-difference nonlinear Optimum Power Flow (OPF) problem; and (b) a probabilistic chance constrained OPF for angular stability, thermal overloads and generation limits that is computationally tractable.Comment: 11 pages, 3 figure