Introducing Interior-Point Methods for Introductory Operations Research Courses and/or Linear Programming Courses

In recent years the introduction and development of Interior-Point Methods has had a profound impact on optimization theory as well as practice, influencing the field of Operations Research and related areas. Development of these methods has quickly led to the design of new and efficient optimization codes particularly for Linear Programming. Consequently, there has been an increasing need to introduce theory and methods of this new area in optimization into the appropriate undergraduate and first year graduate courses such as introductory Operations Research and/or Linear Programming courses, Industrial Engineering courses and Math Modeling courses. The objective of this paper is to discuss the ways of simplifying the introduction of Interior-Point Methods for students who have various backgrounds or who are not necessarily mathematics majors

Positive Semidefinite Metric Learning with Boosting

The learning of appropriate distance metrics is a critical problem in image classification and retrieval. In this work, we propose a boosting-based technique, termed \BoostMetric, for learning a Mahalanobis distance metric. One of the primary difficulties in learning such a metric is to ensure that the Mahalanobis matrix remains positive semidefinite. Semidefinite programming is sometimes used to enforce this constraint, but does not scale well. \BoostMetric is instead based on a key observation that any positive semidefinite matrix can be decomposed into a linear positive combination of trace-one rank-one matrices. \BoostMetric thus uses rank-one positive semidefinite matrices as weak learners within an efficient and scalable boosting-based learning process. The resulting method is easy to implement, does not require tuning, and can accommodate various types of constraints. Experiments on various datasets show that the proposed algorithm compares favorably to those state-of-the-art methods in terms of classification accuracy and running time.Comment: 11 pages, Twenty-Third Annual Conference on Neural Information Processing Systems (NIPS 2009), Vancouver, Canad

A primal-dual decomposition based interior point approach to two-stage stochastic linear programming

Decision making under uncertainty is a challenge faced by many decision makers. Stochastic programming is a major tool developed to deal with optimization with uncertainties that has found applications in, e.g. finance, such as asset-liability and bond-portfolio management. Computationally however, many models in stochastic programming remain unsolvable because of overwhelming dimensionality. For a model to be well solvable, its special structure must be explored. Most of the solution methods are based on decomposing the data. In this paper we propose a new decomposition approach for two-stage stochastic programming, based on a direct application of the path-following method combined with the homogeneous self-dual technique. Numerical experiments show that our decomposition algorithm is very efficient for solving stochastic programs. In particular, we apply our deompostition method to a two-period portfolio selection problem using options on a stock index. In this model the investor can invest in a money-market account, a stock index, and European options on this index with different maturities. We experiment our model with market prices of options on the S&P500

Facets for Art Gallery Problems

The Art Gallery Problem (AGP) asks for placing a minimum number of stationary guards in a polygonal region P, such that all points in P are guarded. The problem is known to be NP-hard, and its inherent continuous structure (with both the set of points that need to be guarded and the set of points that can be used for guarding being uncountably infinite) makes it difficult to apply a straightforward formulation as an Integer Linear Program. We use an iterative primal-dual relaxation approach for solving AGP instances to optimality. At each stage, a pair of LP relaxations for a finite candidate subset of primal covering and dual packing constraints and variables is considered; these correspond to possible guard positions and points that are to be guarded. Particularly useful are cutting planes for eliminating fractional solutions. We identify two classes of facets, based on Edge Cover and Set Cover (SC) inequalities. Solving the separation problem for the latter is NP-complete, but exploiting the underlying geometric structure, we show that large subclasses of fractional SC solutions cannot occur for the AGP. This allows us to separate the relevant subset of facets in polynomial time. We also characterize all facets for finite AGP relaxations with coefficients in {0, 1, 2}. Finally, we demonstrate the practical usefulness of our approach. Our cutting plane technique yields a significant improvement in terms of speed and solution quality due to considerably reduced integrality gaps as compared to the approach by Kr\"oller et al.Comment: 29 pages, 18 figures, 1 tabl

Combinatorial Continuous Maximal Flows

Maximum flow (and minimum cut) algorithms have had a strong impact on computer vision. In particular, graph cuts algorithms provide a mechanism for the discrete optimization of an energy functional which has been used in a variety of applications such as image segmentation, stereo, image stitching and texture synthesis. Algorithms based on the classical formulation of max-flow defined on a graph are known to exhibit metrication artefacts in the solution. Therefore, a recent trend has been to instead employ a spatially continuous maximum flow (or the dual min-cut problem) in these same applications to produce solutions with no metrication errors. However, known fast continuous max-flow algorithms have no stopping criteria or have not been proved to converge. In this work, we revisit the continuous max-flow problem and show that the analogous discrete formulation is different from the classical max-flow problem. We then apply an appropriate combinatorial optimization technique to this combinatorial continuous max-flow CCMF problem to find a null-divergence solution that exhibits no metrication artefacts and may be solved exactly by a fast, efficient algorithm with provable convergence. Finally, by exhibiting the dual problem of our CCMF formulation, we clarify the fact, already proved by Nozawa in the continuous setting, that the max-flow and the total variation problems are not always equivalent.Comment: 26 page

Positive Semidefinite Metric Learning Using Boosting-like Algorithms

The success of many machine learning and pattern recognition methods relies heavily upon the identification of an appropriate distance metric on the input data. It is often beneficial to learn such a metric from the input training data, instead of using a default one such as the Euclidean distance. In this work, we propose a boosting-based technique, termed BoostMetric, for learning a quadratic Mahalanobis distance metric. Learning a valid Mahalanobis distance metric requires enforcing the constraint that the matrix parameter to the metric remains positive definite. Semidefinite programming is often used to enforce this constraint, but does not scale well and easy to implement. BoostMetric is instead based on the observation that any positive semidefinite matrix can be decomposed into a linear combination of trace-one rank-one matrices. BoostMetric thus uses rank-one positive semidefinite matrices as weak learners within an efficient and scalable boosting-based learning process. The resulting methods are easy to implement, efficient, and can accommodate various types of constraints. We extend traditional boosting algorithms in that its weak learner is a positive semidefinite matrix with trace and rank being one rather than a classifier or regressor. Experiments on various datasets demonstrate that the proposed algorithms compare favorably to those state-of-the-art methods in terms of classification accuracy and running time.Comment: 30 pages, appearing in Journal of Machine Learning Researc