On the facet-to-facet property of solutions to convex parametric quadratic programs

In some of the recently developed algorithms for convex parametric quadratic programs it is implicitly assumed that the intersection of the closures of two adjacent critical regions is a facet of both closures; this will be referred to as the facet-to-facet property. It is shown by an example, whose solution is unique, that the facet-to-facet property does not hold in general. Consequently, some existing algorithms cannot guarantee that the entire parameter space will be explored. A simple modification, applicable to several existing algorithms, is presented for the purpose of overcoming this problem. Numerical results indicate that, compared to the original algorithms for parametric quadratic programs, the proposed method has lower computational complexity for problems whose solutions consist of a large number of critical regions

An Exponential Lower Bound on the Complexity of Regularization Paths

For a variety of regularized optimization problems in machine learning, algorithms computing the entire solution path have been developed recently. Most of these methods are quadratic programs that are parameterized by a single parameter, as for example the Support Vector Machine (SVM). Solution path algorithms do not only compute the solution for one particular value of the regularization parameter but the entire path of solutions, making the selection of an optimal parameter much easier. It has been assumed that these piecewise linear solution paths have only linear complexity, i.e. linearly many bends. We prove that for the support vector machine this complexity can be exponential in the number of training points in the worst case. More strongly, we construct a single instance of n input points in d dimensions for an SVM such that at least \Theta(2^{n/2}) = \Theta(2^d) many distinct subsets of support vectors occur as the regularization parameter changes.Comment: Journal version, 28 Pages, 5 Figure

An improved multi-parametric programming algorithm for flux balance analysis of metabolic networks

Flux balance analysis has proven an effective tool for analyzing metabolic networks. In flux balance analysis, reaction rates and optimal pathways are ascertained by solving a linear program, in which the growth rate is maximized subject to mass-balance constraints. A variety of cell functions in response to environmental stimuli can be quantified using flux balance analysis by parameterizing the linear program with respect to extracellular conditions. However, for most large, genome-scale metabolic networks of practical interest, the resulting parametric problem has multiple and highly degenerate optimal solutions, which are computationally challenging to handle. An improved multi-parametric programming algorithm based on active-set methods is introduced in this paper to overcome these computational difficulties. Degeneracy and multiplicity are handled, respectively, by introducing generalized inverses and auxiliary objective functions into the formulation of the optimality conditions. These improvements are especially effective for metabolic networks because their stoichiometry matrices are generally sparse; thus, fast and efficient algorithms from sparse linear algebra can be leveraged to compute generalized inverses and null-space bases. We illustrate the application of our algorithm to flux balance analysis of metabolic networks by studying a reduced metabolic model of Corynebacterium glutamicum and a genome-scale model of Escherichia coli. We then demonstrate how the critical regions resulting from these studies can be associated with optimal metabolic modes and discuss the physical relevance of optimal pathways arising from various auxiliary objective functions. Achieving more than five-fold improvement in computational speed over existing multi-parametric programming tools, the proposed algorithm proves promising in handling genome-scale metabolic models.Comment: Accepted in J. Optim. Theory Appl. First draft was submitted on August 4th, 201

Solution Techniques for Classes of Biobjective and Parametric Programs

Mathematical optimization, or mathematical programming, has been studied for several decades. Researchers are constantly searching for optimization techniques which allow one to de-termine the ideal course of action in extremely complex situations. This line of scientific inquiry motivates the primary focus of this dissertation — nontraditional optimization problems having either multiple objective functions or parametric input. Utilizing multiple objective functions al-lows one to account for the fact that the decision process in many real-life problems in engineering, business, and management is often driven by several conflicting criteria such as cost, performance, reliability, safety, and productivity. Additionally, incorporating parametric input allows one to ac-count for uncertainty in models’ data, which can arise for a number of reasons, including a changing availability of resources, estimation or measurement errors, or implementation errors caused by stor-ing data in a fixed precision format. However, when a decision problem has either parametric input or multiple objectives, one cannot hope to find a single, satisfactory solution. Thus, in this work we develop techniques which can be used to determine sets of desirable solutions. The two main problems we consider in this work are the biobjective mixed integer linear program (BOMILP) and the multiparametric linear complementarity problem (mpLCP). BOMILPs are optimization problems in which two linear objectives are optimized over a polyhedron while restricting some of the decision variables to be integer. We present a new data structure in the form of a modified binary tree that can be used to store the solution set of BOMILP. Empirical evidence is provided showing that this structure is able to store these solution sets more efficiently than other data structures that are typically used for this purpose. We also develop a branch-and-bound (BB) procedure that can be used to compute the solution set of BOMILP. Computational experiments are conducted in order to compare the performance of the new BB procedure with current state-of-the-art methods for determining the solution set of BOMILP. The results provide strong evidence of the utility of the proposed BB method. We also present new procedures for solving two variants of the mpLCP. Each of these proce-dures consists of two phases. In the first phase an initial feasible solution to mpLCP which satisfies certain criteria is determined. This contribution alone is significant because the question of how such an initial solution could be generated was previously unanswered. In the second phase the set of fea-sible parameters is partitioned into regions such that the solution of the mpLCP, as a function of the parameters, is invariant over each region. For the first variant of mpLCP, the worst-case complex-ity of the presented procedure matches that of current state-of-the-art methods for nondegenerate problems and is lower than that of current state-of-the-art methods for degenerate problems. Addi-tionally, computational results show that the proposed procedure significantly outperforms current state-of-the-art methods in practice. The second variant of mpLCP we consider was previously un-solved. In order to develop a solution strategy, we first study the structure of the problem in detail. This study relies on the integration of several key concepts from algebraic geometry and topology into the field of operations research. Using these tools we build the theoretical foundation necessary to solve the mpLCP and propose a strategy for doing so. Experimental results indicate that the presented solution method also performs well in practice

Conjectures on an algorithm for convex parametric quadratic programs

An algorithm for convex parametric QPs is studied. The algorithm explores the parameter space by stepping a sufficiently small distance over the facets of each critical region and thereby identifying the neighboring regions. Some conjectures concerning this algorithm and the structure of the solution of a parametric QP are presented

Lexicographic perturbation for multiparametric linear programming with applications to control

Accepted versio

Regularized Optimal Transport and the Rot Mover's Distance

This paper presents a unified framework for smooth convex regularization of discrete optimal transport problems. In this context, the regularized optimal transport turns out to be equivalent to a matrix nearness problem with respect to Bregman divergences. Our framework thus naturally generalizes a previously proposed regularization based on the Boltzmann-Shannon entropy related to the Kullback-Leibler divergence, and solved with the Sinkhorn-Knopp algorithm. We call the regularized optimal transport distance the rot mover's distance in reference to the classical earth mover's distance. We develop two generic schemes that we respectively call the alternate scaling algorithm and the non-negative alternate scaling algorithm, to compute efficiently the regularized optimal plans depending on whether the domain of the regularizer lies within the non-negative orthant or not. These schemes are based on Dykstra's algorithm with alternate Bregman projections, and further exploit the Newton-Raphson method when applied to separable divergences. We enhance the separable case with a sparse extension to deal with high data dimensions. We also instantiate our proposed framework and discuss the inherent specificities for well-known regularizers and statistical divergences in the machine learning and information geometry communities. Finally, we demonstrate the merits of our methods with experiments using synthetic data to illustrate the effect of different regularizers and penalties on the solutions, as well as real-world data for a pattern recognition application to audio scene classification

Enumeration of PLCP-orientations of the 4-cube

The linear complementarity problem (LCP) provides a unified approach to many problems such as linear programs, convex quadratic programs, and bimatrix games. The general LCP is known to be NP-hard, but there are some promising results that suggest the possibility that the LCP with a P-matrix (PLCP) may be polynomial-time solvable. However, no polynomial-time algorithm for the PLCP has been found yet and the computational complexity of the PLCP remains open. Simple principal pivoting (SPP) algorithms, also known as Bard-type algorithms, are candidates for polynomial-time algorithms for the PLCP. In 1978, Stickney and Watson interpreted SPP algorithms as a family of algorithms that seek the sink of unique-sink orientations of $n$-cubes. They performed the enumeration of the arising orientations of the $3$-cube, hereafter called PLCP-orientations. In this paper, we present the enumeration of PLCP-orientations of the $4$-cube.The enumeration is done via construction of oriented matroids generalizing P-matrices and realizability classification of oriented matroids.Some insights obtained in the computational experiments are presented as well