Conic Sampling: An Efficient Method for Solving Linear and Quadratic Programming by Randomly Linking Constraints within the Interior

Linear programming (LP) problems are commonly used in analysis and resource allocation, frequently surfacing as approximations to more difficult problems. Existing approaches to LP have been dominated by a small group of methods, and randomized algorithms have not enjoyed popularity in practice. This paper introduces a novel randomized method of solving LP problems by moving along the facets and within the interior of the polytope along rays randomly sampled from the polyhedral cones defined by the bounding constraints. This conic sampling method is then applied to randomly sampled LPs, and its runtime performance is shown to compare favorably to the simplex and primal affine-scaling algorithms, especially on polytopes with certain characteristics. The conic sampling method is then adapted and applied to solve a certain quadratic program, which compute a projection onto a polytope; the proposed method is shown to outperform the proprietary software Mathematica on large, sparse QP problems constructed from mass spectometry-based proteomics

Finding Nash equilibria of bimatrix games

This thesis concerns the computational problem of finding one Nash equilibrium of a bimatrix game, a two-player game in strategic form. Bimatrix games are among the most basic models in non-cooperative game theory, and finding a Nash equilibrium is important for their analysis. The Lemke—Howson algorithm is the classical method for finding one Nash equilib-rium of a bimatrix game. In this thesis, we present a class of square bimatrix games for which this algorithm takes, even in the best case, an exponential number of steps in the dimension d of the game. Using polytope theory, the games are constructed using pairs of dual cyclic polytopes with 2d suitably labelled facets in d-space. The construc-tion is extended to two classes of non-square games where, in addition to exponentially long Lemke—Howson computations, finding an equilibrium by support enumeration takes exponential time on average. The Lemke—Howson algorithm, which is a complementary pivoting algorithm, finds at least one solution to the linear complementarity problem (LCP) derived from a bimatrix game. A closely related complementary pivoting algorithm by Lemke solves more general LCPs. A unified view of these two algorithms is presented, for the first time, as far as we know. Furthermore, we present an extension of the standard version of Lemke's algorithm that allows one more freedom than before when starting the algorithm

Optimization under uncertainty and risk: Quadratic and copositive approaches

Robust optimization and stochastic optimization are the two main paradigms for dealing with the uncertainty inherent in almost all real-world optimization problems. The core principle of robust optimization is the introduction of parameterized families of constraints. Sometimes, these complicated semi-infinite constraints can be reduced to finitely many convex constraints, so that the resulting optimization problem can be solved using standard procedures. Hence flexibility of robust optimization is limited by certain convexity requirements on various objects. However, a recent strain of literature has sought to expand applicability of robust optimization by lifting variables to a properly chosen matrix space. Doing so allows to handle situations where convexity requirements are not met immediately, but rather intermediately. In the domain of (possibly nonconvex) quadratic optimization, the principles of copositive optimization act as a bridge leading to recovery of the desired convex structures. Copositive optimization has established itself as a powerful paradigm for tackling a wide range of quadratically constrained quadratic optimization problems, reformulating them into linear convex-conic optimization problems involving only linear constraints and objective, plus constraints forcing membership to some matrix cones, which can be thought of as generalizations of the positive-semidefinite matrix cone. These reformulations enable application of powerful optimization techniques, most notably convex duality, to problems which, in their original form, are highly nonconvex. In this text we want to offer readers an introduction and tutorial on these principles of copositive optimization, and to provide a review and outlook of the literature that applies these to optimization problems involving uncertainty

Cardinality Constrained Optimization Problems

In this thesis, we examine optimization problems with a constraint that allows for only a certain number of variables to be nonzero. This constraint, which is called a cardinality constraint, has received considerable attention in a number of areas such as machine learning, statistics, computational finance, and operations management. Despite their practical needs, most optimization problems with a cardinality constraints are hard to solve due to their nonconvexity. We focus on constructing tight convex relaxations to such problems

Interior point methods and simulated annealing for nonsymmetric conic optimization

This thesis explores four methods for convex optimization. The first two are an interior point method and a simulated annealing algorithm that share a theoretical foundation. This connection is due to the interior point method’s use of the so-called entropic barrier, whose derivatives can be approximated through sampling. Here, the sampling will be carried out with a technique known as hit-and-run. By carefully analyzing the properties of hit-and-run sampling, it is shown that both the interior point method and the simulated annealing algorithm can solve a convex optimization problem in the membership oracle setting. The number of oracle calls made by these methods is bounded by a polynomial in the input size. The third method is an analytic center cutting plane method that shows promising performance for copositive optimization. It outperforms the first two methods by a significant margin on the problem of separating a matrix from the completely positive cone. The final method is based on Mosek’s algorithm for nonsymmetric conic optimization. With their scaling matrix, search direction, and neighborhood, we define a method that converges to a near-optimal solution in polynomial time

Optimization with mixed-integer, complementarity and bilevel constraints with applications to energy and food markets

In this dissertation, we discuss three classes of nonconvex optimization problems, namely, mixed-integer programming, nonlinear complementarity problems, and mixed-integer bilevel programming. For mixed-integer programming, we identify a class of cutting planes, namely the class of cutting planes derived from lattice-free cross-polytopes, which are proven to provide good approximations to the problem while being efficient to compute. We show that the closure of these cuts gives an approximation that depends only on the ambient dimension and that the cuts can be computed efficiently by explicitly providing an algorithm to compute the cut coefficients in $O(n2^n)$ time, as opposed to solving a nearest lattice-vector problem, which could be much harder. For complementarity problems, we develop a first-order approximation algorithm to efficiently approximate the covariance of the decision in a stochastic complementarity problem. The method can be used to approximate the covariance for large-scale problems by solving a system of linear equations. We also provide bounds to the error incurred in this technique. We then use the technique to analyze policies related to the North American natural gas market. Further, we use this branch of nonconvex problems in the Ethiopian food market to analyze the regional effects of exogenous shocks on the market. We develop a detailed model of the food production, transportation, trade, storage, and consumption in Ethiopia, and test it against exogenous shocks. These shocks are motivated by the prediction that teff, a food grain whose export is banned now, could become a super grain. We present the regional effects of different government policies in response to this shock. For mixed-integer bilevel programming, we develop algorithms that run in polynomial time, provided a subset of the input parameters are fixed. Besides the $\Sigma^p_2$-hardness of the general version of the problem, we show polynomial solvability and $NP$-completeness of certain restricted versions of this problem. Finally, we completely characterize the feasible regions represented by each of these different types of nonconvex optimization problems. We show that the representability of linear complementarity problems, continuous bilevel programs, and polyhedral reverse-convex programs are the same, and they coincide with that of mixed-integer programs if the feasible region is bounded. We also show that the feasible region of any mixed-integer bilevel program is a union of the feasible regions of finitely many mixed-integer programs up to projections and closures