Min Max Generalization for Two-stage Deterministic Batch Mode Reinforcement Learning: Relaxation Schemes

We study the minmax optimization problem introduced in [22] for computing policies for batch mode reinforcement learning in a deterministic setting. First, we show that this problem is NP-hard. In the two-stage case, we provide two relaxation schemes. The first relaxation scheme works by dropping some constraints in order to obtain a problem that is solvable in polynomial time. The second relaxation scheme, based on a Lagrangian relaxation where all constraints are dualized, leads to a conic quadratic programming problem. We also theoretically prove and empirically illustrate that both relaxation schemes provide better results than those given in [22]

A sequential semidefinite programming method and an application in passive reduced-order modeling

We consider the solution of nonlinear programs with nonlinear semidefiniteness constraints. The need for an efficient exploitation of the cone of positive semidefinite matrices makes the solution of such nonlinear semidefinite programs more complicated than the solution of standard nonlinear programs. In particular, a suitable symmetrization procedure needs to be chosen for the linearization of the complementarity condition. The choice of the symmetrization procedure can be shifted in a very natural way to certain linear semidefinite subproblems, and can thus be reduced to a well-studied problem. The resulting sequential semidefinite programming (SSP) method is a generalization of the well-known SQP method for standard nonlinear programs. We present a sensitivity result for nonlinear semidefinite programs, and then based on this result, we give a self-contained proof of local quadratic convergence of the SSP method. We also describe a class of nonlinear semidefinite programs that arise in passive reduced-order modeling, and we report results of some numerical experiments with the SSP method applied to problems in that class

(Global) Optimization: Historical notes and recent developments

Recent developments in (Global) Optimization are surveyed in this paper. We collected and commented quite a large number of recent references which, in our opinion, well represent the vivacity, deepness, and width of scope of current computational approaches and theoretical results about nonconvex optimization problems. Before the presentation of the recent developments, which are subdivided into two parts related to heuristic and exact approaches, respectively, we briefly sketch the origin of the discipline and observe what, from the initial attempts, survived, what was not considered at all as well as a few approaches which have been recently rediscovered, mostly in connection with machine learning

Computational Methods for Discrete Conic Optimization Problems

This thesis addresses computational aspects of discrete conic optimization. Westudy two well-known classes of optimization problems closely related to mixedinteger linear optimization problems. The case of mixed integer second-ordercone optimization problems (MISOCP) is a generalization in which therequirement that solutions be in the non-negative orthant is replaced by arequirement that they be in a second-order cone. Inverse MILP, on the otherhand, is the problem of determining the objective function that makes a givensolution to a given MILP optimal.Although these classes seem unrelated on the surface, the proposedsolution methodology for both classes involves outer approximation of a conicfeasible region by linear inequalities. In both cases, an iterative algorithmin which a separation problem is solved to generate the approximation isemployed. From a complexity standpoint, both MISOCP and inverse MILP areNP--hard. As in the case of MILPs, the usual decision version ofMISOCP is NP-complete, whereas in contrast to MILP, we provide the firstproof that a certain decision version of inverse MILP is rathercoNP-complete.With respect to MISOCP, we first introduce a basic outer approximationalgorithm to solve SOCPs based on a cutting-plane approach. As expected, theperformance of our implementation of such an algorithm is shown to lag behindthe well-known interior point method. Despite this, such a cutting-planeapproach does have promise as a method of producing bounds when embedded withina state-of-the-art branch-and-cut implementation due to its superior ability towarm-start the bound computation after imposing branching constraints. Ourouter-approximation-based branch-and-cut algorithm relaxes both integrality andconic constraints to obtain a linear relaxation. This linear relaxation isstrengthened by the addition of valid inequalities obtained by separatinginfeasible points. Valid inequalities may be obtained by separation from theconvex hull of integer solution lying within the relaxed feasible region or byseparation from the feasible region described by the (relaxed) conicconstraints. Solutions are stored when both integer and conic feasibility isachieved. We review the literature on cutting-plane procedures for MISOCP andmixed integer convex optimization problems.With respect to inverse MILP, we formulate this problem as a conicproblem and derive a cutting-plane algorithm for it. The separation problem inthis algorithm is a modified version of the original MILP. We show that thereis a close relationship between this algorithm and a similar iterativealgorithm for separating infeasible points from the convex hull of solutions tothe original MILP that forms part of the basis for the well-known result ofGrotschel-Lovasz-Schrijver that demonstrates the complexity-wiseequivalence of separation and optimization.In order to test our ideas, we implement a number of software librariesthat together constitute DisCO, a full-featured solver for MISOCP. Thefirst of the supporting libraries is OsiConic, an abstract base classin C++ for interfacing to SOCP solvers. We provide interfaces using thislibrary for widely used commercial and open source SOCP/nonlinear problemsolvers. We also introduce CglConic, a library that implements cuttingprocedures for MISOCP feasible set. We perform extensive computationalexperiments with DisCO comparing a wide range of variants of our proposedalgorithm, as well as other approaches. As DisCO is built on top of a libraryfor distributed parallel tree search algorithms, we also perform experimentsshowing that our algorithm is effective and scalable when parallelized

Mixed-integer Nonlinear Optimization: a hatchery for modern mathematics

The second MFO Oberwolfach Workshop on Mixed-Integer Nonlinear Programming (MINLP) took place between 2nd and 8th June 2019. MINLP refers to one of the hardest Mathematical Programming (MP) problem classes, involving both nonlinear functions as well as continuous and integer decision variables. MP is a formal language for describing optimization problems, and is traditionally part of Operations Research (OR), which is itself at the intersection of mathematics, computer science, engineering and econometrics. The scientific program has covered the three announced areas (hierarchies of approximation, mixed-integer nonlinear optimal control, and dealing with uncertainties) with a variety of tutorials, talks, short research announcements, and a special "open problems'' session

Asymptotic Cones of Quadratically Defined Sets and Their Applications to QCQPs

Quadratically constrained quadratic programs (QCQPs) are a set of optimization problems defined by a quadratic objective function and quadratic constraints. QCQPs cover a diverse set of problems, but the nonconvexity and unboundedness of quadratic constraints lead to difficulties in globally solving a QCQP. This thesis covers properties of unbounded quadratic constraints via a description of the asymptotic cone of a set defined by a single quadratic constraint. A description of the asymptotic cone is provided, including properties such as retractiveness and horizon directions. Using the characterization of the asymptotic cone, we generalize existing results for bounded quadratically defined regions with non-intersecting constraints. The newer result provides a sufficient condition for when the intersection of the lifted convex hulls of quadratically defined sets equals the lifted convex hull of the intersection. This document goes further by expanding the non-intersecting property to cover affine linear constraints. The Frank-Wolfe theorem provides conditions for when a problem defined by a quadratic objective function over affine linear constraints has an optimal solution. Over time, this theorem has been extended to cover cases involving convex quadratic constraints. We discuss more current results through the lens of the asymptotic cone of a quadratically defined set. This discussion expands current results and provides a sufficient condition for when a QCQP with one quadratic constraint with an indefinite Hessian has an optimal solution