14 research outputs found
Solving a Class of Cut-Generating Linear Programs via Machine Learning
Cut-generating linear programs (CGLPs) play a key role as a separation oracle
to produce valid inequalities for the feasible region of mixed-integer
programs. When incorporated inside branch-and-bound, the cutting planes
obtained from CGLPs help to tighten relaxations and improve dual bounds.
However, running the CGLPs at the nodes of the branch-and-bound tree is
computationally cumbersome due to the large number of node candidates and the
lack of a priori knowledge on which nodes admit useful cutting planes. As a
result, CGLPs are often avoided at default settings of branch-and-cut
algorithms despite their potential impact on improving dual bounds. In this
paper, we propose a novel framework based on machine learning to approximate
the optimal value of a CGLP class that determines whether a cutting plane can
be generated at a node of the branch-and-bound tree. Translating the CGLP as an
indicator function of the objective function vector, we show that it can be
approximated through conventional data classification techniques. We provide a
systematic procedure to efficiently generate training data sets for the
corresponding classification problem based on the CGLP structure. We conduct
computational experiments on benchmark instances using classification methods
such as logistic regression. These results suggest that the approximate CGLP
obtained from classification can improve the solution time compared to that of
conventional cutting plane methods. Our proposed framework can be efficiently
applied to a large number of nodes in the branch-and-bound tree to identify the
best candidates for adding a cut
On the Structure of Decision Diagram-Representable Mixed Integer Programs with Application to Unit Commitment
Over the past decade, decision diagrams (DDs) have been used to model and
solve integer programming and combinatorial optimization problems. Despite
successful performance of DDs in solving various discrete optimization
problems, their extension to model mixed integer programs (MIPs) such as those
appearing in energy applications has been lacking. More broadly, the question
which problem structures admit a DD representation is still open in the DDs
community. In this paper, we address this question by introducing a geometric
decomposition framework based on rectangular formations that provides both
necessary and sufficient conditions for a general MIP to be representable by
DDs. As a special case, we show that any bounded mixed integer linear program
admits a DD representation through a specialized Benders decomposition
technique. The resulting DD encodes both integer and continuous variables, and
therefore is amenable to the addition of feasibility and optimality cuts
through refinement procedures. As an application for this framework, we develop
a novel solution methodology for the unit commitment problem (UCP) in the
wholesale electricity market. Computational experiments conducted on a
stochastic variant of the UCP show a significant improvement of the solution
time for the proposed method when compared to the outcome of modern solvers
Computational Aspects of Bayesian Solution Estimators in Stochastic Optimization
We study a class of stochastic programs where some of the elements in the objective function are random, and their probability distribution has unknown parameters. The goal is to find a good estimate for the optimal solution of the stochastic program using data sampled from the distribution of the random elements. We investigate two common optimization criteria for evaluating the quality of a solution estimator, one based on the difference in objective values, and the other based on the Euclidean distance between solutions. We use risk as the expected value of such criteria over the sample space. Under a Bayesian framework, where a prior distribution is assumed for the unknown parameters, two natural estimation-optimization strategies arise. A separate scheme first finds an estimator for the unknown parameters, and then uses this estimator in the optimization problem. A joint scheme combines the estimation and optimization steps by directly adjusting the distribution in the stochastic program. We analyze the risk difference between the solutions obtained from these two schemes for several classes of stochastic programs, while providing insight on the computational effort to solve these problems
Solving Cut-Generating Linear Programs via Machine Learning
Cut-generating linear programs (CGLPs) play a key role as a separation oracle to produce valid inequalities for the feasible region of optimization problems. When incorporated inside of branch-and-bound, the cutting planes obtained from CGLPs help to tighten relaxations and improve dual bounds. Running CGLPs at nodes of the branch-and-bound tree, however, is computationally cumbersome due to the large number of node candidates and the lack of a priori knowledge on which nodes admit useful cutting planes. As a result, CGLPs are often avoided at default settings of branch-and-cut algorithms despite their potential impact on improving dual bounds. In this paper, we propose a novel framework based on machine learning to approximate the optimal value of the CGLP, which is the deciding factor in generating cutting planes. Translating the CGLP as an indicator function of the objective function vector, we show that it can be approximated through conventional data classification techniques. We provide a systematic procedure to efficiently generate train data sets for the corresponding classification problem based on the CGLP structure. We conduct computational experiments using classification methods such as logistic regression, support vector machines, and neural networks. Computational results suggest that the outcome of the approximate CGLP obtained from classification achieves a high accuracy rate in a significantly smaller amount of time compared to modern LP solvers. Our pro- posed framework can be efficiently applied to a large number of nodes in the branch-and-bound tree to identify the best candidates for running the CGLP--a feature that can be implemented at the preprocessing phase of any branch-and-cut algorithm to improve solution time and bound quality.This is a pre-print of the article Rajabalizadeh, Atefeh, and Danial Davarnia. "Solving Cut-Generating Linear Programs via Machine Learning." (2021). Posted with permission.</p
Achieving Consistency with Cutting Planes
The primary role of cutting planes is to separate fractional solutions of the linear programming relaxation, which results in tighter bounds for pruning the search tree and reducing its size. Bounding, however, has an indirect impact on the size of the search tree. Cutting planes can also reduce backtracking by excluding inconsistent partial assignments that occur in the course of branching, which directly reduces the tree size. A partial assignment is inconsistent with a constraint set when it cannot be extended to a full feasible assignment. The constraint programming community has studied consistency extensively and used it as an effective tool for the reduction of backtracking. We extend this approach to integer programming by defining concepts of consistency that are useful in a branch-and-bound context. We present a theoretical framework for studying these concepts, their connection with the convex hull and their power to exclude infeasible partial assignments. We introduce a new class of cutting planes that target achieving consistency rather than improving dual bounds. Computational experiments on both synthetic and benchmark instances show that the new class of cutting planes can significantly outperform classical cutting planes, such as disjunctive cuts, by reducing the size of the search tree and the solution time. More broadly, we suggest that consistency concepts offer a new perspective on integer programming that can lead to a better understanding of what makes cutting planes work when used in branch-and-bound search.This is a manuscript of the article Davarnia, Danial, Atefeh Rajabalizadeh, and John Hooker. "Achieving Consistency with Cutting Planes." Mathematical Programming (2022).</p
Computational Aspects of Bayesian Solution Estimators in Stochastic Optimization
We study a class of stochastic programs where some of the elements in the objective function are random, and their probability distribution has unknown parameters. The goal is to find a good estimate for the optimal solution of the stochastic program using data sampled from the distribution of the random elements. We investigate two common optimization criteria for evaluating the quality of a solution estimator, one based on the difference in objective values, and the other based on the Euclidean distance between solutions. We use risk as the expected value of such criteria over the sample space. Under a Bayesian framework, where a prior distribution is assumed for the unknown parameters, two natural estimation-optimization strategies arise. A separate scheme first finds an estimator for the unknown parameters, and then uses this estimator in the optimization problem. A joint scheme combines the estimation and optimization steps by directly adjusting the distribution in the stochastic program. We analyze the risk difference between the solutions obtained from these two schemes for several classes of stochastic programs, while providing insight on the computational effort to solve these problems.This is a manuscript of the article published as Davarnia, Danial, Burak Kocuk, and Gérard Cornuéjols. "Computational Aspects of Bayesian Solution Estimators in Stochastic Optimization." INFORMS Journal on Optimization 2, no. 4 (2020): 256-272. Posted with permission.</p
Convexification of bilinear terms over network polytopes
It is well-known that the McCormick relaxation for the bilinear constraint z=xy gives the convex hull over the box domains for x and y. In network applications where the domain of bilinear variables is described by a network polytope, the McCormick relaxation, also referred to as linearization, fails to provide the convex hull and often leads to poor dual bounds. We study the convex hull of the set containing bilinear constraints zi,j=xiyj where xi represents the arc-flow variable in a network polytope, and yj is in a simplex. For the case where the simplex contains a single y variable, we introduce a systematic procedure to obtain the convex hull of the above set in the original space of variables, and show that all facet-defining inequalities of the convex hull can be obtained explicitly through identifying a special tree structure in the underlying network. For the generalization where the simplex contains multiple y variables, we design a constructive procedure to obtain an important class of facet-defining inequalities for the convex hull of the underlying bilinear set that is characterized by a special forest structure in the underlying network. Computational experiments are presented to evaluate the effectiveness of the proposed methods.This is a pre-print of the article Khademnia, Erfan, and Danial Davarnia. "Convexification of bilinear terms over network polytopes." arXiv preprint arXiv:2302.14151 (2023).
DOI: 10.48550/arXiv.2302.14151.
Attribution 4.0 International (CC BY 4.0).
Copyright 2023 The Authors.
Posted with permission
Consistency for 0-1 programming
Concepts of consistency have long played a key role in constraint programming but never developed in integer programming (IP). Consistency nonetheless plays a role in IP as well. For example, cutting planes can reduce backtracking by achieving various forms of consistency as well as by tightening the linear programming (LP) relaxation. We introduce a type of consistency that is particularly suited for 0–1 programming and develop the associated theory. We define a 0–1 constraint set as LP-consistent when any partial assignment that is consistent with its linear programming relaxation is consistent with the original 0–1 constraint set. We prove basic properties of LP-consistency, including its relationship with Chvátal-Gomory cuts and the integer hull. We show that a weak form of LP-consistency can reduce or eliminate backtracking in a way analogous to k-consistency. This work suggests a new approach to the reduction of backtracking in IP that focuses on cutting off infeasible partial assignments rather than fractional solutions.This is a post-peer-review, pre-copyedit version of a proceeding published as Davarnia, Danial, and John N. Hooker. "Consistency for 0–1 programming." In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pp. 225-240. Springer, Cham, 2019.
The final authenticated version is available online at DOI: 10.1007/978-3-030-19212-9_15.
Copyright 2019 Springer Nature Switzerland AG.
Posted with permission
Solving Unsplittable Network Flow Problems with Decision Diagrams
In unsplittable network flow problems, certain nodes must satisfy a combina-torial requirement that the incoming arc flows cannot be split or merged when routed through outgoing arcs. This so-called no-split no-merge requirement arises in unit train scheduling where train consists should remain intact at stations that lack necessary equipment and manpower to attach/detach them. Solving the unsplittable network flow problems with standard mixed-integer programming formulations is computationally difficult due to the large number of binary variables needed to determine matching pairs between incoming and outgoing arcs of nodes with no-split no-merge constraint. In this paper, we study a stochastic variant of the unit train scheduling problem where the demand is uncertain. We develop a novel decision diagram (DD)-based framework that decomposes the underlying two-stage formulation into a master problem that con-tains the combinatorial requirements and a subproblem that models a continuous net-work flow problem. The master problem is modeled by a DD in a transformed space of variables with a smaller dimension, leading to a substantial improvement in solution time. Similar to the Benders decomposition technique, the subproblems output cutting planes that are used to refine the master DD. Computational experiments show a signif-icant improvement in solution time of the DD framework compared with that of stan-dard methods.This is a manuscript of an article published as Salemi, Hosseinali, and Danial Davarnia. "Solving Unsplittable Network Flow Problems with Decision Diagrams." Transportation Science (2023).
DOI: 10.1287/trsc.2022.1194.
Copyright 2023 INFORMS.
Posted with permission