Computationally efficient solution of mixed integer model predictive control problems via machine learning aided Benders Decomposition

Mixed integer Model Predictive Control (MPC) problems arise in the operation of systems where discrete and continuous decisions must be taken simultaneously to compensate for disturbances. The efficient solution of mixed integer MPC problems requires the computationally efficient and robust online solution of mixed integer optimization problems, which are generally difficult to solve. In this paper, we propose a machine learning-based branch and check Generalized Benders Decomposition algorithm for the efficient solution of such problems. We use machine learning to approximate the effect of the complicating variables on the subproblem by approximating the Benders cuts without solving the subproblem, therefore, alleviating the need to solve the subproblem multiple times. The proposed approach is applied to a mixed integer economic MPC case study on the operation of chemical processes. We show that the proposed algorithm always finds feasible solutions to the optimization problem, given that the mixed integer MPC problem is feasible, and leads to a significant reduction in solution time (up to 97% or 50x) while incurring small error (in the order of 1%) compared to the application of standard and accelerated Generalized Benders Decomposition

A Stochastic Benders Decomposition Scheme for Large-Scale Data-Driven Network Design

Network design problems involve constructing edges in a transportation or supply chain network to minimize construction and daily operational costs. We study a data-driven version of network design where operational costs are uncertain and estimated using historical data. This problem is notoriously computationally challenging, and instances with as few as fifty nodes cannot be solved to optimality by current decomposition techniques. Accordingly, we propose a stochastic variant of Benders decomposition that mitigates the high computational cost of generating each cut by sampling a subset of the data at each iteration and nonetheless generates deterministically valid cuts (as opposed to the probabilistically valid cuts frequently proposed in the stochastic optimization literature) via a dual averaging technique. We implement both single-cut and multi-cut variants of this Benders decomposition algorithm, as well as a k-cut variant that uses clustering of the historical scenarios. On instances with 100-200 nodes, our algorithm achieves 4-5% optimality gaps, compared with 13-16% for deterministic Benders schemes, and scales to instances with 700 nodes and 50 commodities within hours. Beyond network design, our strategy could be adapted to generic two-stage stochastic mixed-integer optimization problems where second-stage costs are estimated via a sample average

Approximate and exact convexification approaches for solving two-stage mixed-integer recourse models

Many practical decision-making problems are subject to uncertainty. A powerful class of mathematical models designed for these problems is the class of mixed-integer recourse models. Such models have a wide range of applications in, e.g., healthcare, energy, and finance. They permit integer decision variables to accurately model, e.g., on/off restrictions or natural indivisibilities. The additional modelling flexibility of integer decision variables, however, comes at the expense of models that are significantly harder to solve. The reason is that including integer decision variables introduces non-convexity in the model, which poses a significant challenge for state-of-the-art solvers.In this thesis, we contribute to better decision making under uncertainty by designing efficient solution methods for mixed-integer recourse models. Our approach is to address the non-convexity caused by integer decision variables by using convexification. That is, we construct convex approximating models that closely approximate the original model. In addition, we derive performance guarantees for the solution obtained by solving the approximating model. Finally, we extensively test the solution methods that we propose and we find that they consistently outperform traditional solution methods on a wide range of benchmark instances

Developing an Enhanced Algorithms to Solve Mixed Integer Non-Linear Programming Problems Based on a Feasible Neighborhood Search Strategy

Engineering optimization problems often involve nonlinear objective functions, which can capture complex relationships and dependencies between variables. This study focuses on a unique nonlinear mathematics programming problem characterized by a subset of variables that can only take discrete values and are linearly separable from the continuous variables. The combination of integer variables and non-linearities makes this problem much more complex than traditional nonlinear programming problems with only continuous variables. Furthermore, the presence of integer variables can result in a combinatorial explosion of potential solutions, significantly enlarging the search space and making it challenging to explore effectively. This issue becomes especially challenging for larger problems, leading to long computation times or even infeasibility. To address these challenges, we propose a method that employs the "active constraint" approach in conjunction with the release of nonbasic variables from their boundaries. This technique compels suitable non-integer fundamental variables to migrate to their neighboring integer positions. Additionally, we have researched selection criteria for choosing a nonbasic variable to use in the integerizing technique. Through implementation and testing on various problems, these techniques have proven to be successful

Branching strategies for mixed-integer programs containing logical constraints and decomposable structure

Decision-making optimisation problems can include discrete selections, e.g. selecting a route, arranging non-overlapping items or designing a network of items. Branch-and-bound (B&B), a widely applied divide-and-conquer framework, often solves such problems by considering a continuous approximation, e.g. replacing discrete variable domains by a continuous superset. Such approximations weaken the logical relations, e.g. for discrete variables corresponding to Boolean variables. Branching in B&B reintroduces logical relations by dividing the search space. This thesis studies designing B&B branching strategies, i.e. how to divide the search space, for optimisation problems that contain both a logical and a continuous structure. We begin our study with a large-scale, industrially-relevant optimisation problem where the objective consists of machine-learnt gradient-boosted trees (GBTs) and convex penalty functions. GBT functions contain if-then queries which introduces a logical structure to this problem. We propose decomposition-based rigorous bounding strategies and an iterative heuristic that can be embedded into a B&B algorithm. We approach branching with two strategies: a pseudocost initialisation and strong branching that target the structure of GBT and convex penalty aspects of the optimisation objective, respectively. Computational tests show that our B&B approach outperforms state-of-the-art solvers in deriving rigorous bounds on optimality. Our second project investigates how satisfiability modulo theories (SMT) derived unsatisfiable cores may be utilised in a B&B context. Unsatisfiable cores are subsets of constraints that explain an infeasible result. We study two-dimensional bin packing (2BP) and develop a B&B algorithm that branches on SMT unsatisfiable cores. We use the unsatisfiable cores to derive cuts that break 2BP symmetries. Computational results show that our B&B algorithm solves 20% more instances when compared with commercial solvers on the tested instances. Finally, we study convex generalized disjunctive programming (GDP), a framework that supports logical variables and operators. Convex GDP includes disjunctions of mathematical constraints, which motivate branching by partitioning the disjunctions. We investigate separation by branching, i.e. eliminating solutions that prevent rigorous bound improvement, and propose a greedy algorithm for building the branches. We propose three scoring methods for selecting the next branching disjunction. We also analyse how to leverage infeasibility to expedite the B&B search. Computational results show that our scoring methods can reduce the number of explored B&B nodes by an order of magnitude when compared with scoring methods proposed in literature. Our infeasibility analysis further reduces the number of explored nodes.Open Acces

Support Vector Machines with the Hard-Margin Loss: Optimal Training via Combinatorial Benders' Cuts

The classical hinge-loss support vector machines (SVMs) model is sensitive to outlier observations due to the unboundedness of its loss function. To circumvent this issue, recent studies have focused on non-convex loss functions, such as the hard-margin loss, which associates a constant penalty to any misclassified or within-margin sample. Applying this loss function yields much-needed robustness for critical applications but it also leads to an NP-hard model that makes training difficult, since current exact optimization algorithms show limited scalability, whereas heuristics are not able to find high-quality solutions consistently. Against this background, we propose new integer programming strategies that significantly improve our ability to train the hard-margin SVM model to global optimality. We introduce an iterative sampling and decomposition approach, in which smaller subproblems are used to separate combinatorial Benders' cuts. Those cuts, used within a branch-and-cut algorithm, permit to converge much more quickly towards a global optimum. Through extensive numerical analyses on classical benchmark data sets, our solution algorithm solves, for the first time, 117 new data sets to optimality and achieves a reduction of 50% in the average optimality gap for the hardest datasets of the benchmark