Non-convex nested Benders decomposition

We propose a new decomposition method to solve multistage non-convex mixed-integer (stochastic) nonlinear programming problems (MINLPs). We call this algorithm non-convex nested Benders decomposition (NC-NBD). NC-NBD is based on solving dynamically improved mixed-integer linear outer approximations of the MINLP, obtained by piecewise linear relaxations of nonlinear functions. Those MILPs are solved to global optimality using an enhancement of nested Benders decomposition, in which regularization, dynamically refined binary approximations of the state variables and Lagrangian cut techniques are combined to generate Lipschitz continuous non-convex approximations of the value functions. Those approximations are then used to decide whether the approximating MILP has to be dynamically refined and in order to compute feasible solutions for the original MINLP. We prove that NC-NBD converges to an -optimal solution in a finite number of steps. We provide promising computational results for some unit commitment problems of moderate size

Support vector machines within a bivariate mixed-integer linear programming framework

Support vector machines (SVMs) are a powerful machine learning paradigm, performing supervised learning for classification and regression analysis. A number of SVM models in the literature have made use of advances in mixed-integer linear programming (MILP) techniques in order to perform this task efficiently. In this work, we present three new models for SVMs that make use of piecewise linear (PWL) functions. This allows effective separation of data points where a simple linear SVM model may not be sufficient. The models we present make use of binary variables to assign data points to SVM segments, and hence fit within a recently presented framework for machine learning MILP models. Alongside presenting an inbuilt feature selection operator, we show that the models can benefit from robust inbuilt outlier detection. Experimental results show when each of the presented models is effective, and we present guidelines on which of the models are preferable in different scenarios

Generating optimal robust continuous piecewise linear regression with outliers through combinatorial Benders decomposition

Using piecewise linear (PWL) functions to model discrete data has applications for example in healthcare, engineering and pattern recognition. Recently, mixed-integer linear programming (MILP) approaches have been used to optimally fit continuous PWL functions. We extend these formulations to allow for outliers. The resulting MILP models rely on binary variables and big-$\textit{M}$ constructs to model logical implications. The combinatorial Benders decomposition (CBD) approach removes the dependency on the big-$\textit{M}$ constraints by separating the MILP model into a master problem of the complicating binary variables and a linear sub problem over the continuous variables, which feeds combinatorial solution information into the master problem. We use the CBD approach to decompose the proposed MILP model and solve for optimal PWL functions. Computational results show that vast speedups can be found using this robust approach, with problem-specific improvements including smart initialization, strong cut generation and special branching approaches leading to even faster solve times, up to more than 12,000 times faster than the standard MILP approach

A unified framework for bivariate clustering and regression problems via mixed-integer linear programming

Clustering and regression are two of the most important problems in data analysis and machine learning. Recently, mixed-integer linear programs (MILPs) have been presented in the literature to solve these problems. By modelling the problems as MILPs, they are able to be solved very quickly by commercial solvers. In particular, MILPs for bivariate clusterwise linear regression (CLR) and (continuous) piecewise linear regression (PWLR) have recently appeared. These MILP models make use of binary variables and logical implications modelled through big-$\mathcal{M}$ constraints. In this paper, we present these models in the context of a unifying MILP framework for bivariate clustering and regression problems. We then present two new formulations within this framework, the first for ordered CLR, and the second for clusterwise piecewise linear regression (CPWLR). The CPWLR problem concerns simultaneously clustering discrete data, while modelling each cluster with a continuous PWL function. Extending upon the framework, we discuss how outlier detection can be implemented within the models, and how specific decomposition methods can be used to find speedups in the runtime. Experimental results show when each model is the most effective

Leveraged least trimmed absolute deviations

The design of regression models that are not affected by outliers is an important task which has been subject of numerous papers within the statistics community for the last decades. Prominent examples of robust regression models are least trimmed squares (LTS), where the k largest squared deviations are ignored, and least trimmed absolute deviations (LTA) which ignores the k largest absolute deviations. The numerical complexity of both models is driven by the number of binary variables and by the value k of ignored deviations. We introduce leveraged least trimmed absolute deviations (LLTA) which exploits that LTA is already immune against y-outliers. Therefore, LLTA has only to be guarded against outlying values in x, so-called leverage points, which can be computed beforehand, in contrast to y-outliers. Thus, while the mixed-integer formulations of LTS and LTA have as many binary variables as data points, LLTA only needs one binary variable per leverage point, resulting in a significant reduction of binary variables. Based on 11 data sets from the literature, we demonstrate that (1) LLTA’s prediction quality improves much faster than LTS and as fast as LTA for increasing values of k and (2) that LLTA solves the benchmark problems about 80 times faster than LTS and about five times faster than LTA, in median

Crossing Minimal Edge-Constrained Layout Planning using Benders Decomposition

We present a new crossing number problem, which we refer to as the edge-constrained weighted two-layer crossing number problem (ECW2CN). The ECW2CN arises in layout planning of hose coupling stations at BASF, where the challenge is to find a crossing minimal assignment of tube-connected units to given positions on two opposing layers. This allows the use of robots in an effort to reduce the probability of operational disruptions and to increase human safety. Physical limitations imply maximal length and maximal curvature conditions on the tubes as well as spatial constraints imposed by the surrounding walls. This is the major difference of ECW2CN to all known variants of the crossing number problem. Such as many variants of the crossing number problem, ECW2CN is NP-hard. Because the optimization model grows fast with respect to the input data, we face out-of-memory errors for the monolithic model. Therefore, we develop two solution methods. In the first method, we tailor Benders decomposition toward the problem. The Benders subproblems are solved analytically and the Benders master problem is strengthened by additional cuts. Furthermore, we combine this Benders decomposition with ideas borrowed from fix-and-relax heuristics to design the Dynamic Fix-and-Relax Pump (DFRP). Based on an initial solution, DFRP improves successively feasible points by solving dynamically sampled smaller problems with Benders decomposition. Because the optimization model is a surrogate model for its time-dependent formulation, we evaluate the obtained solutions for different choices of the objective function via a simulation model. All algorithms are implemented efficiently using advanced features of the GuRoBi-Python API, such as callback functions and lazy constraints. We present a case study for BASF using real data and make the real-world data openly available

MINLP formulations for continuous piecewise linear function fitting

We consider a nonconvex mixed-integer nonlinear programming (MINLP) model proposed by Goldberg et al. (Comput Optim Appl 58:523–541, 2014. https://doi.org/10.1007/s10589-014-9647-y) for piecewise linear function fitting. We show that this MINLP model is incomplete and can result in a piecewise linear curve that is not the graph of a function, because it misses a set of necessary constraints. We provide two counterexamples to illustrate this effect, and propose three alternative models that correct this behavior. We investigate the theoretical relationship between these models and evaluate their computational performance

An introduction to optimal power flow: Theory, formulation, and examples

<p>The set of optimization problems in electric power systems engineering known collectively as Optimal Power Flow (OPF) is one of the most practically important and well-researched subfields of constrained nonlinear optimization. OPF has enjoyed a rich history of research, innovation, and publication since its debut five decades ago. Nevertheless, entry into OPF research is a daunting task for the uninitiated—both due to the sheer volume of literature and because OPF's ubiquity within the electric power systems community has led authors to assume a great deal of prior knowledge that readers unfamiliar with electric power systems may not possess. This article provides an introduction to OPF from an operations research perspective; it describes a complete and concise basis of knowledge for beginning OPF research. The discussion is tailored for the operations researcher who has experience with nonlinear optimization but little knowledge of electrical engineering. Topics covered include power systems modeling, the power flow equations, typical OPF formulations, and common OPF extensions.</p