Many optimization problems involve integer and continuous variables that can be modeled as  mixed integer nonlinear programming (MINLP) problems. This has led to a wide range of  applications, in particular in some engineering areas. Here, we provide a brief overview on MINLP, and present a simple idea for a future nonconvex MINLP solution technique.Fundação para a Ciência e a Tecnologia (FCT

Ch. Tsitouras

Edite M. G. P. Fernandes

Florbela P. Fernandes

George Psihoyios

M. Fernanda P. Costa

Theodore E. Simos

Crossref

Overview on Mixed Integer Nonlinear Programming
Problems
Florbela P. Fernandes∗,†, M. Fernanda P. Costa∗∗,† and Edite M.G.P. Fernandes‡
∗Department of Mathematics, Polytechnic Institute of Braganca, Portugal
†CMAT-University of Minho
∗∗Department of Mathematics for Science and Technology, University of Minho, Portugal
‡Department of Production and Systems, University of Minho, Portugal
Abstract. Many optimization problems involve integer and continuous variables that can be modeled as mixed integer
nonlinear programming (MINLP) problems. This has led to a wide range of applications, in particular in some engineering
areas. Here, we provide a brief overview on MINLP, and present a simple idea for a future nonconvex MINLP solution
technique.
Keywords: mixed-integer nonlinear programs; branch-and-bound; outer-approximation.
PACS: 02.60.Pn
INTRODUCTION
Many optimization problems involve discrete and continuous variables that can be modeled as mixed integer nonlinear
programming (MINLP) problems. For instance, integer variables can represent the number of workers needed to
perform a certain task whereas continuous variables can denote physical values, such as pressure or temperature.
This has led to a wide range of applications in the field of process systems engineering [10]. In particular, one may
find applications which include gas network problems, nuclear core reloaded problems, cyclic scheduling trim-loss
optimization in the paper industry, synthesis problems, layout problems [1]. Other examples are efficient management
of electricity transmission, contingency analysis and blackout prevention of electric power systems [15].
A mixed integer nonlinear program formulation can be represented as:
min f (x,y)
s. t. g j(x,y)≤ 0, j ∈ J
x ∈ X , y ∈ Y (1)
where X ⊆ Rn is assumed to be a convex compact set, Y ⊆ Zp corresponds to a polyhedral set of integer points,
f :Rn+p →R and g :Rn+p →Rm are continuously differentiable functions, J is the index set of inequality constraints,
and x and y are the continuous and discrete/integer variables, respectively. If the objective function f and the constraint
functions g are convex, the problem is known as convex, otherwise the problem is a nonconvex MINLP. In the last
decades, significant progress has been made in the solution techniques for convex MINLP. However, techniques and
solvers for nonconvex MINLP problem have just started to appear in the literature [14].
The paper is organized as follows. First, we briefly present the main techniques for solving convex MINLP problems.
We then present a future idea for nonconvex MINLP based on a heuristic linear/quadratic interpolation model-based
method.
MINLP SOLUTION TECHNIQUES
Here, we assume that the problem is a convex MINLP. The available methods to solve this type of problem can
be derived from three basic nonlinear programming (NLP) subproblems and from one cutting plane mixed integer
linear programming (MILP) problem, related with (1) [2, 10]. Different methodologies have been proposed to solve
convex MINLP, in particular: Branch-and-Bound (BB), Outer-Approximation (OA), LP/NLP-based branch-and-bound
(LP/NLP-BB), Generalized Benders Decomposition, Extended Cutting Plane, and Branch-and-Cut, [9, 10, 11, 15, 19].
Our brief presentation below will focus mainly on the BB, OA and LP/NLP-BB methodologies.
Branch-and-Bound. Although BB was originally devised for MILP, it can also be applied to MINLP. We refer the
reader to Leyffer [13] and the references therein included. The method solves a NLP relaxation subproblem at each
node of the tree, instead of an LP, represented by
LBk ≡min f (x,y)
s. t. g j(x,y)≤ 0, j ∈ J
x ∈ X , y ∈ YR
yi ≤ uki ,
yi ≥ lki
(NLP-rel-k)
for some i, where YR is a continuous relaxation of the set Y and uki = byhi c (floor function applied to the noninteger value
yhi of a previous step h < k), l
k
i = dymi e (ceiling function applied to the noninteger value ymi of a previous step m < k).
When k = 0, problem (NLP-rel-k) corresponds to the continuous NLP relaxation of the MINLP problem (1). The BB
method starts by solving the subproblem (NLP-rel-k) with k = 0. If all the integer variables of the problem take integer
values, the optimal solution is found and the algorithm stops. Otherwise, a tree search is performed in the space of the
integer variables. Simple bounds yi ≤ uki ,yi ≥ lki are added successively (at nodes of the tree) yielding subproblems
(NLP-rel-k) at each kth step in the branch and bound search. The solution of each subproblem (NLP-rel-k) provides
a lower bound for the subproblems in the descent nodes of the tree. This process continues until the lower bound
exceeds the current upper bound, the NLP subproblem is infeasible, or the solution provides integer values for the
integer variables. The integer solutions (at the nodes of the tree) give upper bounds on the optimal integer solution. An
example of the enumeration tree is represented on the left of Figure 1. Double boxed nodes represent integer solutions.
The optimal solution is LB5. The BB method is only attractive if the NLP relaxation subproblems are inexpensive to
solve, or when only a few of them need to be solved. Otherwise the tree can grow very large and solving a large number
of NLP can be time-consuming. The well-known solvers Bonmin [3], KNITRO [4], MINLP-BB [12] and SBB [8], are
branch-and-bound based methodologies and available for convex MINLP.
Outer-Approximation. The OA method was introduced by Duran and Grossman [6] and later used by Fletcher and
Leyffer [7]. The OA method performs a cycle of major iterations, k = 1, . . . ,K, in which the NLP reduction subproblem
(NLP-red-k), for a given yk,
f kU ≡min f (x,yk)
s. t. g j(x,yk)≤ 0, j ∈ J
x ∈ X (NLP-red-k)
is solved, being xk the solution, and the MILP relaxation master problem (M-OA):
LBK ≡min l
s. t. f
(
xk,yk
)
+∇ f
(
xk,yk
)T [ x− xk
y− yk
]
≤ l
g j
(
xk,yk
)
+∇g j
(
xk,yk
)T [ x− xk
y− yk
]
≤ 0, j ∈ J
x ∈ X , y ∈ Y
k = 1, . . . ,K
(M-OA)
is updated with the corresponding objective and constraint function linearizations at the point (xk,yk), and solved to
generate a new vector of discrete values. Usually, not all yk ∈ Y give rise to a feasible (NLP-red-k) subproblem, i.e.,
does not exist xk ∈ X that satisfies g j(xk,yk)≤ 0, j ∈ J. If subproblem (NLP-red-k) is feasible, the solution xk is used
to define the first MILP master problem. Otherwise, the following feasibility NLP subproblem
min u
s. t. g j(x,yk)≤ u, i ∈ J
x ∈ X , u ∈ R+ (NLP-F)
FIGURE 1. Enumeration tree in branch-and-bound (on the left) and flowchart of outer-approximation (on the right)
for a fixed yk, is solved to provide the corresponding xk. The feasibility NLP subproblem (NLP-F) can be interpreted
as the minimization of the infinity-norm measure of infeasibility of the corresponding NLP subproblem. The OA
algorithm starts with a vector (x0,y0) that can be either a feasible solution to (1) or to the continuous NLP relaxation
of the MINLP problem. The solution of problem (M-OA) LBK corresponds to a lower bound for problem (1).
Since the linearizations are accumulated as k increases, the master problems (M-OA) generate a sequence of lower
bounds LB1 ≤ LB2 ≤ . . . ≤ LBK . This cycle of iterations continues until the difference between the upper bound
UBK = mink f kU , provided by subproblems (NLP-red-k), and the lower bound provided by the MILP relaxation master
problem LBK is sufficiently small. The flowchart on the right of Figure 1 shows the major steps in OA type algorithms.
The successive accumulation of linearizations generating master problems (M-OA) with large number of constraints
is the main drawback of OA based methods. Keeping only the last linearization point is not appropriate since
the monotonic increase of the lower bound is not guaranteed. It is shown in [18] that the aggregation of linear
approximations to the objective and constraint functions can be carried out for the efficient solving of MILP master
problem. Experience shows that the OA method requires in general a reduced number of major iterations. If f and
g are linear functions in x and y then OA method converges in one iteration, since the MILP master program (M-
OA) corresponds to the problem (1). Furthermore, the master problem (M-OA) need not be solved to optimality. It is
sufficient to find a new point (yk,xk) that gives a solution to the MILP that is below UBK within a specified tolerance.
LP/NLP-based Branch-and-Bound. The LP/NLP-BB is a clever extension of OA, which avoids solving an alter-
nating sequence of MILP master problems (M-OA) and NLP subproblems. The basic idea consists of performing
an LP-based branch-and-bound for the continuous relaxation to (M-OA) and enforcing integrality of the variables y
by branching. This approach avoids solving multiple MILP master problems by interrupting the MILP tree search
whenever an integer feasible solution yk, say, is found to solve the NLP subproblem ((NLP-red-k) or (NLP-F)). Then
it updates the representation of the master problem in the current open nodes of the tree with the corresponding lin-
earizations, and the MILP tree search continues. The algorithm is stated in full detail in [11]. A list of some available
solvers based on linearizations is: Bonmin [3], Dicopt (http://www.gams.com/dd/docs/solvers) and FilMINT [1].
Recent and future work
Branch-and-bound type methods have already been proposed for solving nonconvex MINLP [14]. The list
of the available software, mostly deterministic optimizers for nonconvex MINLP is the following: BARON
(http://archimedes.cheme.cmu.edu/baron/baron.html), Couenne (https://projects.coin-or.org/Couenne) and LaGO
[16]. Most of these codes are available for use free of charge and/or are open source. The difficulty with nonconvex
MINLP is that common optimizers when solving NLP problems with nonconvex f and g functions are able to
guarantee convergence only to a local minimum. This means that the solution of the NLP relaxation subproblem does
not provide a lower bound on the solution of the original problem (1). An additional relaxation process is required,
typically involving the decomposition of the nonlinear functions into components, see [15].
An alternative idea, keeping a classic branch-and-bound approach, is to solve nonconvex NLP subproblems using
a heuristic combination of deterministic and stochastic algorithms, that is able to guarantee convergence to a global
solution with a high probability. This is a matter for future research. Since our proposal aims to focus on derivative-
free methods, the required linear and/or quadratic approximation models are typically constructed via interpolation
model-based methods [5, 17].
ACKNOWLEDGMENTS
This work has been partially supported by the FCT (Fundação para a Ciência e a Tecnologia), Portugal.
REFERENCES
1. Abhishek, K., Leyffer, S. and Linderoth, J. (2006). FilMINT: An Outer-Approximation-Based Solver for Nonlinear Mixed
Integer Programs. Technical Report ANL/MCS-P1374-0906, Mathematics and Computer Science Division, Argonne National
Laboratory.
2. Biegler, L. and Grossmann, I. (2004). Retrospective on optimization. Computers and Chemical Engineering 28: 1169–1192.
3. Bonami, P., Biegler, L., Conn, A., Cornuejols, G., Grossmann, I., Laird, C., Lee, J., Lodi, A., Margot, F., Sawaya, N.,
Wachter, A., (2005). An algorihmic framework for convex mixed integer nonlinear programs. Technical Report RC23771,
IBM Research Report.
4. Byrd, R. H., Nocedal, J. and Waltz, R. A. (2006). KNITRO: An Integrated Package for Nonlinear Optimization in Large-Scale
Nonlinear Optimization, G. di Pillo and M. Roma, eds, 35-59 , Springer-Verlag.
5. Fasano, G., Morales, J. L., Nocedal. J. (2009) On the geometry phase in model-based algorithms for derivative-free optimization.
Optimization Methods and Software 24: 145–154.
6. Duran, M. and Grossmann, I. (1986). An outer approximation algorithm for a class of mixed-integer nonlinear programs.
Mathematical Programming 36: 307–339.
7. Fletcher, F. and Leyffer, S. (1994). Solving mixed-integer nonlinear programs by outer approximation. Mathematical
Programming 66: 327–349.
8. Gams Development Corp. (2003). Gams – The Solver Manuals. GAMS Development Corp., Washington, DC.
9. Geoffrion, A. M. (1972). Generalized Benders decomposition. Journal of Optimization Theory and Applications 10:4 237–260.
10. Grossmann, I. (2002). Review of Nonlinear Mixed-Integer and Disjunctive Programming Techniques. Optimization and
Engineering 3: 227–252.
11. Leyffer, S. (1993). Deterministic Methods for Mixed Integer Nonlinear Programming. PhD Thesis, University of Dundee,
United Kingdom.
12. Leyffer, S. (1999). User Manual for MINLP BB, University of Dundee
13. Leyffer, S. (2001). Integrating SQP and branch and bound for mixed integer nonlinear programming. Computational
Optimization and Applications 18: 295–309.
14. Leyffer, S., Sartenaer, A., Wanufelle, E. (2008). Branch-and-Refine for Mixed Integer Nonconvex Global Optimization.
Preprint ANL/MCS-P1547-0908, Mathematics and Computer Science Division, Argonne National Laboratory.
15. Leyffer, S., Linderoth, J., Luedtke, J., Miller, A., Munson, T. (2009). Applications and Algorithms for Mixed Integer Nonlinear
Programming. Preprint ANL/MCS-P1630-0509, Mathematics and Computer Science Division, Argonne National Laboratory.
16. Nowak, I., Vigerske, S. (2008). LaGO: a (heuristic) branch and cut algorithm for nonconvex MINLPs. Central European
Journal of Operations Research 16: 127–138.
17. Powell, M. J. C. (2004). Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. Mathematical
Programming 100: 183–215.
18. Quesada, I., Grossmann, I. E. (1992). An LP/NLP based branch and bound algorithm for convex MINLP optimization
problems. Computers and Chemical Engineering 16:2 937–947.
19. Westerlund, T., Pettersson, F. (1995). An Extended cutting plane method for solving convex MINL problems.Computers and
Chemical Engineering 19: 131–136.


English

Overview on Mixed Integer Nonlinear Programming Problems

Overview on Mixed Integer Nonlinear ProgrammingProblemsFlorbela P. Fernandes∗,†, M. Fernanda P. Costa∗∗,† and Edite M.G.P. Fernandes‡∗Department of Mathematics, Polytechnic Institute of Braganca, Portugal†CMAT-University of Minho∗∗Department of Mathematics for Science and Technology, University of Minho, Portugal‡Department of Production and Systems, University of Minho, PortugalAbstract. Many optimization problems involve integer and continuous variables that can be modeled as mixed integernonlinear programming (MINLP) problems. This has led to a wide range of applications, in particular in some engineeringareas. Here, we provide a brief overview on MINLP, and present a simple idea for a future nonconvex MINLP solutiontechnique.Keywords: mixed-integer nonlinear programs; branch-and-bound; outer-approximation.PACS: 02.60.PnINTRODUCTIONMany optimization problems involve discrete and continuous variables that can be modeled as mixed integer nonlinearprogramming (MINLP) problems. For instance, integer variables can represent the number of workers needed toperform a certain task whereas continuous variables can denote physical values, such as pressure or temperature.This has led to a wide range of applications in the field of process systems engineering [10]. In particular, one mayfind applications which include gas network problems, nuclear core reloaded problems, cyclic scheduling trim-lossoptimization in the paper industry, synthesis problems, layout problems [1]. Other examples are efficient managementof electricity transmission, contingency analysis and blackout prevention of electric power systems [15].A mixed integer nonlinear program formulation can be represented as:min f (x,y)s. t. g j(x,y)≤ 0, j ∈ Jx ∈ X , y ∈ Y (1)where X ⊆ Rn is assumed to be a convex compact set, Y ⊆ Zp corresponds to a polyhedral set of integer points,f :Rn+p →R and g :Rn+p →Rm are continuously differentiable functions, J is the index set of inequality constraints,and x and y are the continuous and discrete/integer variables, respectively. If the objective function f and the constraintfunctions g are convex, the problem is known as convex, otherwise the problem is a nonconvex MINLP. In the lastdecades, significant progress has been made in the solution techniques for convex MINLP. However, techniques andsolvers for nonconvex MINLP problem have just started to appear in the literature [14].The paper is organized as follows. First, we briefly present the main techniques for solving convex MINLP problems.We then present a future idea for nonconvex MINLP based on a heuristic linear/quadratic interpolation model-basedmethod.MINLP SOLUTION TECHNIQUESHere, we assume that the problem is a convex MINLP. The available methods to solve this type of problem canbe derived from three basic nonlinear programming (NLP) subproblems and from one cutting plane mixed integerlinear programming (MILP) problem, related with (1) [2, 10]. Different methodologies have been proposed to solveconvex MINLP, in particular: Branch-and-Bound (BB), Outer-Approximation (OA), LP/NLP-based branch-and-bound(LP/NLP-BB), Generalized Benders Decomposition, Extended Cutting Plane, and Branch-and-Cut, [9, 10, 11, 15, 19].Our brief presentation below will focus mainly on the BB, OA and LP/NLP-BB methodologies.Branch-and-Bound. Although BB was originally devised for MILP, it can also be applied to MINLP. We refer thereader to Leyffer [13] and the references therein included. The method solves a NLP relaxation subproblem at eachnode of the tree, instead of an LP, represented byLBk ≡min f (x,y)s. t. g j(x,y)≤ 0, j ∈ Jx ∈ X , y ∈ YRyi ≤ uki ,yi ≥ lki(NLP-rel-k)for some i, where YR is a continuous relaxation of the set Y and uki = byhi c (floor function applied to the noninteger valueyhi of a previous step h < k), lki = dymi e (ceiling function applied to the noninteger value ymi of a previous step m < k).When k = 0, problem (NLP-rel-k) corresponds to the continuous NLP relaxation of the MINLP problem (1). The BBmethod starts by solving the subproblem (NLP-rel-k) with k = 0. If all the integer variables of the problem take integervalues, the optimal solution is found and the algorithm stops. Otherwise, a tree search is performed in the space of theinteger variables. Simple bounds yi ≤ uki ,yi ≥ lki are added successively (at nodes of the tree) yielding subproblems(NLP-rel-k) at each kth step in the branch and bound search. The solution of each subproblem (NLP-rel-k) providesa lower bound for the subproblems in the descent nodes of the tree. This process continues until the lower boundexceeds the current upper bound, the NLP subproblem is infeasible, or the solution provides integer values for theinteger variables. The integer solutions (at the nodes of the tree) give upper bounds on the optimal integer solution. Anexample of the enumeration tree is represented on the left of Figure 1. Double boxed nodes represent integer solutions.The optimal solution is LB5. The BB method is only attractive if the NLP relaxation subproblems are inexpensive tosolve, or when only a few of them need to be solved. Otherwise the tree can grow very large and solving a large numberof NLP can be time-consuming. The well-known solvers Bonmin [3], KNITRO [4], MINLP-BB [12] and SBB [8], arebranch-and-bound based methodologies and available for convex MINLP.Outer-Approximation. The OA method was introduced by Duran and Grossman [6] and later used by Fletcher andLeyffer [7]. The OA method performs a cycle of major iterations, k = 1, . . . ,K, in which the NLP reduction subproblem(NLP-red-k), for a given yk,f kU ≡min f (x,yk)s. t. g j(x,yk)≤ 0, j ∈ Jx ∈ X (NLP-red-k)is solved, being xk the solution, and the MILP relaxation master problem (M-OA):LBK ≡min ls. t. f(xk,yk)+∇ f(xk,yk)T [ x− xky− yk]≤ lg j(xk,yk)+∇g j(xk,yk)T [ x− xky− yk]≤ 0, j ∈ Jx ∈ X , y ∈ Yk = 1, . . . ,K(M-OA)is updated with the corresponding objective and constraint function linearizations at the point (xk,yk), and solved togenerate a new vector of discrete values. Usually, not all yk ∈ Y give rise to a feasible (NLP-red-k) subproblem, i.e.,does not exist xk ∈ X that satisfies g j(xk,yk)≤ 0, j ∈ J. If subproblem (NLP-red-k) is feasible, the solution xk is usedto define the first MILP master problem. Otherwise, the following feasibility NLP subproblemmin us. t. g j(x,yk)≤ u, i ∈ Jx ∈ X , u ∈ R+ (NLP-F)FIGURE 1. Enumeration tree in branch-and-bound (on the left) and flowchart of outer-approximation (on the right)for a fixed yk, is solved to provide the corresponding xk. The feasibility NLP subproblem (NLP-F) can be interpretedas the minimization of the infinity-norm measure of infeasibility of the corresponding NLP subproblem. The OAalgorithm starts with a vector (x0,y0) that can be either a feasible solution to (1) or to the continuous NLP relaxationof the MINLP problem. The solution of problem (M-OA) LBK corresponds to a lower bound for problem (1).Since the linearizations are accumulated as k increases, the master problems (M-OA) generate a sequence of lowerbounds LB1 ≤ LB2 ≤ . . . ≤ LBK . This cycle of iterations continues until the difference between the upper boundUBK = mink f kU , provided by subproblems (NLP-red-k), and the lower bound provided by the MILP relaxation masterproblem LBK is sufficiently small. The flowchart on the right of Figure 1 shows the major steps in OA type algorithms.The successive accumulation of linearizations generating master problems (M-OA) with large number of constraintsis the main drawback of OA based methods. Keeping only the last linearization point is not appropriate sincethe monotonic increase of the lower bound is not guaranteed. It is shown in [18] that the aggregation of linearapproximations to the objective and constraint functions can be carried out for the efficient solving of MILP masterproblem. Experience shows that the OA method requires in general a reduced number of major iterations. If f andg are linear functions in x and y then OA method converges in one iteration, since the MILP master program (M-OA) corresponds to the problem (1). Furthermore, the master problem (M-OA) need not be solved to optimality. It issufficient to find a new point (yk,xk) that gives a solution to the MILP that is below UBK within a specified tolerance.LP/NLP-based Branch-and-Bound. The LP/NLP-BB is a clever extension of OA, which avoids solving an alter-nating sequence of MILP master problems (M-OA) and NLP subproblems. The basic idea consists of performingan LP-based branch-and-bound for the continuous relaxation to (M-OA) and enforcing integrality of the variables yby branching. This approach avoids solving multiple MILP master problems by interrupting the MILP tree searchwhenever an integer feasible solution yk, say, is found to solve the NLP subproblem ((NLP-red-k) or (NLP-F)). Thenit updates the representation of the master problem in the current open nodes of the tree with the corresponding lin-earizations, and the MILP tree search continues. The algorithm is stated in full detail in [11]. A list of some availablesolvers based on linearizations is: Bonmin [3], Dicopt (http://www.gams.com/dd/docs/solvers) and FilMINT [1].Recent and future workBranch-and-bound type methods have already been proposed for solving nonconvex MINLP [14]. The listof the available software, mostly deterministic optimizers for nonconvex MINLP is the following: BARON(http://archimedes.cheme.cmu.edu/baron/baron.html), Couenne (https://projects.coin-or.org/Couenne) and LaGO[16]. Most of these codes are available for use free of charge and/or are open source. The difficulty with nonconvexMINLP is that common optimizers when solving NLP problems with nonconvex f and g functions are able toguarantee convergence only to a local minimum. This means that the solution of the NLP relaxation subproblem doesnot provide a lower bound on the solution of the original problem (1). An additional relaxation process is required,typically involving the decomposition of the nonlinear functions into components, see [15].An alternative idea, keeping a classic branch-and-bound approach, is to solve nonconvex NLP subproblems usinga heuristic combination of deterministic and stochastic algorithms, that is able to guarantee convergence to a globalsolution with a high probability. This is a matter for future research. Since our proposal aims to focus on derivative-free methods, the required linear and/or quadratic approximation models are typically constructed via interpolationmodel-based methods [5, 17].ACKNOWLEDGMENTSThis work has been partially supported by the FCT (Fundação para a Ciência e a Tecnologia), Portugal.REFERENCES1. Abhishek, K., Leyffer, S. and Linderoth, J. (2006). FilMINT: An Outer-Approximation-Based Solver for Nonlinear MixedInteger Programs. Technical Report ANL/MCS-P1374-0906, Mathematics and Computer Science Division, Argonne NationalLaboratory.2. Biegler, L. and Grossmann, I. (2004). Retrospective on optimization. Computers and Chemical Engineering 28: 1169–1192.3. Bonami, P., Biegler, L., Conn, A., Cornuejols, G., Grossmann, I., Laird, C., Lee, J., Lodi, A., Margot, F., Sawaya, N.,Wachter, A., (2005). An algorihmic framework for convex mixed integer nonlinear programs. Technical Report RC23771,IBM Research Report.4. Byrd, R. H., Nocedal, J. and Waltz, R. A. (2006). KNITRO: An Integrated Package for Nonlinear Optimization in Large-ScaleNonlinear Optimization, G. di Pillo and M. Roma, eds, 35-59 , Springer-Verlag.5. Fasano, G., Morales, J. L., Nocedal. J. (2009) On the geometry phase in model-based algorithms for derivative-free optimization.Optimization Methods and Software 24: 145–154.6. Duran, M. and Grossmann, I. (1986). An outer approximation algorithm for a class of mixed-integer nonlinear programs.Mathematical Programming 36: 307–339.7. Fletcher, F. and Leyffer, S. (1994). Solving mixed-integer nonlinear programs by outer approximation. MathematicalProgramming 66: 327–349.8. Gams Development Corp. (2003). Gams – The Solver Manuals. GAMS Development Corp., Washington, DC.9. Geoffrion, A. M. (1972). Generalized Benders decomposition. Journal of Optimization Theory and Applications 10:4 237–260.10. Grossmann, I. (2002). Review of Nonlinear Mixed-Integer and Disjunctive Programming Techniques. Optimization andEngineering 3: 227–252.11. Leyffer, S. (1993). Deterministic Methods for Mixed Integer Nonlinear Programming. PhD Thesis, University of Dundee,United Kingdom.12. Leyffer, S. (1999). User Manual for MINLP BB, University of Dundee13. Leyffer, S. (2001). Integrating SQP and branch and bound for mixed integer nonlinear programming. ComputationalOptimization and Applications 18: 295–309.14. Leyffer, S., Sartenaer, A., Wanufelle, E. (2008). Branch-and-Refine for Mixed Integer Nonconvex Global Optimization.Preprint ANL/MCS-P1547-0908, Mathematics and Computer Science Division, Argonne National Laboratory.15. Leyffer, S., Linderoth, J., Luedtke, J., Miller, A., Munson, T. (2009). Applications and Algorithms for Mixed Integer NonlinearProgramming. Preprint ANL/MCS-P1630-0509, Mathematics and Computer Science Division, Argonne National Laboratory.16. Nowak, I., Vigerske, S. (2008). LaGO: a (heuristic) branch and cut algorithm for nonconvex MINLPs. Central EuropeanJournal of Operations Research 16: 127–138.17. Powell, M. J. C. (2004). Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. MathematicalProgramming 100: 183–215.18. Quesada, I., Grossmann, I. E. (1992). An LP/NLP based branch and bound algorithm for convex MINLP optimizationproblems. Computers and Chemical Engineering 16:2 937–947.19. Westerlund, T., Pettersson, F. (1995). An Extended cutting plane method for solving convex MINL problems.Computers andChemical Engineering 19: 131–136.

MUCC (Crossref)

Many optimization problems involve integer and continuous variables that can be modeled as mixed integer nonlinear programming (MINLP) problems. This has led to a wide range of applications, in particular in some engineering areas. Here, we provide a brief overview on MINLP, and present a simple idea for a future nonconvex MINLP solution technique.Fundação para a Ciência e a Tecnologia (FCT

Fernandes, Florbela P.

Costa, Maria F.P.

Fernandes, Edite M.G.P.

Biblioteca Digital do IPB

Overview on Mixed Integer Nonlinear ProgrammingProblemsFlorbela P. Fernandes∗,†, M. Fernanda P. Costa∗∗,† and Edite M.G.P. Fernandes‡∗Department of Mathematics, Polytechnic Institute of Braganca, Portugal†CMAT-University of Minho∗∗Department of Mathematics for Science and Technology, University of Minho, Portugal‡Department of Production and Systems, University of Minho, PortugalAbstract. Many optimization problems involve integer and continuous variables that can be modeled as mixed integernonlinear programming (MINLP) problems. This has led to a wide range of applications, in particular in some engineeringareas. Here, we provide a brief overview on MINLP, and present a simple idea for a future nonconvex MINLP solutiontechnique.Keywords: mixed-integer nonlinear programs; branch-and-bound; outer-approximation.PACS: 02.60.PnINTRODUCTIONMany optimization problems involve discrete and continuous variables that can be modeled as mixed integer nonlinearprogramming (MINLP) problems. For instance, integer variables can represent the number of workers needed toperform a certain task whereas continuous variables can denote physical values, such as pressure or temperature.This has led to a wide range of applications in the field of process systems engineering [10]. In particular, one mayfind applications which include gas network problems, nuclear core reloaded problems, cyclic scheduling trim-lossoptimization in the paper industry, synthesis problems, layout problems [1]. Other examples are efficient managementof electricity transmission, contingency analysis and blackout prevention of electric power systems [15].A mixed integer nonlinear program formulation can be represented as:min f (x,y)s. t. g j(x,y)≤ 0, j ∈ Jx ∈ X , y ∈ Y (1)where X ⊆ Rn is assumed to be a convex compact set, Y ⊆ Zp corresponds to a polyhedral set of integer points,f :Rn+p →R and g :Rn+p →Rm are continuously differentiable functions, J is the index set of inequality constraints,and x and y are the continuous and discrete/integer variables, respectively. If the objective function f and the constraintfunctions g are convex, the problem is known as convex, otherwise the problem is a nonconvex MINLP. In the lastdecades, significant progress has been made in the solution techniques for convex MINLP. However, techniques andsolvers for nonconvex MINLP problem have just started to appear in the literature [14].The paper is organized as follows. First, we briefly present the main techniques for solving convex MINLP problems.We then present a future idea for nonconvex MINLP based on a heuristic linear/quadratic interpolation model-basedmethod.MINLP SOLUTION TECHNIQUESHere, we assume that the problem is a convex MINLP. The available methods to solve this type of problem canbe derived from three basic nonlinear programming (NLP) subproblems and from one cutting plane mixed integerlinear programming (MILP) problem, related with (1) [2, 10]. Different methodologies have been proposed to solveconvex MINLP, in particular: Branch-and-Bound (BB), Outer-Approximation (OA), LP/NLP-based branch-and-bound(LP/NLP-BB), Generalized Benders Decomposition, Extended Cutting Plane, and Branch-and-Cut, [9, 10, 11, 15, 19].Our brief presentation below will focus mainly on the BB, OA and LP/NLP-BB methodologies.Branch-and-Bound. Although BB was originally devised for MILP, it can also be applied to MINLP. We refer thereader to Leyffer [13] and the references therein included. The method solves a NLP relaxation subproblem at eachnode of the tree, instead of an LP, represented byLBk ≡min f (x,y)s. t. g j(x,y)≤ 0, j ∈ Jx ∈ X , y ∈ YRyi ≤ uki ,yi ≥ lki(NLP-rel-k)for some i, where YR is a continuous relaxation of the set Y and uki = byhi c (floor function applied to the noninteger valueyhi of a previous step h < k), lki = dymi e (ceiling function applied to the noninteger value ymi of a previous step m < k).When k = 0, problem (NLP-rel-k) corresponds to the continuous NLP relaxation of the MINLP problem (1). The BBmethod starts by solving the subproblem (NLP-rel-k) with k = 0. If all the integer variables of the problem take integervalues, the optimal solution is found and the algorithm stops. Otherwise, a tree search is performed in the space of theinteger variables. Simple bounds yi ≤ uki ,yi ≥ lki are added successively (at nodes of the tree) yielding subproblems(NLP-rel-k) at each kth step in the branch and bound search. The solution of each subproblem (NLP-rel-k) providesa lower bound for the subproblems in the descent nodes of the tree. This process continues until the lower boundexceeds the current upper bound, the NLP subproblem is infeasible, or the solution provides integer values for theinteger variables. The integer solutions (at the nodes of the tree) give upper bounds on the optimal integer solution. Anexample of the enumeration tree is represented on the left of Figure 1. Double boxed nodes represent integer solutions.The optimal solution is LB5. The BB method is only attractive if the NLP relaxation subproblems are inexpensive tosolve, or when only a few of them need to be solved. Otherwise the tree can grow very large and solving a large numberof NLP can be time-consuming. The well-known solvers Bonmin [3], KNITRO [4], MINLP-BB [12] and SBB [8], arebranch-and-bound based methodologies and available for convex MINLP.Outer-Approximation. The OA method was introduced by Duran and Grossman [6] and later used by Fletcher andLeyffer [7]. The OA method performs a cycle of major iterations, k= 1, . . . ,K, in which the NLP reduction subproblem(NLP-red-k), for a given yk,f kU ≡min f (x,yk)s. t. g j(x,yk)≤ 0, j ∈ Jx ∈ X (NLP-red-k)is solved, being xk the solution, and the MILP relaxation master problem (M-OA):LBK ≡min ls. t. f(xk,yk)+∇ f(xk,yk)T [ x− xky− yk]≤ lg j(xk,yk)+∇g j(xk,yk)T [ x− xky− yk]≤ 0, j ∈ Jx ∈ X , y ∈ Yk = 1, . . . ,K(M-OA)is updated with the corresponding objective and constraint function linearizations at the point (xk,yk), and solved togenerate a new vector of discrete values. Usually, not all yk ∈ Y give rise to a feasible (NLP-red-k) subproblem, i.e.,does not exist xk ∈ X that satisfies g j(xk,yk)≤ 0, j ∈ J. If subproblem (NLP-red-k) is feasible, the solution xk is usedto define the first MILP master problem. Otherwise, the following feasibility NLP subproblemmin us. t. g j(x,yk)≤ u, i ∈ Jx ∈ X , u ∈ R+ (NLP-F)FIGURE 1. Enumeration tree in branch-and-bound (on the left) and flowchart of outer-approximation (on the right)for a fixed yk, is solved to provide the corresponding xk. The feasibility NLP subproblem (NLP-F) can be interpretedas the minimization of the infinity-norm measure of infeasibility of the corresponding NLP subproblem. The OAalgorithm starts with a vector (x0,y0) that can be either a feasible solution to (1) or to the continuous NLP relaxationof the MINLP problem. The solution of problem (M-OA) LBK corresponds to a lower bound for problem (1).Since the linearizations are accumulated as k increases, the master problems (M-OA) generate a sequence of lowerbounds LB1 ≤ LB2 ≤ . . . ≤ LBK . This cycle of iterations continues until the difference between the upper boundUBK = mink f kU , provided by subproblems (NLP-red-k), and the lower bound provided by the MILP relaxation masterproblem LBK is sufficiently small. The flowchart on the right of Figure 1 shows the major steps in OA type algorithms.The successive accumulation of linearizations generating master problems (M-OA) with large number of constraintsis the main drawback of OA based methods. Keeping only the last linearization point is not appropriate sincethe monotonic increase of the lower bound is not guaranteed. It is shown in [18] that the aggregation of linearapproximations to the objective and constraint functions can be carried out for the efficient solving of MILP masterproblem. Experience shows that the OA method requires in general a reduced number of major iterations. If f andg are linear functions in x and y then OA method converges in one iteration, since the MILP master program (M-OA) corresponds to the problem (1). Furthermore, the master problem (M-OA) need not be solved to optimality. It issufficient to find a new point (yk,xk) that gives a solution to the MILP that is below UBK within a specified tolerance.LP/NLP-based Branch-and-Bound. The LP/NLP-BB is a clever extension of OA, which avoids solving an alter-nating sequence of MILP master problems (M-OA) and NLP subproblems. The basic idea consists of performingan LP-based branch-and-bound for the continuous relaxation to (M-OA) and enforcing integrality of the variables yby branching. This approach avoids solving multiple MILP master problems by interrupting the MILP tree searchwhenever an integer feasible solution yk, say, is found to solve the NLP subproblem ((NLP-red-k) or (NLP-F)). Thenit updates the representation of the master problem in the current open nodes of the tree with the corresponding lin-earizations, and the MILP tree search continues. The algorithm is stated in full detail in [11]. A list of some availablesolvers based on linearizations is: Bonmin [3], Dicopt (http://www.gams.com/dd/docs/solvers) and FilMINT [1].Recent and future workBranch-and-bound type methods have already been proposed for solving nonconvex MINLP [14]. The listof the available software, mostly deterministic optimizers for nonconvex MINLP is the following: BARON(http://archimedes.cheme.cmu.edu/baron/baron.html), Couenne (https://projects.coin-or.org/Couenne) and LaGO[16]. Most of these codes are available for use free of charge and/or are open source. The difficulty with nonconvexMINLP is that common optimizers when solving NLP problems with nonconvex f and g functions are able toguarantee convergence only to a local minimum. This means that the solution of the NLP relaxation subproblem doesnot provide a lower bound on the solution of the original problem (1). An additional relaxation process is required,typically involving the decomposition of the nonlinear functions into components, see [15].An alternative idea, keeping a classic branch-and-bound approach, is to solve nonconvex NLP subproblems usinga heuristic combination of deterministic and stochastic algorithms, that is able to guarantee convergence to a globalsolution with a high probability. This is a matter for future research. Since our proposal aims to focus on derivative-free methods, the required linear and/or quadratic approximation models are typically constructed via interpolationmodel-based methods [5, 17].ACKNOWLEDGMENTSThis work has been partially supported by the FCT (Fundação para a Ciência e a Tecnologia), Portugal.REFERENCES1. Abhishek, K., Leyffer, S. and Linderoth, J. (2006). FilMINT: An Outer-Approximation-Based Solver for Nonlinear MixedInteger Programs. Technical Report ANL/MCS-P1374-0906, Mathematics and Computer Science Division, Argonne NationalLaboratory.2. Biegler, L. and Grossmann, I. (2004). Retrospective on optimization. Computers and Chemical Engineering 28: 1169–1192.3. Bonami, P., Biegler, L., Conn, A., Cornuejols, G., Grossmann, I., Laird, C., Lee, J., Lodi, A., Margot, F., Sawaya, N.,Wachter, A., (2005). An algorihmic framework for convex mixed integer nonlinear programs. Technical Report RC23771,IBM Research Report.4. Byrd, R. H., Nocedal, J. and Waltz, R. A. (2006). KNITRO: An Integrated Package for Nonlinear Optimization in Large-ScaleNonlinear Optimization, G. di Pillo and M. Roma, eds, 35-59 , Springer-Verlag.5. Fasano, G., Morales, J. L., Nocedal. J. (2009) On the geometry phase in model-based algorithms for derivative-free optimization.Optimization Methods and Software 24: 145–154.6. Duran, M. and Grossmann, I. (1986). An outer approximation algorithm for a class of mixed-integer nonlinear programs.Mathematical Programming 36: 307–339.7. Fletcher, F. and Leyffer, S. (1994). Solving mixed-integer nonlinear programs by outer approximation. MathematicalProgramming 66: 327–349.8. Gams Development Corp. (2003). Gams – The Solver Manuals. GAMS Development Corp., Washington, DC.9. Geoffrion, A. M. (1972). Generalized Benders decomposition. Journal of Optimization Theory and Applications 10:4 237–260.10. Grossmann, I. (2002). Review of Nonlinear Mixed-Integer and Disjunctive Programming Techniques. Optimization andEngineering 3: 227–252.11. Leyffer, S. (1993). Deterministic Methods for Mixed Integer Nonlinear Programming. PhD Thesis, University of Dundee,United Kingdom.12. Leyffer, S. (1999). User Manual for MINLP BB, University of Dundee13. Leyffer, S. (2001). Integrating SQP and branch and bound for mixed integer nonlinear programming. ComputationalOptimization and Applications 18: 295–309.14. Leyffer, S., Sartenaer, A., Wanufelle, E. (2008). Branch-and-Refine for Mixed Integer Nonconvex Global Optimization.Preprint ANL/MCS-P1547-0908, Mathematics and Computer Science Division, Argonne National Laboratory.15. Leyffer, S., Linderoth, J., Luedtke, J., Miller, A., Munson, T. (2009). Applications and Algorithms for Mixed Integer NonlinearProgramming. Preprint ANL/MCS-P1630-0509, Mathematics and Computer Science Division, Argonne National Laboratory.16. Nowak, I., Vigerske, S. (2008). LaGO: a (heuristic) branch and cut algorithm for nonconvex MINLPs. Central EuropeanJournal of Operations Research 16: 127–138.17. Powell, M. J. C. (2004). Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. MathematicalProgramming 100: 183–215.18. Quesada, I., Grossmann, I. E. (1992). An LP/NLP based branch and bound algorithm for convex MINLP optimizationproblems. Computers and Chemical Engineering 16:2 937–947.19. Westerlund, T., Pettersson, F. (1995). An Extended cutting plane method for solving convex MINL problems.Computers andChemical Engineering 19: 131–136.

Overview on mixed integer nonlinear programming problems

Costa, M. Fernanda P.

Fernandes, Edite Manuela da G. P.

Universidade do Minho: RepositoriUM

http://bibliotecadigital.ipb.pt/bitstream/10198/1565/1/Overview%20on%20MINLP_ICNAAM2009.pdf

Overview on mixed integer nonlinear programming problems

Abstract

Similar works

Full text

Available Versions

Crossref

Crossref

Crossref

Crossref

MUCC (Crossref)

Crossref

Crossref

Biblioteca Digital do IPB

Universidade do Minho: RepositoriUM