We discuss two models from the literature that have been developed to formulate piecewise
linear approximation of separable nonlinear functions by way of mixed-integer programs. We
show that the most commonly proposed method is computationally inferior to a lesser known
technique by comparing analytically the linear programming relaxations of the two formulations.
A third way of formulating the problem, that shares the advantages of the better of the two
known methods, is also proposed.Statistics Working Papers Serie

Padberg, M.

We discuss two models from the literature that have been developed to formulate piecewise linear approximation of separable nonlinear functions by way of mixed-integer programs. We show that the most commonly proposed method is computationally inferior to a lesser known technique by comparing analytically the linear programming relaxations of the two formulations. A third way of formulating the problem, that shares the advantages of the better of the two known methods, is also proposed

M. Padberg

CiteSeerX

APPROXIMATING SEPARABLE NONLINEAR FUNCTIONS VIA MIXED ZERO-ONE PROGRAMS

New York University Faculty Digital Archive

ISTITUTO DI ANALISI DEI SISTEMI ED INFORMATICACONSIGLIO NAZIONALE DELLE RICERCHEM. PadbergAPPROXIMATINGSEPARABLE NONLINEAR FUNCTIONSVIA MIXED ZERO-ONE PROGRAMSR. 476 Ottobre 1998Manfred Padberg { Stern School of Business, New York University, 44 West 4th Street -New York, New York 10012, USA.Supported in part by a grant from the Oce of Naval Research (N00014-96-0327). Work done in partwhile visiting IASI-CNR in Rome, Italy, and the Forschungsinstitut fur Diskrete Mathematik, UniversitatBonn, Germany.Istituto di Analisi dei Sistemi ed Informatica, CNRviale Manzoni 3000185 ROMA, Italytel. ++39-06-77161fax ++39-06-7716461email: iasi@iasi.rm.cnr.itURL: http://www.iasi.rm.cnr.itAbstractWe discuss two models from the literature that have been developed to formulate piecewiselinear approximation of separable nonlinear functions by way of mixed-integer programs. Weshow that the most commonly proposed method is computationally inferior to a lesser knowntechnique by comparing analytically the linear programming relaxations of the two formulations.A third way of formulating the problem, that shares the advantages of the better of the twoknown methods, is also proposed.3.IntroductionApplications of linear programming technology often require the modeling of nonlinearities inthe objective function or in some of the constraints of an otherwise linear optimization model.Such nonlinearities may come about due to economies or diseconomies of scale, \kinked" de-mand or production cost curves, etc. Already in the early 1950's it has been recognized that suchoccurrences can be dealt with adequately by approximating nonlinearities by piecewise linearfunctions and modeling these in a mixed-integer framework by introducing new 0-1 variables;see e.g. Balinski and Spielberg (1969) for an overview and historical references. Most text-books in Operations Research/Integer Programming, see e.g. Nemhauser and Wolsey (1988),Wagner (1969) and others, oer one or two possibilities of expressing piecewise linear approxi-mations of separable nonlinear functions in this manner.The two classical formulations (Model I and II, below) can be found e.g. in Dantzig (1963). Weare, however, not aware of a discussion of the quality or thightness of the various formulationsthat have been proposed a long while ago. Such considerations play indeed an essential rolewhen the resulting mixed zero-one program is subsequently solved by branch-and-bound orbranch-and-cut using linear programming algorithms in the solution process.Besides reviewing the two classical formulations, the issue of the quality of the formulation iswhat we address here. We show that an analytical comparison of the two dierent formulationsof the problem reveals that one of them is always inferior to the other, i.e., that the linearprogramming relaxation produces always worse bounds in one case than in the other. We thenpropose a third way of formulating piecewise linear approximation via a mixed zero-one programthat shares the (local) properties of the better of the two classical formulations.Let (x1; : : : ; xn) be any separable, nonlinear function from Rninto R. Separability meansthat we assume that the function can be written as(x1; : : : ; xn) =nXj=1j(xj) ; (1)aj0bj0aj1bj1aj2bj2aj3bj3aj4bj4aj5= ajubj5xjj(xj)Figure 1: Piecewise linear approximationwhere each j(xj) maps R into R. Given -nite intervals [aj0; aju] for each variable xjwherej 2 f1; : : : ; ng we approximate each j(xj) bya piecewise linear functionbj(xj) over this in-terval. To do so we choose a partitioning aj0<aj1< aj2< : : : < ajkj= ajuof the interval [aj0; aju]for each j 2 f1; : : : ; ng, see Figure 1. It is well-known that by rening the partitioning, i.e., bychoosing kjlarge enough and the distance be-tween any two consecutive points of the par-titioning small enough, we can {under certaintechnical conditions{ approximate j(xj) arbi-trarily closely by such piecewise linear functions.We denote bybj`= j(aj`) for 0  `  kj; (2)the function values at the points aj`which we can calculate where j 2 f1; : : : ; ng. In the followingwe drop the index j for notational convenience because we will consider a single term of the right-hand of (1) only. Of course, as there are typically constraints linking the variables x1; : : : ; xn4.our proceeding is a \local" analysis. What are looking for is a \locally ideal" formulation of theapproximation problem, i.e., a formulation that in the absence of other constraints models theproblem perfectly; see Padberg and Rijal (1996) for more detail.1. The rst modelIn Model I we write for the single, continuous variable xx = a0+ y1+ : : :+ yk:We require that each y`is a continuous variable satisfying(i) 0  y` a`  a` 1for 1  `  kand moreover, the following dichotomy:(ii) either yi= ai  ai 1for 1  i  ` or y`+1= 0 for 1  `  k   1:Assuming that this can be \formulated" conveniently, we then get the piecewise linear approxi-mation for any term of the right-hand side of (1) by way ofb(x) = b0+b1  b0a1  a0y1+b2  b1a2  a1y2+ : : :+bk  bk 1ak  ak 1yk:From (i) it follows that (ii) can be replaced by the requirement(ii0) either yi ai  ai 1for 1  i  ` or y`+1 0 for 1  `  k   1:To formulate this in linear inequalities using integer variables we introduce zero-one variables z`and consider the mixed zero-one modelx = a0+kX`=1y`;b(x) = b0+kX`=1b`  b` 1a`  a` 1y`; (3)y1 a1  a0; yk 0 ; (4)y` (a`  a` 1)z`; y`+1 (a`+1  a`)z`for 1  `  k   1 ; (5)where z`2 f0; 1g for 1  `  k   1 are the \new" 0-1 variables. For k = 1 there is no needfor a zero-one variable and (3), (4) describe the linear approximation correctly. For k = 2 thecorrectness follows by examining the two cases where z1= 0 and z1= 1, respectively. Thecorrectness of the mixed zero-one model (3), : : : , (5) for the piecewise linear approximation ofa nonlinear function follows by induction on k.It follows from (5) that every solution to (4) and (5) satises automatically1  z1 z2 : : :  zk 1 0;thus the upper and lower bounds on the 0-1 variables are not required in the formulation. Ina computer model, however, we would declare the variables z`to be \binary" variables ratherthan general \integer" variables. A similar remark applies to the 0-1 variables of the second andthird model below. In Theorem 1 (below) we prove that Model I is a locally ideal formulationfor piecewise linear approximation.5.2. The second modelIn Model II {which is the only one that one nds e.g. in Nemhauser and Wolsey (1988){ weexploit the fact that given a partitioning a0< a1< : : : < ak= auevery real x 2 [a0; au] canbe written uniquely as a convex combination of at most two consecutive points a`; a`+1of thepartitioning. Thus we write for the continuous variable xx = a00+ a11+ : : :+ akk;where we require that the continuous variables `satisfy(i)kX`=0`= 1 ; ` 0 for 0  `  k;(ii) at most two consecutive `and `+1; say, are positive:If the requirement (ii) can be expressed conveniently using integer variables, we then get thepiecewise linear approximation for any term of the right-hand side of (1) by way ofb(x) = b00+ b11+ : : :+ bkk:To formulate (i) and (ii) as the set of solutions to a mixed zero-one program we introduce 0-1variables `for 0  `  k   1 and consider the modelx =kX`=0a``;b(x) =kX`=0b``; (6)0  0 0; 0  ` ` 1+ `for 1  `  k   1 ; 0  k k 1; (7)kX`=0`= 1 ;k 1X`=0`= 1 ; (8)` 0 for 1  `  k   2 ; (9)where `2 f0; 1g for 0  `  k   1 are the \new" 0-1 variables. Note that the nonnegativity of0and k 1is implied by (7). For k = 1 the formulation (6), : : : , (9) of the problem at hand isevidently correct. The correctness of Model II for arbitrary k  1 follows inductively.3. Comparison of Model I and Model IIIn the following we assume that k  3, because for k  2 either model is locally ideal. ModelI has k real variables and k   1 0-1 variables, while Model II has k + 1 real variables and k 0-1variables. To compare the two models we use the equations (8) to eliminate 0and 0from theformulation. Using the variable transformation for the remaining continuous variablesy`= (a`  a` 1)kXj=`jfor 1  `  kand its inverse mapping that we calculate to bej=yjaj  aj 1 yj+1aj+1  ajfor 1  j  k   1 ; k=1ak  ak 1yk;6.we obtain the following equivalent formulation of Model II :x = a0+kX`=1y`;b(x) = b0+kX`=1b`  b` 1a`  a` 1y`; (10)y1 a1  a0; y1 (a1  a0)k 1X`=1`; (11)(a`  a` 1)y`+1 (a`+1  a`)y`for 1  `  k   1 ; (12)y1a1  a0 y2a2  a1 1 k 1X`=2`;y`a`  a` 1 y`+1a`+1  a` ` 1+ `for 2  `  k   1; (13)yk 0 ; yk (ak  ak 1)k 1; (14)` 0 for 1  `  k   2 ; (15)where `2 f0; 1g for 1  `  k   1. Note that (11) implies thatPk 1`=1` 1 and thusPk 1`=j` 1 for all 1  j  k   1 and feasible 0-1 values `, 1  `  k   1. Using the variablesubstitutionzj=k 1X`=j`for 1  j  k   1 ;which is integrality preserving because its inverse is given byj= zj  zj+1for 1  j  k   2 ; k 1= zk 1;the above constraints (11), : : : , (15) can be written equivalently as follows:y1 a1  a0; y1 (a1  a0)z1; (16)(a`  a` 1)y`+1 (a`+1  a`)y`for 1  `  k   1 ; (17)y1a1  a0 y2a2  a1 1  z2;y`a`  a` 1 y`+1a`+1  a` z` 1  z`+1for 2  `  k   1; (18)yk 0 ; yk (ak  ak 1)zk 1; (19)z`  z`+1 0 ; for 1  `  k   2 ; (20)where for ` = k   1 we simply let zk= 0 in (18) and the integer variables z`are 0-1 valued for1  `  k 1. It follows that the (equivalently) changed Model II has now the same variable setas Model I and we are in the position to compare the two formulations in the context of a linearprogramming based approach to the solution of the corresponding mixed-integer programmingproblem. Note that like in Model I the constraints (16), (19) and (20) imply that every feasiblesolution to (16), : : : , (20) automatically satises 1  z1 z2 : : :  zk 1 0.We denote the linear programming (LP) relaxation of Model I byFILP= f(y; z) 2 R2k 1: (y; z) satises (4) and (5)g: (21)Likewise we denote the LP relaxation of the (equivalently) changed Model II byFIILP= f(y; z) 2 R2k 1: (y; z) satises (16); : : : ; (20)g: (22)It is an immediate consequence of the respective formulations that both FILPand FIILParebounded subsets and thus polytopes in R2k 1.7.Theorem 1. (i) Model I is locally ideal, i.e., z 2 f0;1gk 1for all extreme points in FILP.(ii) FILPis properly contained in FIILP. FIILPhas extreme points (y; z) with z 62 f0;1gk 1.Proof. We scale the continuous variables of Model I by introducing new variablesy0`= y`=(a`  a` 1) for 1  `  k: (23)The constraint set dening FILPcan thus be written asy01 1; y0k 0; y0` z`; y0`+1 z`for 1  `  k   1: (24)It follows that the constraint matrix given by (24) is totally unimodular and hence by Cramer'srule every extreme point of the feasible given by (24) has all components equal to zero or one.This implies (i).(ii) Let (y; z) 2 FILP, i.e., (y; z) satises (4) and (5). Then (y; z) satises (16) and (19)trivially. From (5) and a`  a` 1> 0 for all 1  `  k we calculate(a`+1  a`)y` (a`+1  a`)(a`  a` 1)z` (a`  a` 1)y`+1for 1  `  k   1 and thus (17) is satised. From (4) and (5) we have y1 a1  a0and y2(a2 a1)z2and thus the rst relation of (18) follows. Again from (5) we have y` (a` a` 1)z` 1and y`+1 (a`+1  a`)z`+1for all 2  `  k   1, where zk= 0, and thus combining thetwo inequalities we see that (18) is satised. As we have noted in the discussion of Model Ievery (y; z) 2 FILPsatises 1  z1 z2 : : :  zk 1 0 and thus (20) is satised as well.Consequently, (y; z) 2 FIILPand thus FILP FIILP. Let (y; z) 2 FIILPbe such that z 2 f0;1gk 1.It follows from (16), : : : , (20) that yi= ai  ai 1for i = 1; : : : ; h, 0  yh+1 ah+1  ah, yi= 0for i = h+2; : : : ; k, zi= 1 for i = 1; : : : ; h, zi= 0 for i = h+1; : : : ; k  1 where 0  h  k  1.From (4) and (5) thus (y; z) 2 FILP. Now consider (y; z) given by y1= (a1  a0)=2, yj= 0 for2  j  k, z1= z2= 1=2, and zj= 0 for 3  j  k   1. It follows that (y; z) satises (16),: : : , (20), i.e., (y; z) 2 FIILP. But (y; z) violates the constraint y2 (a2  a1)z2of Model I andthus (y; z) 62 FILP. Since FIILPis a polytope, it follows that it has extreme points (y; z) withz 62 f0;1gk 1.By Theorem 1 FIILPhas extreme points with fractional components for z and indeed it hasmany such extreme points. It is not overly dicult to characterize all of them, which we leave asa good exercise for graduate students. It is amazing that most textbooks treat only Model II inthe context of using mixed-integer programming to approximate separable nonlinear functionsby piecewise linear ones.Model I, which has been known since the 1950's, is locally far better than Model II since all ofits extreme points (y; z) satisfy z 2 f0;1gk 1. Of course, this does not mean that the \overall"model {of which the piecewise linear approximation is but a part{ has the same property. Butthe proper inclusion FILP FIILPshows that the linear programming bound obtained from usingModel I must always be equal to or better than the one obtained from Model II in any case, i.e.,even in the worst case.It is now an easy exercise to derive ex post a formulation of Model II in  and  variablesthat guarantees the same outcome as Model I. This may sometimes be desirable because allvariables of Model II assume values between zero and one. (The same eect can, of course, alsobe obtained by scaling the continuous variables of Model I like we did in the proof of Theorem 1;see (23).) We leave it as an exercise to prove that the following Model III is a correct formulation,8.which is obtained from Model I by reversing the various transformations that we have used toanalyze Model II.x =kX`=0a``;b(x) =kX`=0b``; (25)kX`=0`= 1;k 1X`=0`= 1; (26)k 1Xj=`jkXj=`+1jk 1Xj=`+1jfor 1  `  k   2; (27)0  0 o; 0  k k 1; (28)where `2 f0; 1g for 0  `  k   1.Evidently, Model III has at rst sight little resemblance to the original Model II except thatthe same set of variables is used. More precisely, letPLP= f(;) 2 R2k+1: (;) satises (7), (8), (9)g ;P#LP= f(;) 2 R2k+1: (;) satises (26), (27), (28)g ;be the linear programming relaxation of Model II and III, respectively. It follows thatP#LP PLPand P#LP= conv (PLP\ (Rk+1 Zk)) :By construction, Model III shares locally the property of Model I of having all its extremepoints (;) satisfy  2 f0; 1gk. Model III can be used in lieu of Model I, but Model II shoulddenitely be abandoned despite its popularity in the textbooks. Model II just happens to bea poor formulation for the piecewise linear approximation problem when linear programmingmethods are used.References[1] Balinski, M.L. and K. Spielberg (1969). \Methods for integer programming: Algebraic,combinatorial and enumerative", in Aronofsky (ed.) Progress in Operations Research,Wiley, New York, 195-292.[2] Dantzig, G.B. (1963). Linear Programming and Extensions, Princeton U. Press, Princeton,New Jersey.[3] Nemhauser, G. and L. Wolsey (1988). Integer and Combinatorial Optimization, Wiley,New York.[4] Padberg, M. (1995). Linear Optimization and Extensions, Springer Verlag, Heidelberg.[5] Padberg, M. and M. Rijal (1996). Location, Scheduling, Design and Integer Programming,Kluwer Academic Publishers, Boston.[6] Wagner, H.M. (1969). Principles of Operations Research, Prentice-Hall, Englewood Clis,New Jersey.

DSpace at New York University

ISTITUTO DI ANALISI DEI SISTEMI ED INFORMATICA
CONSIGLIO NAZIONALE DELLE RICERCHE
M. Padberg
APPROXIMATING
SEPARABLE NONLINEAR FUNCTIONS
VIA MIXED ZERO-ONE PROGRAMS
R. 476 Ottobre 1998
Manfred Padberg { Stern School of Business, New York University, 44 West 4th Street -
New York, New York 10012, USA.
Supported in part by a grant from the Oce of Naval Research (N00014-96-0327). Work done in part
while visiting IASI-CNR in Rome, Italy, and the Forschungsinstitut fur Diskrete Mathematik, Universitat
Bonn, Germany.
Istituto di Analisi dei Sistemi ed Informatica, CNR
viale Manzoni 30
00185 ROMA, Italy
tel. ++39-06-77161
fax ++39-06-7716461
email: iasi@iasi.rm.cnr.it
URL: http://www.iasi.rm.cnr.it
Abstract
We discuss two models from the literature that have been developed to formulate piecewise
linear approximation of separable nonlinear functions by way of mixed-integer programs. We
show that the most commonly proposed method is computationally inferior to a lesser known
technique by comparing analytically the linear programming relaxations of the two formulations.
A third way of formulating the problem, that shares the advantages of the better of the two
known methods, is also proposed.

3.
Introduction
Applications of linear programming technology often require the modeling of nonlinearities in
the objective function or in some of the constraints of an otherwise linear optimization model.
Such nonlinearities may come about due to economies or diseconomies of scale, \kinked" de-
mand or production cost curves, etc. Already in the early 1950's it has been recognized that such
occurrences can be dealt with adequately by approximating nonlinearities by piecewise linear
functions and modeling these in a mixed-integer framework by introducing new 0-1 variables;
see e.g. Balinski and Spielberg (1969) for an overview and historical references. Most text-
books in Operations Research/Integer Programming, see e.g. Nemhauser and Wolsey (1988),
Wagner (1969) and others, oer one or two possibilities of expressing piecewise linear approxi-
mations of separable nonlinear functions in this manner.
The two classical formulations (Model I and II, below) can be found e.g. in Dantzig (1963). We
are, however, not aware of a discussion of the quality or thightness of the various formulations
that have been proposed a long while ago. Such considerations play indeed an essential role
when the resulting mixed zero-one program is subsequently solved by branch-and-bound or
branch-and-cut using linear programming algorithms in the solution process.
Besides reviewing the two classical formulations, the issue of the quality of the formulation is
what we address here. We show that an analytical comparison of the two dierent formulations
of the problem reveals that one of them is always inferior to the other, i.e., that the linear
programming relaxation produces always worse bounds in one case than in the other. We then
propose a third way of formulating piecewise linear approximation via a mixed zero-one program
that shares the (local) properties of the better of the two classical formulations.
Let (x
1
; : : : ; x
n
) be any separable, nonlinear function from R
n
into R. Separability means
that we assume that the function can be written as
(x
1
; : : : ; x
n
) =
n
X
j=1

j
(x
j
) ; (1)
a
j
0
b
j
0
a
j
1
b
j
1
a
j
2
b
j
2
a
j
3
b
j
3
a
j
4
b
j
4
a
j
5
= a
j
u
b
j
5
x
j

j
(x
j
)
Figure 1: Piecewise linear approximation
where each 
j
(x
j
) maps R into R. Given -
nite intervals [a
j
0
; a
j
u
] for each variable x
j
where
j 2 f1; : : : ; ng we approximate each 
j
(x
j
) by
a piecewise linear function
b

j
(x
j
) over this in-
terval. To do so we choose a partitioning a
j
0
<
a
j
1
< a
j
2
< : : : < a
j
k
j
= a
j
u
of the interval [a
j
0
; a
j
u
]
for each j 2 f1; : : : ; ng, see Figure 1. It is well-
known that by rening the partitioning, i.e., by
choosing k
j
large enough and the distance be-
tween any two consecutive points of the par-
titioning small enough, we can {under certain
technical conditions{ approximate 
j
(x
j
) arbi-
trarily closely by such piecewise linear functions.
We denote by
b
j
`
= 
j
(a
j
`
) for 0  `  k
j
; (2)
the function values at the points a
j
`
which we can calculate where j 2 f1; : : : ; ng. In the following
we drop the index j for notational convenience because we will consider a single term of the right-
hand of (1) only. Of course, as there are typically constraints linking the variables x
1
; : : : ; x
n
4.
our proceeding is a \local" analysis. What are looking for is a \locally ideal" formulation of the
approximation problem, i.e., a formulation that in the absence of other constraints models the
problem perfectly; see Padberg and Rijal (1996) for more detail.
1. The rst model
In Model I we write for the single, continuous variable x
x = a
0
+ y
1
+ : : :+ y
k
:
We require that each y
`
is a continuous variable satisfying
(i) 0  y
`
 a
`
  a
` 1
for 1  `  k
and moreover, the following dichotomy:
(ii) either y
i
= a
i
  a
i 1
for 1  i  ` or y
`+1
= 0 for 1  `  k   1:
Assuming that this can be \formulated" conveniently, we then get the piecewise linear approxi-
mation for any term of the right-hand side of (1) by way of
b
(x) = b
0
+
b
1
  b
0
a
1
  a
0
y
1
+
b
2
  b
1
a
2
  a
1
y
2
+ : : :+
b
k
  b
k 1
a
k
  a
k 1
y
k
:
From (i) it follows that (ii) can be replaced by the requirement
(ii
0
) either y
i
 a
i
  a
i 1
for 1  i  ` or y
`+1
 0 for 1  `  k   1:
To formulate this in linear inequalities using integer variables we introduce zero-one variables z
`
and consider the mixed zero-one model
x = a
0
+
k
X
`=1
y
`
;
b
(x) = b
0
+
k
X
`=1
b
`
  b
` 1
a
`
  a
` 1
y
`
; (3)
y
1
 a
1
  a
0
; y
k
 0 ; (4)
y
`
 (a
`
  a
` 1
)z
`
; y
`+1
 (a
`+1
  a
`
)z
`
for 1  `  k   1 ; (5)
where z
`
2 f0; 1g for 1  `  k   1 are the \new" 0-1 variables. For k = 1 there is no need
for a zero-one variable and (3), (4) describe the linear approximation correctly. For k = 2 the
correctness follows by examining the two cases where z
1
= 0 and z
1
= 1, respectively. The
correctness of the mixed zero-one model (3), : : : , (5) for the piecewise linear approximation of
a nonlinear function follows by induction on k.
It follows from (5) that every solution to (4) and (5) satises automatically
1  z
1
 z
2
 : : :  z
k 1
 0;
thus the upper and lower bounds on the 0-1 variables are not required in the formulation. In
a computer model, however, we would declare the variables z
`
to be \binary" variables rather
than general \integer" variables. A similar remark applies to the 0-1 variables of the second and
third model below. In Theorem 1 (below) we prove that Model I is a locally ideal formulation
for piecewise linear approximation.
5.
2. The second model
In Model II {which is the only one that one nds e.g. in Nemhauser and Wolsey (1988){ we
exploit the fact that given a partitioning a
0
< a
1
< : : : < a
k
= a
u
every real x 2 [a
0
; a
u
] can
be written uniquely as a convex combination of at most two consecutive points a
`
; a
`+1
of the
partitioning. Thus we write for the continuous variable x
x = a
0

0
+ a
1

1
+ : : :+ a
k

k
;
where we require that the continuous variables 
`
satisfy
(i)
k
X
`=0

`
= 1 ; 
`
 0 for 0  `  k;
(ii) at most two consecutive 
`
and 
`+1
; say, are positive:
If the requirement (ii) can be expressed conveniently using integer variables, we then get the
piecewise linear approximation for any term of the right-hand side of (1) by way of
b
(x) = b
0

0
+ b
1

1
+ : : :+ b
k

k
:
To formulate (i) and (ii) as the set of solutions to a mixed zero-one program we introduce 0-1
variables 
`
for 0  `  k   1 and consider the model
x =
k
X
`=0
a
`

`
;
b
(x) =
k
X
`=0
b
`

`
; (6)
0  
0
 
0
; 0  
`
 
` 1
+ 
`
for 1  `  k   1 ; 0  
k
 
k 1
; (7)
k
X
`=0

`
= 1 ;
k 1
X
`=0

`
= 1 ; (8)

`
 0 for 1  `  k   2 ; (9)
where 
`
2 f0; 1g for 0  `  k   1 are the \new" 0-1 variables. Note that the nonnegativity of

0
and 
k 1
is implied by (7). For k = 1 the formulation (6), : : : , (9) of the problem at hand is
evidently correct. The correctness of Model II for arbitrary k  1 follows inductively.
3. Comparison of Model I and Model II
In the following we assume that k  3, because for k  2 either model is locally ideal. Model
I has k real variables and k   1 0-1 variables, while Model II has k + 1 real variables and k 0-1
variables. To compare the two models we use the equations (8) to eliminate 
0
and 
0
from the
formulation. Using the variable transformation for the remaining continuous variables
y
`
= (a
`
  a
` 1
)
k
X
j=`

j
for 1  `  k
and its inverse mapping that we calculate to be

j
=
y
j
a
j
  a
j 1
 
y
j+1
a
j+1
  a
j
for 1  j  k   1 ; 
k
=
1
a
k
  a
k 1
y
k
;
6.
we obtain the following equivalent formulation of Model II :
x = a
0
+
k
X
`=1
y
`
;
b
(x) = b
0
+
k
X
`=1
b
`
  b
` 1
a
`
  a
` 1
y
`
; (10)
y
1
 a
1
  a
0
; y
1
 (a
1
  a
0
)
k 1
X
`=1

`
; (11)
(a
`
  a
` 1
)y
`+1
 (a
`+1
  a
`
)y
`
for 1  `  k   1 ; (12)
y
1
a
1
  a
0
 
y
2
a
2
  a
1
 1 
k 1
X
`=2

`
;
y
`
a
`
  a
` 1
 
y
`+1
a
`+1
  a
`
 
` 1
+ 
`
for 2  `  k   1; (13)
y
k
 0 ; y
k
 (a
k
  a
k 1
)
k 1
; (14)

`
 0 for 1  `  k   2 ; (15)
where 
`
2 f0; 1g for 1  `  k   1. Note that (11) implies that
P
k 1
`=1

`
 1 and thus
P
k 1
`=j

`
 1 for all 1  j  k   1 and feasible 0-1 values 
`
, 1  `  k   1. Using the variable
substitution
z
j
=
k 1
X
`=j

`
for 1  j  k   1 ;
which is integrality preserving because its inverse is given by

j
= z
j
  z
j+1
for 1  j  k   2 ; 
k 1
= z
k 1
;
the above constraints (11), : : : , (15) can be written equivalently as follows:
y
1
 a
1
  a
0
; y
1
 (a
1
  a
0
)z
1
; (16)
(a
`
  a
` 1
)y
`+1
 (a
`+1
  a
`
)y
`
for 1  `  k   1 ; (17)
y
1
a
1
  a
0
 
y
2
a
2
  a
1
 1  z
2
;
y
`
a
`
  a
` 1
 
y
`+1
a
`+1
  a
`
 z
` 1
  z
`+1
for 2  `  k   1; (18)
y
k
 0 ; y
k
 (a
k
  a
k 1
)z
k 1
; (19)
z
`
  z
`+1
 0 ; for 1  `  k   2 ; (20)
where for ` = k   1 we simply let z
k
= 0 in (18) and the integer variables z
`
are 0-1 valued for
1  `  k 1. It follows that the (equivalently) changed Model II has now the same variable set
as Model I and we are in the position to compare the two formulations in the context of a linear
programming based approach to the solution of the corresponding mixed-integer programming
problem. Note that like in Model I the constraints (16), (19) and (20) imply that every feasible
solution to (16), : : : , (20) automatically satises 1  z
1
 z
2
 : : :  z
k 1
 0.
We denote the linear programming (LP) relaxation of Model I by
F
I
LP
= f(y; z) 2 R
2k 1
: (y; z) satises (4) and (5)g: (21)
Likewise we denote the LP relaxation of the (equivalently) changed Model II by
F
II
LP
= f(y; z) 2 R
2k 1
: (y; z) satises (16); : : : ; (20)g: (22)
It is an immediate consequence of the respective formulations that both F
I
LP
and F
II
LP
are
bounded subsets and thus polytopes in R
2k 1
.
7.
Theorem 1. (i) Model I is locally ideal, i.e., z 2 f0;1g
k 1
for all extreme points in F
I
LP
.
(ii) F
I
LP
is properly contained in F
II
LP
. F
II
LP
has extreme points (y; z) with z 62 f0;1g
k 1
.
Proof. We scale the continuous variables of Model I by introducing new variables
y
0
`
= y
`
=(a
`
  a
` 1
) for 1  `  k: (23)
The constraint set dening F
I
LP
can thus be written as
y
0
1
 1; y
0
k
 0; y
0
`
 z
`
; y
0
`+1
 z
`
for 1  `  k   1: (24)
It follows that the constraint matrix given by (24) is totally unimodular and hence by Cramer's
rule every extreme point of the feasible given by (24) has all components equal to zero or one.
This implies (i).
(ii) Let (y; z) 2 F
I
LP
, i.e., (y; z) satises (4) and (5). Then (y; z) satises (16) and (19)
trivially. From (5) and a
`
  a
` 1
> 0 for all 1  `  k we calculate
(a
`+1
  a
`
)y
`
 (a
`+1
  a
`
)(a
`
  a
` 1
)z
`
 (a
`
  a
` 1
)y
`+1
for 1  `  k   1 and thus (17) is satised. From (4) and (5) we have y
1
 a
1
  a
0
and y
2

(a
2
 a
1
)z
2
and thus the rst relation of (18) follows. Again from (5) we have y
`
 (a
`
 a
` 1
)z
` 1
and y
`+1
 (a
`+1
  a
`
)z
`+1
for all 2  `  k   1, where z
k
= 0, and thus combining the
two inequalities we see that (18) is satised. As we have noted in the discussion of Model I
every (y; z) 2 F
I
LP
satises 1  z
1
 z
2
 : : :  z
k 1
 0 and thus (20) is satised as well.
Consequently, (y; z) 2 F
II
LP
and thus F
I
LP
 F
II
LP
. Let (y; z) 2 F
II
LP
be such that z 2 f0;1g
k 1
.
It follows from (16), : : : , (20) that y
i
= a
i
  a
i 1
for i = 1; : : : ; h, 0  y
h+1
 a
h+1
  a
h
, y
i
= 0
for i = h+2; : : : ; k, z
i
= 1 for i = 1; : : : ; h, z
i
= 0 for i = h+1; : : : ; k  1 where 0  h  k  1.
From (4) and (5) thus (y; z) 2 F
I
LP
. Now consider (y; z) given by y
1
= (a
1
  a
0
)=2, y
j
= 0 for
2  j  k, z
1
= z
2
= 1=2, and z
j
= 0 for 3  j  k   1. It follows that (y; z) satises (16),
: : : , (20), i.e., (y; z) 2 F
II
LP
. But (y; z) violates the constraint y
2
 (a
2
  a
1
)z
2
of Model I and
thus (y; z) 62 F
I
LP
. Since F
II
LP
is a polytope, it follows that it has extreme points (y; z) with
z 62 f0;1g
k 1
.
By Theorem 1 F
II
LP
has extreme points with fractional components for z and indeed it has
many such extreme points. It is not overly dicult to characterize all of them, which we leave as
a good exercise for graduate students. It is amazing that most textbooks treat only Model II in
the context of using mixed-integer programming to approximate separable nonlinear functions
by piecewise linear ones.
Model I, which has been known since the 1950's, is locally far better than Model II since all of
its extreme points (y; z) satisfy z 2 f0;1g
k 1
. Of course, this does not mean that the \overall"
model {of which the piecewise linear approximation is but a part{ has the same property. But
the proper inclusion F
I
LP
 F
II
LP
shows that the linear programming bound obtained from using
Model I must always be equal to or better than the one obtained from Model II in any case, i.e.,
even in the worst case.
It is now an easy exercise to derive ex post a formulation of Model II in  and  variables
that guarantees the same outcome as Model I. This may sometimes be desirable because all
variables of Model II assume values between zero and one. (The same eect can, of course, also
be obtained by scaling the continuous variables of Model I like we did in the proof of Theorem 1;
see (23).) We leave it as an exercise to prove that the following Model III is a correct formulation,
8.
which is obtained from Model I by reversing the various transformations that we have used to
analyze Model II.
x =
k
X
`=0
a
`

`
;
b
(x) =
k
X
`=0
b
`

`
; (25)
k
X
`=0

`
= 1;
k 1
X
`=0

`
= 1; (26)
k 1
X
j=`

j

k
X
j=`+1

j

k 1
X
j=`+1

j
for 1  `  k   2; (27)
0  
0
 
o
; 0  
k
 
k 1
; (28)
where 
`
2 f0; 1g for 0  `  k   1.
Evidently, Model III has at rst sight little resemblance to the original Model II except that
the same set of variables is used. More precisely, let
P
LP
= f(;) 2 R
2k+1
: (;) satises (7), (8), (9)g ;
P
#
LP
= f(;) 2 R
2k+1
: (;) satises (26), (27), (28)g ;
be the linear programming relaxation of Model II and III, respectively. It follows that
P
#
LP
 P
LP
and P
#
LP
= conv (P
LP
\ (R
k+1
 Z
k
)) :
By construction, Model III shares locally the property of Model I of having all its extreme
points (;) satisfy  2 f0; 1g
k
. Model III can be used in lieu of Model I, but Model II should
denitely be abandoned despite its popularity in the textbooks. Model II just happens to be
a poor formulation for the piecewise linear approximation problem when linear programming
methods are used.
References
[1] Balinski, M.L. and K. Spielberg (1969). \Methods for integer programming: Algebraic,
combinatorial and enumerative", in Aronofsky (ed.) Progress in Operations Research,
Wiley, New York, 195-292.
[2] Dantzig, G.B. (1963). Linear Programming and Extensions, Princeton U. Press, Princeton,
New Jersey.
[3] Nemhauser, G. and L. Wolsey (1988). Integer and Combinatorial Optimization, Wiley,
New York.
[4] Padberg, M. (1995). Linear Optimization and Extensions, Springer Verlag, Heidelberg.
[5] Padberg, M. and M. Rijal (1996). Location, Scheduling, Design and Integer Programming,
Kluwer Academic Publishers, Boston.
[6] Wagner, H.M. (1969). Principles of Operations Research, Prentice-Hall, Englewood Clis,
New Jersey.


http://archive.nyu.edu/bitstream/2451/14785/1/SOR-99-4.pdf

APPROXIMATING SEPARABLE NONLINEAR FUNCTIONS VIA MIXED ZERO-ONE PROGRAMS

Abstract

Similar works

Full text

Available Versions

CiteSeerX

New York University Faculty Digital Archive

DSpace at New York University