n this technical note, an algorithm for binary quadratic programs defined by matrices with band structure is proposed. It was shown in the article by T. Alamo, D. M. de la Pentildea, D. Limon, and E. F. Camacho, ldquoConstrained min-max predictive control: modifications of the objective function leading to polynomial complexity,rdquo IEEE Tran. Autom. Control , vol. 50, pp. 710-714, May 2005, that this class of problems arise in robust model predictive control when min-max techniques are applied. Although binary quadratic problems belongs to a class of NP-complete problems, the computational burden of the proposed maximization algorithm for band matrices is polynomial with the dimension of the optimization variable and exponential with the band size. Computational results and comparisons on several hundred test problems demonstrate the efficiency of the algorithm

Alamo, Teodoro

Camacho, Eduardo F.

Muñoz de la Peña Sequedo, David

English

In this technical note, an algorithm for binary quadratic programs defined by matrices with band structure is proposed. It was shown in the article by T. Alamo, D. M. de la Pentildea, D. Limon, and E. F. Camacho, ldquoConstrained min-max predictive control: modifications of the objective function leading to polynomial complexity,rdquo IEEE Tran. Autom. Control , vol. 50, pp. 710-714, May 2005, that this class of problems arise in robust model predictive control when min-max techniques are applied. Although binary quadratic problems belongs to a class of NP-complete problems, the computational burden of the proposed maximization algorithm for band matrices is polynomial with the dimension of the optimization variable and exponential with the band size. Computational results and comparisons on several hundred test problems demonstrate the efficiency of the algorithm

idUS. Depósito de Investigación Universidad de Sevilla

AN EFFICIENT MAXIMIZATION ALGORITHMWITH IMPLICATIONS IN MIN-MAX PREDICTIVECONTROLT. Alamo ∗, D. Mun˜oz de la Pen˜a∗ and E.F. Camacho∗∗ Dpto. de Ingenierı´a de Sistemas y Automa´tica. Universidad de SevillaCamino de los Descubrimientos s/n, 41092 Sevilla, Spaine-mail:{alamo,davidmps,eduardo}@cartuja.us.esKeywords: Binary quadratic programming, bandmatrices, combinatorial optimization, model predic-tive control, min-maxAbstractBinary Quadratic Programming (BQP) problemsarise frecuently in robust MPC when min-max tech-niques are used. In this paper, an efficient algorithmthat solves the problem for L-Band matrix structuresis presented. The L-Band matrix algorithm has adirect application to min-max MPC. The computa-tional burden of the L-Band max algorithm is poly-nomial with the dimension of the optimization vari-able and exponential with L, the band size. The pro-posed algorithm makes the implementation in realtime of min-max predictive controllers possible.1 IntroductionThe objective of a binary quadratic programming(BQP) problem is to find a binary vector z ∈{−1, 1}N that maximizes:F (z) =N∑i=1N∑j=1hijzizj +N∑i=1qizi (1)where H = {hij} is a symmetric N×N matrix andq = {qi} a real vector.BQP problems arise in a variety of applications,e.g. capital budgeting and financial analysis [9],traffic message problems [4], machine scheduling[2], molecular formation [1], etc. Furthermore, itis well known that the BQP is equivalent to manyclassical combinatorial problems such as maximumcut, maximum clique, maximum vertex packing,minimum vertex cover, maximum independent setand maximum weight independent set problems [8].In Model Predictive Control (MPC), BQP prob-lems arise when min-max techniques are used withbounded additive uncertainties models [7].The BQP belongs to the class of NP hard problems.At the present time, the computation of the exactsolution is only possible for small size instances.For this reason, min-max predictive controllers arehardly implementable in real plants.In this paper, an efficient algorithm for solving BQPL-Band problems is presented. A BQP L-Bandproblem arises when matrix H has a band structure,that is, hij = 0 if |i − j| ≥ L. The computationalburden of the proposed L-Band maximization algo-rithm is polynomial with the dimension of the opti-mization variable and exponential with L, the bandsize. To the knowledge of the authors, there is noequivalent result in the literature.When the control action of a min-max MPC is ob-tained computing a sequence of correction controlactions to a given stabilizing control law [3, 5, 6],the contribution of a perturbation in the predictionvanishes in an exponential way. This implies that|hij | decreases in an exponential way with |i − j|.In this way, the maximization problem arising froma min-max MPC can be approximated by an L-Bandmaximization problem. The error in the approxima-tion can be made arbitrarily small choosing conve-niently the band width L.This paper is organized as follows. Section 2presents the BQP L-Band optimization problem.Section 3 presents the preliminary notation. In sec-tion 4, the L-Band algorithm is presented. In section5, an example is provided. Section 6 is devoted tosome preliminary results needed to prove the maincontribution of the paper. In section 7 the main the-orem is proved. The computational efficiency of thealgorithm is addressed in section 8. The paper endswith a section of conclusions.2 L-Band maximization problemThe L-Band maximization problem is:max‖z‖∞≤1F (z) = zTHz + qT z (2)with z ∈ IRN and H a semidefinite positive L-Bandmatrix:|i− j| ≤ L ⇒ hi,j = 0We propose an algorithm that builds a set of 2Lhipothesis that contains the optimum solution. Tobuild this set we need to make (N −L− 1)2L com-parations. So, the algorithm requires a number ofevaluations of F (z) proportional to (N − L)2L.Note that for L = 1 the optimization problem is triv-ial and the maximum value of the quadratic functionis tr(H)+‖q‖1 and it is attained at in x = sign(q).3 Preliminary notationNotation 1 (Hypercube in IRn)Bnr denotes the following hypercube in IRn:Bnr = { z ∈ IRn : ‖z‖∞ ≤ r }Notation 2 (Vertices of a Hypercube)The vertices of the unit hypercube Bn1 in IRn will beenumerated as follows:ϑni =1− 2b0(i− 1)1− 2b1(i− 1)...1− 2bn−1(i− 1) , i = 1, . . . , 2nwhere bj(i), j = 0, . . . , n − 1 are the binary digitsof i, that is, i =n−1∑j=0bj(i)2j .Notation 3 Suffix and Prefix1. Denote the length of vector x asl(x) ={0 if x = 0max { i : x(i) = 0 } otherwisethat is, l(x) is zero if x = 0. If x = 0 then l(x)is defined as the greatest index i that satisfiesx(i) = 0.2. If n ≤ l(x) then sn(x) denotes the suffix oflength n of x, that is,sn(x) =x(l(x) + 1− n)x(l(x) + 2− n)...x(l(x))3. Given vector x ∈ IRm and n ≤ m, pn(x) de-notes the prefix of length n of x, that is,pn(x) =x(1)x(2)...x(n)The following property can be easily obtained fromthe properties of bj(.).Property 1 .1. The vertices ϑLi , i = 1, . . . , 2L can be obtainedfrom the vertices ϑL−1j , j = 1, . . . , 2L−1:ϑLi =[ϑL−1i1]if i ≤ 2L−1[ϑL−1i−2L−1−1]otherwise2. sL−1(ϑL2i) = sL−1(ϑL2i−1) = ϑL−1i4 L-Band Max AlgorithmThe following algorithm solves the L-Band maxproblem making a number of operations that, for afixed width band L, grows in a polynomial way withthe number of optimization variables.Theorem 1 Let us consider the quadratic functionF (z) = zTHz + qT z where H ∈ IRN×N is asemidefinite positive matrix with band structure, thatis, Hij = 0 for all |i− j| ≥ L > 1. Thenmax‖z‖∞≤1F (z) = maxi=1...2L−1F (zNi )where zNi , i = 1, . . . , 2L−1 are obtained from thefollowing recursion:1. Define zL−1i ∈ IRN , i = 1, . . . , 2L−1 aszL−1i =L−2∑j=0(1− 2bj(i− 1))ej+1where ej is the j-esime unitary vector in IRNand bj(n), j = 0, . . . , L − 2 are the binarydigits of n ∈ [0, 2L−1 − 1], that is, n =L−2∑j=0bj(n)2j .2. Given an integer k ∈ [L,N ], and vectors zk−1i ,i = 1, . . . , 2L−1, vectors mki , i = 1, . . . , 2Lare obtained from:mki =zk−1i + ek if i ≤ 2L−1zk−1i−2L−1 − ek otherwise3. Given an integer k ∈ [L,N ], and vectors mki ,i = 1, . . . , 2L, vectors zki , i = 1, . . . , 2L−1 areobtained from:zki =mk2i−1 if F (mk2i−1) ≥ F (mk2i)mk2i otherwise5 ExampleThe algorithm proposed in theorem 1 constructs ina recursive way a set of candidates for the optimumsolution to problem (2). This algorithm is shown inthe following example.The algorithm starts with a set of 2L−1 different vec-tors of L−1 non zero components. These are all thepossible endings of a vector. Therefore, it is guar-anteed that the optimum will have one of these end-ings. This is the set of candidates of length L− 1.In each iteration, the algorithm builds a candidate setof 2L−1 vectors but with one more non zero compo-nent than the previous iteration. When the vectorshave N nonzero components the algorithm finishesand the optimum can be evaluated.When a new component is added, the number of vec-tors is doubled, one for each of the possible valuesof the new component. This is the hypothesis setand it is made of 2L vectors. To eliminate half ofthis vectors we use lemma 1. This property allowsus to compare vectors with the same suffix of lengthL−1, eliminating the one with the lower functional.We obtain the 2L−1 new set of candidates vectors.N = 4L = 3H =6 1 −2 01 6 1 −2−2 1 5 20 −2 2 4 q =2−78−1z21 z22 z23 z241 −1 1 −11 1 −1 −10 0 0 00 0 0 0This is the initial candidates set. All possible end-ings are present.m31 m32 m33 m34 m35 m36 m37 m381 −1 1 −1 1 −1 1 −11 1 −1 −1 1 1 −1 −11 1 1 1 −1 −1 −1 −10 0 0 0 0 0 0 0F 31 F32 F33 F34 F35 F36 F37 F3820 20 26 34 8 −8 22 14This is the hypothesis set. It has the double ofvectors because another component has been addedwith the two possible values. The functional is eval-uated for each vector, and then, the next candidatesset is made of the ones with the higher functionalbetween those that have the same suffix of lengthL− 1.z31 z32 z33 z341 −1 1 11 −1 1 −11 1 −1 −10 0 0 0The new candidates set has again 2L−1 vectors, butnow with one more non zero component.m41 m42 m43 m44 m45 m46 m47 m481 −1 1 1 1 −1 1 11 −1 1 −1 1 −1 1 −11 1 −1 −1 1 1 −1 −11 1 1 1 −1 −1 −1 −1F 41 F42 F43 F44 F45 F46 F47 F4823 45 3 25 25 31 21 27Again the hypothesis set is constructed and the func-tional evaluated.z41 z42 z43 z44−1 1 −1 1−1 −1 −1 −11 −1 1 −11 1 −1 −1F 41 F42 F43 F4445 25 31 27This is the last candidates set, where the optimum isguaranteed to be in. After evaluating each vector’sfunctional, the optimum is z41 and the value of thefunctional is 45. This result can be proved by evalu-ating the max with a full matrix algorithm.6 Preliminary resultsIn this section some preliminary results needed forthe proof of the main contribution of the paper arepresented.Lemma 1 Let us consider the quadratic functionF (z) = zTHz + qT z where H ∈ IRN×N is a ma-trix with band structure, that is, Hij = 0 for all|i− j| ≥ L > 1. Suppose also that s ∈ IRL−1 thenF ( pasy)− F ( pbsy) =F ( pas0)− F ( pbs0), ∀y,∀pa, pbPROOF :Taking into account that H is L-Band and given s ∈IRL−1, F (.) can be rewritten asF psy = q psy+ psy  Hpp Hps 0Hps Hss Hsy0 Hsy Hyy psyDue to this structure, it is easy to see thatF ( pasy)− F ( pbsy) =F ( pas0)− F ( pbs0), ∀y,∀pa, pbLemma 2 Let us suppose that the vectors mki , i =1, . . . , 2L, k = L, . . . , N are computed accordingto Theorem 1. ThensL−1(mk2i−1) = sL−1(mk2i), i = 1, . . . , 2L−1PROOF :Note that zL−1i , i = 1, . . . , 2L−1 are defined aszL−1i =L−2∑j=0(1− 2bj(i− 1))ej+1 =[ϑL−1i0N−L+1]where i = 1, . . . , 2L−1 therefore sL−1(zL−1i ) =ϑL−1iThe lemma will be proved in a recursive way. Sup-pose that sL−1(zk−1i ) = ϑL−1i . Then it is easy tosee thatsL(mki ) =[sL−1(zk−1i )1]if i ≤ 2L−1[sL−1(zk−1i−2L−1)−1]otherwise==[ϑL−1i1]if i ≤ 2L−1[ϑL−1i−2L−1−1]otherwise= ϑLiNote that the last equality is due to the propertiesof the proposed enumeration of the vertices of ahyper-cube (see property 1). Thus, it is inferredthat sL(mki ) = ϑLi . Recall (see property 1) thatsL−1(ϑL2i) = sL−1(ϑL2i−1) = ϑL−1i , this equalityyieldssL−1(mk2i−1) = sL−1(mk2i) = ϑL−1i (3)Now from the definition of zki it is easy to see thatsL−1(zki ) is equal to{sL−1(mk2i−1) if F (mk2i−1) ≥ F (mk2i)sL−1(mk2i) otherwiseTaking into account equation (3) it results thatsL−1(zki ) = ϑL−1i7 Proof of the algorithmLet us suppose that y ∈ BN1 and thatF (y) = max‖z‖≤1F (z)Suppose also that there is k ∈ [2, N − 1] and i ∈[1, 2L−1] such that pk−1(zk−1i ) = pk−1(y). Thenthere is j ∈ [1, 2L−1] and yˆ ∈ BN1 such that{F (y) = F (yˆ)pk(zkj ) = pk(yˆ)In effect, denoter ={i if y(k) = 1i+ 2L−1 if y(k) = −1From the definition of mki , and from the equalitypk−1(zk−1i ) = pk−1(y), it is easily inferred thatpk(y) =[pk−1(y)y(k)]=[pk−1(zk−1i )y(k)]= pk(mkr )Note that in order to compute the set of vectors {zki },i = 1, . . . , 2L−1, mkr will be compared with mkr+a,where a = 0 if r is odd and a = 1 if r is even. Inwhat follows, it will be shown thatF (mkr ) ≥ F (mkr+a)Note that from lemma 2, sL−1(mkr ) =sL−1(mkr+a). Thus, applying lemma 1 andthe fact that y is a maximizer of F (.) in theunit-hypercube:F (mkr )− F (mkr+a) =F ([pk(mkr )sN−k(y)])− F ([pk(mkr+a)sN−k(y)]) =F (y)− F ([pk(mkr )sN−k(y)]) ≥ 0From the definition of zki it results that choosing j =(r − a)/2 and yˆ =[pk(zkj )sN−k(y)]:{F (y) = F (yˆ) = max‖z‖≤1 F (z)pk(zkj ) = pk(yˆ)Following this reasoning in a recursive way it is ob-tained that there is j ∈ 2L−1 and yˆ ∈ BN1 such thatF (yˆ) = max‖z‖≤1F (z)zNj = pN (zNj ) = pN (yˆ) = yˆ8 Computational efficiencyA BQP problem is a NP optimization problem. Thecomplexity of the computational effort is exponen-tial with N , the dimension of the optimization vari-able. The algorithm presented in this paper for L-band matrices has a computational effort exponen-tial with L, the band size, but polynomial with N .Figure (1) shows the polynomial nature of the L-Band algorithm and the exponential nature of a fullmatrix problem. The simulations have been made inMathlab 5.3 with the same band optimization prob-lem. Notice than the axis scale is different. The fullmatrix algorithm explores all possible vertexes andthe computation time is much higher.In figure (2) the dependence of the computationalburden of the L-Band algorithm is shown.9 ConclusionsIn this paper an efficient algorithm for solving aBQP L-Band problem is presented. The algorithmrequires only (N−L)2L evaluations of the quadraticfunctional.As the simulations results show, the computationalburden is much lower than the corresponding to afull matrix BQP algorithm.The application of the L-Band algorithm makes itpossible the implementation of min-max predictivecontroller in real processes.10 AcknowledgementsThe authors are indebted with Daniel Limon, forhis helpful comments and remarks. The authorsacknowledge MCYT-Spain for funding this work(contract DPI2002-04375-c03-01).0 5 10 15024681012141618x 106 Flops of a Full Matrix AlgorithmNflops5 6 7 8 9 10 11 12 13 14 1500.511.522.53x 104 Flops of a L Band Algortihm with L = 5NflopsFigure 1: Comparative of the computational effortbetween the Full Matrix algorithm and the L-BandalgorithmReferences[1] A.Phillips and J. Rosen. A quadratic assig-ment formulation for the molecular conforma-tion. Jnl. of Global Optimization, 4:229–241,1994.[2] B.Alidaee, B. Kochenberger, and A. Ahmadiam.01-quadratic programming approach for the op-timal solution of two scheduling problems. Int.Jnl. of Systems Science, 25:401–408, 1994.[3] A. Bemporad. Reducing conservativeness inpredictive control of constrained systems withdisturbances. In Proceedings of the 37st IEEEConference on Decision and Control, pages1384–1389, Tampa, Florida USA, December1998.[4] G.Gallo, P. Hammer, and B. Simeone.Quadratic knapsack problems. Mathemati-cal Programming, 12:132–149, 1980.7 8 9 10 11 12024681012141618x 106 Computational burden dependence with LLflopsN=45 N=40 N=35 N=30 N=25 N=20 15 20 25 30 35 40 4500.511.522.5x 107 Computational burden dependence with NNflopsL=12 L=11 L=10 L=9 L=8 Figure 2: Computational burden of the L-Band al-gorithm[5] B. Kouvaritakis, J.A. Rossiter, and J. Schu-urmans. Efficient robust predictive con-trol. IEEE Transactions on Automatic Control,45(8):1545–1549, 2000.[6] Y.I. Lee and B. Kouvaritakis. A linear program-ming approach to constrained robust predictivecontrol. IEEE Transactions on Automatic Con-trol, 45(9):1765–1770, 2000.[7] D.Q. Mayne, J.B. Rawlings, C.V. Rao, andP.O.M. Scokaert. Constrained model predictivecontrol: Stability and optimality. Automatica,36:789–814, 2000.[8] P.M.Pardalos and J.Xue. The maximum cliqueproblem. Jnl. of Global Optimization, 4:301–328, 1994.[9] R.Bridge and J. Yormark. An evolutionaryheuristic for quadratic 0-1 programming. Man-agement Science, 26(3):282–296, 2980.

An Efficient Maximization Algorithm With Implications in Min-Max Predictive Control

https://idus.us.es/bitstream/handle/11441/94375/245.pdf?sequence=1&isAllowed=y

An Efficient Maximization Algorithm With Implications in Min-Max Predictive Control

Abstract

Similar works

Full text

Available Versions

idUS. Depósito de Investigación Universidad de Sevilla