Dantzig, G.B.

International Institute for Applied Systems Analysis (IIASA)

A Generalized Programming Solution to a Convex Programming Problem with a Homogeneous ObjectiveDantzig, G.B. IIASA Research ReportDecember 1973Dantzig, G.B. (1973) A Generalized Programming Solution to a Convex Programming Problem with a Homogeneous Objective. IIASA Research Report. Copyright © December 1973 by the author(s). http://pure.iiasa.ac.at/8/ All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage. All copies must bear this notice and the full citation on the first page. For other purposes, to republish, to post on servers or to redistribute to lists, permission must be sought by contacting repository@iiasa.ac.at A Genera l i zed  Programming S o l u t i o n  t o  a  Convex Proaramming Problem w i t h  a  Homogeneous O b j e c t i v e *  George B.  Dantz ig  A t  a  r e c e n t  IIASA Seminar ,  Youri  A. Rozanov [I] posed  t h e  f o l l o w i n g  convex programming problem: F ind  Min F ( x )  such  t h a t :  -F ( X )  = cx t /x2  + x2 t + x 2  1 2  k  ( 1  1 A x = b , x z O  , (2  ) where 0  2 k < n ,  x  = (x l ,  ..., x,), c = ( C  l , . . . , ~ n ) ,  and A i s  an m x n  m a t r i x  o f  r a n k  m .  Our o b j e c t i v e  i s  t o  p r o v i d e  an  e f f i c i e n t  a l g o r i t h m  f o r  s o l v i n g  such  a  problem. I n i t i a l  B We b e g i n  by c o n s t r u c t i n g  a  n o n - s i n g u l a r  m x m m a t r i x  B = [P 1 ,P 2,"" pml 9 ( 3  ) i where each  Pi i s  g e n e r a t e d  by some x  = x  by t h e  r e l a t i o n  i pi = AX , xi - > o , o r  i = 1 , .  . . m . ( 4 )  It i s  not r e q u i r e d  t h a t  each  xi s a t i s f y  *xi = b ,  b u t  we do r e q u i r e  t h a t  i f  we s o l v e  * T h i s  p a p e r  w i l l  a p p e a r  i n  Mathemat ica l  P r o ~ r a m m i n g  and  I t s  A p p l i c a t i o n s ,  p u b l i s h e d  by t h e  N a t i o n a l  I n s t i t u t e  o f  Higher  Fk thema t i c s ,  Rome U n i v e r s i t y .  f o r  x = (A1,  ..., X 1, we o b t a i n  m and t h u s  t h e  weighted  sum o f  t h e  xi,  i s  s t a r t i n g  f e a s i b l e  s o l u t i o n ,  i . e .  s a t i s f i e s  ( 2 ) .  One way t o  o b t a i n  t h e  i n i t i a l  s e t  o f  columns Pi f o r  B i s  t o  s o l v e  t h e  l i n e a r  program Min cx  A x - b  , x > O  . The columns o f  t h e  f i n a l  b a s i s  o f  t h e  l i n e a r  program ( o r  o f  any f e a s i b l e  b a s i s  g e n e r a t e d  by t h e  s imp lex  method) can  be  used t o  d e f i n e  t h e  i n i t i a l  B where t h e  i - t h  column i n  t h i s  c a s e  i s  some column ji of  A and t h u s  x1 h a s  a l l  z e r o  compo- n e n t s  excep t  component ji  i s  u n i t y .  -[of c o u r s e ,  i f  t h e r e  e x i s t s  no f e a s i b l e  s o l u t i o n  t o  ( 8 ) ,  t h e n  t h e r e  would e x i s t  none f o r  ( 2 )  and t h e  procedure  would t e r m i n a t e  a t  t h i s  po in t . ]  Algor i thm The i t e r a t i v e  p rocedure  on c y c l e  t g e n e r a t e s  t h e  l i n e a r  program. F ind  Min Z and X 2 0  such  t h a t  - j where Each c y c l e  augments t h e  l i n e a r  program by one more column. On t h e  i n i t i a l  c y c l e ,  we i n c l u d e ,  b e s i d e  t h e  columns cor respon-  d i n g  t o  B,  a l l  t h e  r ema in ing  columns o f  A c o r r e s p o n d i n g  t o  t h e  l i n e a r  te rms  o f  F ( x ) ,  namely t h e  p  columns A f o r  j > k not  i n  'j B; f o r  t h e s e  p  columns Pi = A . j  , yi = c  j '  Thus on t h e  i n i t i a l  c y c l e  t h e r e  a r e  m + p  columns. I n s t e a d  of  s t a r t i n g  w i t h  t = 0, i t  s i m p l i f i e s  subsequen t  n o t a t i o n  i f  we f o r m a l l y  c a l l  t h i s  i n i t i a l  c y c l e  c y c l e  t = m + p.  t Le t  A = A .  f o r  j = (1 , .  . . , t )  be  t h e  o p t i m a l  b a s i c  f e a s -  j J i b l e  s o l u t i o n  t o  t h e  l i n e a r  program ( 9 ) .  It is easy  t o  s e e  t h a t  t h e  weighted  sum is  a f e a s i b l e  s o l u t i o n  of  ( 2 ) .  Denote t h e  i n d i c e s  o f  t h e  columns forming t h e  op t ima l  f e a s i b l e  b a s i s  B~ o f  ( 9 )  by ( j l , j 2 , .  . . , j m )  which,  o f  c o u r s e ,  depend on t and l e t  Subproblem To f i n d  an  improved s o l u t i o n ,  when such  e x i s t s ,  we d e t e r -  mine an x = xt+' which minimizes  s u b j e c t  t o  x  > 0  f o r  j = (1, ..., n )  and t h e  n o r m a l i z a t i o n  j - c o n d i t i o n  where ct > o f o r  j z k j - h o l d s  because  o f  t h e  i n c l u s i o n  o f  a l l  columns FJ j] f o r  a l l  j > k i n  t h e  d e f i n i t i o n  o f  ( 9 ) .  I t  fo l l ows  by t h e  d u a l i t y  theorem t h a t  an  o p t i m a l  s o l u t i o n  t o  ( 9 )  w i l l  y i e l d  a  nt such  t h a t  c  - ntA, = ct > 0  j j f o r  a l l  j > k  . j - Assuming some ct < 0  f o r  j 5 k ,  t h e  minimum f o r  t h e  sub-problem j ( 1 4 ) ,  ( 1 5 )  i s  o b t a i n e d  by s e t t i n g  ' X t+l  = 0  j f o r  a l l  c t  > o j - and x f o r  ct < 0 ,  j c t < 0  . I f  a l l  c! > 0  f o r  a l l  j ,  t h e  a l g o r i t h m  i s  t e r m i n a t e d  ( s e e  be low) .  J O p t i m a l i t y  C r i t e r i o n  The a l g o r i t h m  is  t e r m i n a t e d  if P ( x )  = F ( x )  - n t ~ x  > 0 f o r  a l l  x  2 0  [which, because  F ( x )  i s  homogeneous, i s  t h e  same a s  f o r  a l l  x  s a t i s f y i n g  (15) ] .  The r e a s o n  i s  t h a t  any f e a s i b l e  s o l u t i o n  9  would t h e n  s a t i s - f y  A9 = b ,  9  2 0  s o  t h a t  F ( 9 )  2 mtb. On t h e  o t h e r  hand ,  i t  is easy  t o  show t h a t  t h e  o p t i m a l  b a s i c  t f e a s i b l e  s o l u t i o n  t o  (9)--namely 9  g iven  by ( 1 1 ) - - s a t i s f i e s  where t h e  i n e q u a l i t y  f o l l o w s  from t h e  convex i ty  and homoge- n e i t y  o f  F. We would t h e n  conclude  t h a t  F ( 9 )  2 ~ ( 9 ~ )  and t h u s  gt would be a n  o p t i m a l  f e a s i b l e  s o l u t i o n  t o  (1) and ( 2 ) .  The new column f o r  c y c l e  t t l  is  t t l  t t l  Y t t l  = F(x  1, Ptt l  = A X  which " p r i c e s  o u t "  u s i n g  t h e  op t ima l  b a s i s  o f  c y c l e  t t o  be  Hence no improvement t o  ( 9 )  can  t a k e  p l a c e  i f  p ( x t t l )  2 0  ; morever  we have  j u s t  shown t h a t  P ( x  t t l )  2 0  a l s o  i m p l i e s  9  t o p t i m a l  f o r  (1) and ( 2 ) .  Accordingly ,  i n  t h i s  c a s e  we t e r m i -  n a t e .  Otherwise  we r e -op t imize  ( 9 )  w i t h  t r e p l a c e d  by t t l .  Convergence A formal  p roo f  o f  convergence t o  an o p t i m a l  s o l u t i o n ,  which we now p r e s e n t ,  depends on { X I A X  = b ,  x > 0 )  b e i n g  bounded and assumes a t  l e a s t  one b a s i c  s o l u t i o n  b e i n g  non- t d e g e n e r a t e ,  i . e .  B X = b  s o l v e s  w i t h  X > 0 f o r  some t .  It a l s o  r e q u i r e s  t h a t  a l l  columns (i:) be k e p t  as p a r t  o f  t h e  l i n e a r  program no m a t t e r  how l a r g e  t becomes. Convergence t can  be shown i n  t h e  s e n s e  t h a t  F ( 9  ) converges  t o  t h e  f i n i t e  minimum. Note f i r s t  t h a t  f i n i t e  l ower  and upper  bounds f o r  m! can  be e s t a b l i s h e d  f o r  each  i. For  t h i s  pu rpose ,  i t  i s  c o n v e n i e n t  t o  l e t  t h e  non-degenera te  b a s i c  s o l u t i o n  be a s s o c i a t e d  w i t h  t h e  f i r s t  m columns B = [ P ~ , P ~ , . .  . pm] and  t h a t  ( 9 )  be m u l t i -  p l i e d  by B-l s o  t h a t  now t h e  f i r s t  m columns form an  i d e n t i t y .  The new r i g h t  hand s i d e  b  w i l l  now have  a l l  p o s i t i v e  components ( s i n c e  t h e  b a s i c  f e a s i b l e  s o l u t i o n  a s s o c i a t e d  w i t h  B i s  non- d e g e n e r a t e ) .  Now by ( 1 6 )  mtb 2 ~ ( 2 ~ )  2 L where L i s  some f i n i t e  lower  bound f o r  F ( x ) .  (Note t h a t  L i s  f i n i t e  because  { X I A X  = b ,  x  > 0 )  i s  bounded.)  S ince  n t  a r e  o p t i m a l  m u l t i p l i e r s  t t f o r  ( g ) ,  we have  n  P  < y  and hence  m i  < yi f o r  i = (1,. .. , m ) .  j -  j From t h i s ,  i t  f o l l o w s  t h a t  With r t  bounded, t h e r e  e x i s t s  a conve rgen t  subsequence  S  such  t h a t  m t  + n* f o r  ~ E S .  On t h i s  subsequence  S  we can  f i n d  a sub-subsequence S '  f o r  which x t + l  + x* + 0 because  t h e r e  i s  a p o i n t  o f  c o n d e n s a t i o n  x* on t h e  hype r sphe re  Fo r  ~ E S '  we have and a t  + TI* . Le t  us  choose rES1 ,  ~ E S ' ,  r < t .  Then -Et = yt+l  , =tpt+l < 0  and Qr = yr+l - a tp r+ l  > 0  f o r  a l l  r < t - because  a t  i s  t h e  op t ima l  v e c t o r  o f  p r i c e s  f o r  t h e  l i n e a r  pro-  gram f o r  c y c l e  t b u t  not  t+l .  We now l e t  r + m ,  t + m.  Then and  t h u s  Qr + 0 ,  E t  + 0  . Le t  2 be  an  op t ima l  s o l u t i o n  (1) and ( 2 ) .  Assuming t h e r e  2 4  i s  a t  l e a s t  one component 9 > 0  f o r  j 5 k ,  l e t  B = ( . Z  Sj ) j J =1 and 2 = 2 / B  be  t h e  "normal ized"  form o f  ?, t h e n  If t h e  assumpt ion  made above does  n o t  h o l d  s o  t h a t  a l l  P j  = 0  f o r  j = l , . . . ,  k ,  t h e n ,  because  T~ j 2 0  f f o  j > k ,  t It f o l l o w s  i n  e i t h e r  c a s e  t h a t  P ( ? )  > Lim n  b  where t h e  l a t t e r  t l i m i t  e x i s t s  because  a b i s  monoton ica l ly  d e c r e a s i n g  and i s  bounded below by nt t lb  > F ( 9  t t l )  2 F ( R )  s o  t h a t  F ( 2 )  = Lim ~ ( 2 ~ " ) .  Genera l  Comments The p rocedure  o u t l i n e d  can be  expec t ed  t o  b e  e f f i c i e n t  because  i t  is  t h e  ana logue  o f  t h e  s imp lex  method. The method i s  s i m i l a r  t o  one proposed ( w i t h o u t  p roo f  o f  convergence)  f o r  t h e  chemica l  e q u i l i b r i u m  problem and found on  l i m i t e d  exper -  i m e n t a t i o n  t o  have good convergence r a t e s .  S t anda rd  g r a d i e n t  p r o c e d u r e s  a r e  n o t  recommended because  convergence  would b e  t o o  s low t o  be p r a c t i c a l .  The method o f  p roo f  p r e s e n t e d  h e r e  is  a l o n g  t h e  l i n e s  f i r s t  p roposed  by t h e  a u t h o r  f o r  convex f u n c t i o n s  which have -  ti f i n i t e  minimum f o r  t h e  subproblem w i t h o u t  t h e  n o r m a l i z a t i o n  c o n d i t i o n .  By i n t r o d u c i n g  min imiza t ion  under  t h e  normal iza-  t i o n  c o n d i t i o n  ( 1 5 ) ,  we have o b t a i n e d  a  p roo f  o f  convergence  when F ( x )  i s  convex and homogeneous o f  d e g r e e  1. F o r  example, t h e  p roo f  g iven  h e r e  i s  e q u a l l y  a p p l i c a b l e  t o  t h e  ehemica l  e q u i l i b r i u m  problem. No bounds f o r  F ( 2 ) ,  however, a r e  g i v e n  e x c e p t  by ( 1 9 )  and ( 1 6 )  ntb -  BE^ ( F ( 2 )  5 ntb , ( 2 1  where u n f o r t u n a t e l y  B = f o r  j < k i s  n o t  known. J - An a l t e r n a t i v e  method i s  t o  s o l v e  t h e  problem by a pa ra -  m e t r i c  q u a d r a t i c  programming scheme where c x  = 0 i s  t h e  un- known pa rame te r  and C x2 f o r  j < k is minimized. j Refe rences  [l] Rosanov, Youri A .  "A Few Methodologica l  Remarks on Op t imiza t ion  o f  Random Cost  F u n c t i o n s  ," I I A S A  Research  Memorandum RM-73-7, December 1973. 

A Generalized Programming Solution to a Convex Programming Problem with a Homogeneous Objective

Abstract

Similar works

Full text

Available Versions

International Institute for Applied Systems Analysis (IIASA)