The bisection method is shown to possess the asymptotically best rate of convergence for infinitely differentiable functions having zeros of arbitrary multiplicity. If the multiplicity of zeros is bounded methods are known which have asymptotically at least quadratic rate of convergence

Sikorski, Krzysztof A.

Trojan, G. M.

Columbia University Academic Commons

Asymptotic Optimality of the Bisection Method CUCS-92-84 K. Sikorski Department of Computer Science Columbia University New York, N.Y. 10027 G.M. Trojan Department of Physics The University of western Ontario Canada N6A3K7 January 1984 AMS(MOS) Subject classifications (1970): 6SH10, CRS.1S. This research was supported in part by the National Science Foundation under Grant MCS-~823676. ~b.tract The bi8ection method is shown to posses the asympto-tically best rate of convergence for infinitely differentiable functions having zeros of arbitrary multiplicity. If the multiplicity of zeros is bounded methods are known which have asymptotically at least quadratic rate of converg~ce. Summary. We seek an approximation to a zero of an infinitely differentiable function f: • ~ • such that f(O) ~ 0 and f(l} 2 O. It is known that the error of the bisection method using n function evaluations is 2-(n+l}. If the information used are function values, then it is known that bisection information and the bisection algorithm are optimal. Traub and wo~niakowski [6] conject-ured that bisection information and the bisection algorithm are optimal even if far more general information is per-mitted. They permit adaptive evaluations of arbitrary linear functionals as information and arbitrary trans for-mations of this information as algorithms. This conjecture was established in [4]. That is, for n fixed, bisection information and bisection algorithm are optimal in the worst case. Thus nothing is lost by restricting oneself to function values. One may then ask whether bisection is optimal in the asymptotic worst case sense,i.e., possesses asymptotically the best rate of convergence. (Asymptotic methods are, of course, widely used in practice.) We prove that the answer to this question is positive for the class F of functions having zeros with arbitrary multiplicity and continuous functional.. Assuming that every f in F has Zeros with bounded multiplicity there are known hybrid methods which have at least quadratic rate of convergence as n tends to infinity, see, e.g.,Brent (11, Traub [51, and Section 5. 1 1. Formulation of the problem. Let G = Coo(E) be the Fr~chet space of infinitely differentiable functions on R with the metric p given by p (f,g) Vf,g e G, where II IIi is the i-th semi-norm, i = 0,1, ... , "h II i = max ( I h (j) (x) I, x e [- i , i], j = 0, 1, . • • , i }, see, e.g., Schaefer [3]. Observe that the semi-norms are monotonic, i.e., IIh!l. 1 2 II h ./I, \j i, 'T/h e G. l.+ l. We seek an approximation to a zero of a function which belongs to the class FO' 00 FO = (f e C [0,1] ,f(O) ~ O,f(l) 2 o,~o.: f(o.) =: 0). Obviously each function f in FO can be extended to the function 1 e G. Therefore without loss of generality we consider the class F: (1. 1) F = (f e G: f(O) ~ O,f(l) 2 O,~o.,f(o.) = OJ. Define the solution operator s: F ~ [0,1] by (1. 2) Our problem i. to find an approximation to S(f). To .olve this problem we use an adaptive information operator CD (briefly information) N: G -+ a defined as follows. Let f € G and (1. 3) N(f) = [L l {f),L2 ,f(f), ... ,Ln ,f(f), ... ] where L. f{') = L'('iYl,···,Y· 1): G -+ a ~, ~ ~-is an arbitrary linear functional and y. = L. (f:Yl"" ,y. 1)' i = 2,3, .... ~ ~ ~-By N (f) we denote n (1. 4) Note that the vector N l(f) contains all components of n+ N (f), N l(f) = [N (f),L 1 f(f»). That is, increasing n n n+ n n+ , we use previously computed information. It is convenient to use the notation N = (N). We may assume without loss n of generality that for some function f in F the 2 functionals L. f{')' i = 1,2, ... ,n, are linearly independent ~, and therefore the functional Ll is not equal to the zero functional. Let us also denote by ~ the class of all information operators of the form (1.3). (1. 5) where Th bi ' 'f t' bis, d f' d b e sect~on ~n orma ~on N ~s e ~ne y Lb, is (f) = f ( ) f x, , ~, ~ i=1,2, ... , x, = (a, 1 + b, 1)/2 ~ ~- ~-with a O = 0, b O = 1 and " 3 { ai-l if f(x,) > 0 ~ b, = [b i _l if f(x,) < 0, ~ a, = ~ x, ~ if f(x,) ~ 0 ~ ~ x, ~ if f(x,)2°· ~ Knowing N (f) we approximate S(f) by an algorithm. By the n algorithm e::l = (~ } we mean a sequence of arbitrary trans-n formations, e::l : N (G) ~ 2, n = 1,2, .... Let ~(N) denote n n the class of all algorithms using the information N. The n-th error of for an element f is defined by (1.6) e (N'CD,f) = IS(f) - e::l (N (f» I. n n n In the asymptotic setting we wish to find ~* and N* such that for any f in F the error e (N*,~*,f) goes to n zero as fast as possible as n tends to infinity, The information N* and algorithm ~* are called optimal iff 'V N C ,"" 'V ~ E I (N) , ~ f* E F such that (1. 7) e (N,~, f*) lim n > 0 ~up en(N*,~*,f) , "If E F. bis _ (,.,.bis) The bisection algorithm ~ .n is defined by biS(NbiS(f» a (an + b )/2. ~n n n It is known that for every f in F, and (NbiS bis f*) = en ,~, { 2- (n+l) /3 2- (n+l) /6 n-even, n-odd, for f*(x) = x - 1/6. It was shown in [4] that for a fixed n: sup\S(f) - ~ (N (f»\2 sup\S(f) _ <;)biS(Nbis(f» I feF n n feF n n = 2 - (n+l) , for every N e ~ and <;) E teN), i.e., that bisection infor-mation and algorithm are optimal for the worst case model with a fixed number n of functional evaluations. Here we show that bisection information and algorithm are nearly optimal for the asymptotic worst case model. 4 5 More precisely, we show that for every continuous N, i.e., Li,f in (1.3) are continuous, Li,f(gk) k~oo) Li,f(g) whenever o (gk,g) -k.+a:/ 0, every algorithm ~ E t(N) and every sequence 00 (~} 1,6 ~O, there exists a function f* in F such that n n= n the upper limit e (N,~, f*) n lim sup ......::.:.....----2-n 6n n~ > 0, and obviously for every f in F since e (N,~, f*) n lim sup ---:;~------( bis bis f) n~ !nen N ,~ , e (N,~,f*) n .2 lim sup --=.;;.....----, -n n~ 5 2 n -n bis bis a 2 .2 26 e (N ,~ ,f). n n n The sequence (5 J may n converge to zero arbitrarily slowly. Therefore we say that the bisection information and algorithm are nearly optimal for the asymptotic worst case model; compare- also to [6, p. 199] and [8]. We formulate this result in Theorem 1.1: For every continuous information N E n, co every algorithm ~ E teN) and every sequence (6nJn=1' 6n~O, there exists a function f* in F such that (1. 8) -n 1 im sup e (N, ~, f* ) / (a 2 ) > o. n n n-+<XI [] 2. Sketch of the proof. First we give a sketch of the proof of Theorem 1.1. The proof is by contradiction. Suppose that there exists an operator N* - (N~) and an algorithm ~* = (~~J such that for every function f in F (2.1) co . We construct a cauchy sequence of functions (g) 1 ~n r r= cc c (K) such that g = lim g is in F and does not satisfy r (2.1). • '\&0 on ' Let cc (6') 1 be any sequence of n n= positive numbers, 1 6i < 2' such that 6 = o{!'). Then we construct a n n . 1 00 1 1 sequence of funct~ons (f) l' f e G such that f (x) = 0 n n= . n n 1 1 1 -n' 1 -n 1 for x e In = [an,enJ and 6~2 ~ diam(I n ) ~ 2 ,fn(X) < 0 11' (resp. > 0) for x e [-co,a ). Crespo x e (e ,~]) and n n 6 N* (f1) 1 ( 1} = N*(f }, q = 0,1, .... Moreover f is a ca~chy n n+q n n n 1 1 sequence and I 1 c:: I , 'in. n+ n We prove that the limit function fl = lim f1 is in F. By the continuity_of N* we get n n n-+eo 'Vn. co Let (m.). 1 be an increasing sequence of integers J J= such that (2.2) 'n < 6-j ~ I for "In > m .. n J Equation (2. 1) implies that there exist n1 > m1 such that (2.3) I~* (N* (f1) ) - S (f1) I < 6- 1 '2-n 'in 2 n 1 · n n 5n ' 1 Then we define gl = f and construct the next Cauchy n 1 2 ~ 2 sequence of functions (f} 1 by setting f = n n* n d . f2 . an construct~ng n' n 2 n 1 + 1 ~n such a way 2 2 2 2 -l-n fn(x) = 0 for x e In = [an,enl and 6 !~2 ~ 1 f for n ~ n 2 that f e G, n diam(I2 ) n ~ 6- 12- n , n > n1,f~(X) < 0 (resp. > 0) for x e [-~,a~) (resp. x e (e 2n, ~ ] ), N* ( f2 ) = N* ( f2), q = 0, 1, ... , n n+q n n· 2 I2 'f nand In+1 C n' We prove that the limit function f2 = lim f2 is in n F n~ and by the continuity of N* get n define g2 we obtain such that "In. = f2 and repeat our construction. n 2 00 r 00 the sequence (nr )r=l' and (fn )n=l' In this way r = ~,2, ... , 7 (2.4) (2. 5) where (2.6 ) (2. 7) and (2.8 ) r CD (t) 1 is a cauchy sequence for every r, n n-< 0 - 0 > 0 n > max (n l' m ), r r- r and n = 0 o belongs to F, N* (fr ) n n r = N*(fr+q ) n n ' r+q Vn q=O,l, •.. , n . r We define g = fr , show that [g }~ 1 is a Cauchy sequence r n r r= r 8 and that the limit function 9 = lim gr belongs to F. By the r-continuity of N* and (2.7) we get n (2.9) ~(g) = N*(g ) n r 'Vr, n,S; n • r construction of g implies that S(g) belongs to the r interval rr+l, Yr. Therefore (2.4), (2.5) and (2.6) yield n+l which contradicts (2.1) and completes the proof. 9 10 3. Auxiliary lemmas. In order to construct the functions in Section 2 'We need a few auxiliary lemmas. Let Ul"",Um be linearly independent continuous linear functionals on G, and El, ... ,Em closed subintervals of R. m Denote A = u· 1 E. and m J= J G: supp ( f) c A ). m Lemma 3.1: For every positive c and every family of nondegenerated intervals E.,i = 1,2, ... ,m-l, such that 1 Ul"",Um_ l are linearly independent on C(Am_ l ), there exists an interval E c a, 'With diam(E ) = e, such that m m Ul,o",Um are linearly independent on Am' o Proof: This is the same as the proof of Lemma 2.1 of [4] = = 'With C [a,b] replaced by C (a), and therefore the proof is omitted. D proposition 3.1: For every n, e, 'f: > 0, and continuous information N E ~ there exist a function f E G, interval n -n In = [an'~n]' diam(I n ) E (0,2 ] and intervals Ej , j = 1,2,.o.,k , where k is the maximal number of linearly n n independent functionals on G among Ll f , .. 0 ,Lon f , n ' n (denote them by L*l , ... ,h* ) such that: ,n -k ,n n (i) L*l , •.. ,Lk* are linearly independent on ,n ,n (ii) k n n C (U. 1 E.), J= J < f (x) = = n > 0 X E 0 X E 0 x E and dis t (E : , I ) J n 21 diarn{E j ), (-00 , Cln) , [Cln,t3n] , (t3n,+<o) , \jj. II proof: The proof is similar to the proof of Lemma 2.2 of [4]. We prove below a more general Lemma 3.3 which 11 combined with Lemma 3.2 yields the proposition by induction. The formulation of Lemma 3.3 enables easy varying the interval I which is needed in the proof of Theorem 1.1.0 n Lemma 3.2: The proposition 3.1 holds for n 2 1 with fl such that diam{I l ) 2 ~i/2, where 5i is the first element of the sequence from Section 2. o Proof: Since Ll F 0 on G then as in the proof of Lemma 2.1of [3] we conclude that there exists an interval El , diam (E l ) ~ 1/4, such that Ll F 0 on C (E l ). Let 2 -exp(-x ) x E (-00 ,.0) , v 1 (x) = 0 X E [0,1/4] , -2 exp(-(x-l/4) ) x € (1/4, +00) , and Note that vi E G. Define f1 by if E1 c [3/8,+=), otherwise. Then diam(I 1 ) = 1/4, so 'i/2 ~ diam(I 1 ) ~ 1/2 and dist(I 1 ,E1 ) 2 1/8 = 1/2 diam(E1 ) which shows that f1 satisfies Lemma 3.2. Lemma 3.3:· Suppose that proposition 3.1 holds for o n, and let Z be an arbitrary interval Z c I. Then pro-n n n position 3.1 holds for n + 1 with I 1 c Z such that n+ n where (a'} is as in Section 2, k / k / k + 1 and on n ~ n+1 ~ n L* = L~ , i = 1,2, ... ,k . i,n+l 1,n n c proof: Let Z n = min(2-n,min diam(E.)/2) j=l, ... ,k J 12 = diarn(Z )(1 n 6' n - ~;1)/2 and M = (e1+e2 )/2 .. n Define the functions, see Fig. 3. 1, H lex) = n, -2 -2 -exp(- (x-a +d) (X-M-b) ) x E ra -d ,M+b J , n n n n n n -2 -2 exp(- (x-e) (x-e -d) ) 2 n n o X E [e2 ,a +d ] n n otherwise 13 -2 -2 exp(-(x-M+bn) (x-en-dn) ) x e [M-b ,e +d J n n n , H 2(x) = n, -2 -2 -exp(-(x-a +d) (x-e) ) n n 1 x e [a -d ,ell n n o otherwise. k n consider the functions Gn,l' Gn,2 e C(Ui=l Ei ) such that L! (H . +G .) = 0, lo,n n,J n,J i = 1,2, ... ,k , j = 1,2. n Such functions k exist since L! are linearly independent lo,n n on C (U. 1 E.). J= J Let (3. 1) h .=H .+G .. n, J n, J n, J Choose a positive constant c so small that n min I fn (x) 1/4 xe(-oo,a -d JU[e +d ,+00) n n n n (3.2) c max\h . (x) 1 ~ min n xea n,J and (3.3) -n C Ilh ./1 ~ 2 n n,J n where hO,j(X) - 1 and Co = 1. max max j=1,2 xe~ for j = 1,2 Ih 1 . (x) I . C 1/2 n- , J n-CI -d n n ! CI n 8 n,:z /- ...... " ........ M-b / 8 ' ! n / n, ~ ,- ...... , / ' ' ... ':-.. "" " ... / e M" t e 2 \\. I 1 " M+b \'_.... I n , 8 'n,2 I \ '8 " ",,/ n,l ... ~-.", Fig 3.1 Let f . = f + c h ., j = 1,2. Then f . E G and n, J n n n, J n, J (3.4) N(f .)=N(f), n n,J n n s inc e L ~ (h .) = 0, i = 1, 2 , ... , k and L. f (h .) = 0, ~,n n,J n~, n n,J i = 1,2, ... ,n. The information operator Nn+l yields a 14 . functional L 1 f . n+ , If L*l , ... ,Lk* ,L 1 f are linearly ,n ,n n+, n n n dependent on G then Lemma 3.3 holds for f 1 = f -n+ n,l (also for f n+l = f 2) and k 1 = k . n, n+ n Of course in both cases dist(E.,I 1) 2 diam(E.)/2, i = 1,2, ... ,k 1. n+ ~ n and diam(I 1) = diam(Z )/2 - b = diam(Z )6' 1/(26'). n+ n n n n+ n If L*l , ... ,Lk* ,L +1 f are linearly independent on G ,n ,n n , n n then Lemma 2.i yields that there exists an interval ~ = [el ,e1+b J, k 1 = k + 1, such that they are n+l n n+ k n linearly independent on C{u. n l E.), Then define f 1 by J= J n+ (3.5) = {ffn, 1 fn+1 n,2 Then if ~ n+1 c= ( -00 , M+b /2] , n otherwise. if ~ c (-oo,M+b /2], K. 1 n n+ otherwise. Obviously dist(E.,I 1) 2 diam(E. }/2, i = 1,2, ... ,k 1 . ~ n+ ~ n+ and diam(I l} = diam(Z }/2 - b = diam(Z ).&' 1/(26'}. n+ n n n n+ n Thus the function f 1 satisfies Lemma 3.3. n+ IJ 15 Lemma 3.4: Let (f } be a sequence of functions constructed n by applying Lemma 3.3 to the function f1 from Lemma 3.2. Then (f } is convergent in G and f = lim f belongs to F. 0 n n n~ Proof: Observe that each of the functions f , n 2 2 is n of the form (3.6 ) f f n-1 h n = 1 + ~i=l i where h. = c.h .. for j = 1 or j = 2, see (3.l), (3.2) and ~ ~ ~,J (3.3), h. E G and ~ (3.7) and (3.8) max\h. (x) l ~ min XEa 1. -i IIh.lI. ~ 2 • 1. 1. min 1 fi (x) 1/4 , x € ( -00 , (l • - d. ] u [t3 . +d. , +00 ) , 1. 1. 1. 1. max \h i _ l (x) 1/2 , We first prove that (f } is a Cauchy sequence, which n combined with completeness of G implies convergence. ~ssume without loss of generality that n > m. Then . (3.6), (3 .. 8) and monotonicity of semi-norms imply that m-l -i 00 -i () "( f , f ) ~ l:i=l 2 Ilfn-fmlli + I:i=m 2 n m m-l -i n-l - (m-l) = I:. 1 2 lit. h·lI. + 2 1.= J=m J 1. m-l -i n-l Ilh·ll· - (m-l) ~ I:. 1 2 I:. + 2 1.= J=m J J ~ - (m-l) 2 . 1 + 2 - (m-l) 4.2-m, = which yields convergence. Let f = lim f . n Now we prove n~ that f is in F, i.e., that it has exactly one zero. and diam(I ) -n Therefore Recall that I n+l C I ~ 2 . n n f(a) = 0, a 00 = nn=l I n' and f (j) (a) = 0, j = 1,2, .. " since convergence in metric p implies uniform convergence with all derivatives on every closed interval in ~. Now we . 16 show that a is the only zero of f. Namely take arbitrary 17 x F a and assume without loss of generality that x > ~. Then we show that f (x) > o. Since I c I and diam(I ) / 2- n n+l n n ~ then there exists an index j* such that '9n .2 j*. -j* Using (3.6), (3.7), the fact that x € (~,*+2 ,~) and J -n d ~ 2 we get n f(x) j*-l ~ ~ = flex) + t, 1 h, (x) + t. '* h, (x) = f,*(x) + t, .*h, (x) J= J J=J J J J=J J ~ ~ j*-j .2 f,*(x) - t J'=J,*\hJ,(X)\.2 f,*(x)-max\h,*(t)lt,_,* 2 J J t€1 J J-J .2 f.*(x) - 2 max\h,* (t) I J t€1. J o which completes the proof. 18 4. Conatructions needed in the proof of Theorem 1.1 In order to complete the proof of Theorem 1.1 we r co construct the sequences (fn)n_l' r = 1,2, ... , by use of Lemmas 3.3 and 3.2 for the information N*. 1 1 co Namely let fl z fl from Lemma 3.2 and let (fn)n=2 be the sequence of functions from Lemma 3.3 with the inter-vale Z equal to I for every n. Lemma 3.4 yields that n n fl = lim fl exists and belongs to F. Moreover (3.4) n ll-+CIO implies that N (fl) = N (fl), '" n. Constructions in the n n n proof of Lemma 3.3 imply that fl has all the properties n from Section 2. r co Now suppose we have constructed the sequence (fn)n=l' r 2 1, by applying Lemma 3.3 to the function f1 from Lemma 3.2, such that (2.4), (2.5) and (2.6) are satisfied, where fr = lim fr exists and belongs to F by Lemma 3.4. n ll-+CIO r We set g = f and define r n r as follows: th (fr+l)~ e next sequence n n=l Set f r + l = fr for n ~ n and let f r + l l be the function from n n r n + r Lemma 3.3 applied to the function f~ , with the interval r Z given by nr 19 (Car ,ar + diam(I r )/6] n n n r r r if 8(fr) 2 (a~ +6~ )/2, r r Zn r :r [!3r diam(I r )/6,l3 r ] n n n otherwise. r r r Then f r + l are constructed by Lemma 3.3 with Z n+l 'n every n 2 n +1. Lemma 3.4 implies that f r + l = r r+l = I for .n lim f r + l n n-+oo exists and belongs to F. Moreover the above definition of ~r combined with (2.5) -n yields (2.8), since diam (I r ) n r .2 6- r & I 2 r n Lemma 3.3 yields also that (2.4) and (2.6) r hold for the sequence (fr + 1). The construction of f r + l n n implies that N* ~fr ) = N* (fr + l ) V n ~ n since n 1 > n . n nr n nr+ l r r+ r Therefore by induction (2.7) holds. 00 Let (Pn}n=l be the sequence of functions r p = f n n for n , r r = 1,2, ... , Then Lemma 3.4 yields that 9 = lim p exists and belongs n .' n~ to F. Moreover 9 = lim 9 ,since (g ) is a subsequence of r r Observe that 8(g) = n:=l I~ since r -n r+l I c n r+l r+l In +1 c I r r n r and diam(I r ) ~ 6- r 2 r Therefore 8(g) € n I r + l n +1 which finally r r completes our constructions. Remark 4.1. Observe that g(j) (x) = 0, j = 0,1, ... , r r x E I . n r Therefore, as in the proof of Lemma 3.4 we conclude that 20 9(j) (5(9» - 0, j • 0,1, ... ,i.e., that 9 has a zero with infinite multiplicity. o 21 5. Final remarks. (i) Remark 4.1 indicates that bisection is nearly optimal in the subclass of F consisting of functions having zeros with arbitrary multiplicity. (ii) The idea of the proof is based on the "remanance" property introduced by Delahaye and Germain-Bonne in [2]. For other applications, see also Trojan [7]. (iii) If the multiplicity of zeros of functions in F is bounded it is possible to construct information N and algorithms ~ which guarantee asymptotically quadratic convergence. If the multiplicity of a zero is known, say m, it is enough to use a combination of bisection and modified Newton's method: x. 1 = x. - mf(x.)/f' (x.), which ~+ ~ ~ ~ converges quadratically for i ~ 00, see [5, p. 127]. If the multiplicity m of a zero is unknown we can calculate it by using a combination of bisection and Newton's method 2 and applying Aitken's & formula, see [5, p. 129, Appendix D]. Then knowing m we proceed as above. 22 ~cknowledgementa. We are greatly indebted to K. Rygielaki, J.F. Traub, , G.W. Wa.ilkowaki, ~. Werachulz and H. Wozniakowaki for the remarks on the manuscript. References. [1] Brent, R.P., ~lgorithms for Minimization without Derivatives, prentice Hall, Englewood Cliffs, New Jersey, 1973. [2] Delahaye, J.P., Germain-Bonne, B., The Set of Logarithmically Convergent Sequences Cannot be Accelerated, SIAM. J. Num. Anal., ~ugust 1982, v. 19, No.4, pp. 840-844. [3] Schaefer, H.H., Topological Vector Spaces, the Mac-Millan Company, New York, 1967. [4] Sikorski, K., Bisection is Optimal, Num. Math., 40, 1982, pp. 111-117. [5] Traub, J.F., Iterative Methods for the Solution of Equations, prentice-Hall, Englewood Cliffs, 1.~64. Reissued by Chelsea, New York, 1982. [6] Traub, J.F., Wozniakowski, H., A General Theory of Optimal Algorithms, Acad. Press, New York, 1980. [7] TrOjan, G.M., An Upper Bound on the Acceleration of Convergence, submitted for publication. [8] Trojan, G.M., Asymptotic Model for Linear problems, in progress. 

Asymptotic Optimality of the Bisection Method

https://academiccommons.columbia.edu/doi/10.7916/D84B388R/download

Asymptotic Optimality of the Bisection Method

Abstract

Similar works

Full text

Available Versions

Columbia University Academic Commons