The concept of conditionally-effective estimation which provides optimum estimates for a given criterion in cases of given limitations is examined. The concept has the best accuracy for given limitation on the suitability of the algorithm concerning the deviation of the law governing error distribution from the proposed law. It is concluded that there must be a greater difference between individual algorithms in terms of difficulty, and that for the linear regression problem algorithms based on excluding lost points should be studied

Gurin, L. S.

Tsoy, K. A.

English

NASA Technical Reports Server

  
 
 
N O T I C E 
 
THIS DOCUMENT HAS BEEN REPRODUCED FROM 
MICROFICHE. ALTHOUGH IT IS RECOGNIZED THAT 
CERTAIN PORTIONS ARE ILLEGIBLE, IT IS BEING RELEASED 
IN THE INTEREST OF MAKING AVAILABLE AS MUCH 
INFORMATION AS POSSIBLE 
https://ntrs.nasa.gov/search.jsp?R=19810014222 2020-03-21T14:19:45+00:00Z
oNASA TECHNICAL MEMORANDUM
	
NASA TM-76218
STEADY (ROBUST) CCJDITI^--)NALLY EFFECTIVE ESTIMATION
OF PARAMETERS
L. S. Uurin and K. A. Tsoy
(NASA-TM-76218) STEADY ( HCBUST)
CONDITIONALLY EFFECTIVE ESIIMA110N CF
BABAKETEbS (National Aeronautics and Space
Adainistration) 14 p HC A02/MF A01 CSCL 12A
N61-22755
Unclas
G3/64 42183
Translation of "Ustoychivyye (Robastnyye) Uslovno-Effektivnyye
Otsenki Paraemtrov," Academy of Sciences USSR, Institute of
Space Research, Moscow, Report Pr-382, 1977, pp. 1-14.
c-
a
	
n	 w t.
NATIONAL AERONAUTICS AND SPACE ADMINISTRATION
WASHINGTON, P. C. 20546	 JUNE 1980
F
%re=a^i^.^ -
STANDARD TIi1.E FACE
1. Rerert Ne,	 2. G•vernntent Ae•eeelon No. 	 I2. ReelpletN * e CN. 1 .1 No.
, NASA 'rM-762L_	 i
d. Title end SW Nitl•	 1 3. Re,or ► Dote
STEADY (ROBiTST) COMPTTIONALLY EFFFCTTVF a. Potformins or,enllohon Code
ESTIMATION OF PARAMFTFRS	 I
1
1. Arth•r(e)	 fl, Per!•rreins 0114nieotlon Royort 144.
L.  S . GL- , in and K. A. Tsoy
10, work unit No.
11 c
r	
t r t to tNon ot a	 on •.
NASw• 1193
13. Type of Rop•tt end Forced Covered
Translatio"
.'. S a^+o ,nq A enc> ham• end nadrooe
^ational Aeronautics and Space Adminiatration
Hasninyton, D.C. 20546	 t <, s,.^.«In, A^•wey 
Code
^ 13. Sv^► lementery Nelee	 .
Translation of "Tlstoycrivyye (Robastnyye) ITslovno-Fffektivnyye
Otsenki Paraemtrov," Academy of Sciences USSR, Institute of
pace Research, Moscow, Report Pr-3P2, 1977, pp. 1-14.
6+. AMstrect
This article considers a generalization of the concept of con-
ditionally-effective estimation (have the best accuracy for
Igiven limitations on the suitability of the algorithm) to the
case when consideration is also given to the steadiness
^(robu:.tness) of the algorithm with respect to the deviation
of the law governing the error distribution from the proposed
? law.
f
c
., Woods %^ S.Iected by Author(o) 	 19. ,^.turrbwtron Steroment 	 t
wnclat;,;ified - Unli=ited
.7. '.trr. r. L(.rt.{.
 
^.{ Fore nyer li	 Se Nnt;	 ..,,.	 .. .I r^^ee	 .. r'ri ee
I
	 4
w
9, PeActm.nq Droenuotren Komi end Addreee
{	 SCiT^.:1.\
Boa 5456
t;.. ..r ..	 ,	 ^^ina
ii
STEADY (ROBUST) CONDITIONALLY EFFECTIVE ESTIMATION
OF PARAMETERS
L. S. Gurin and K. A. Tsoy
Section 1. Formulation of the Problem
	 13*
This study is a direct continuation of the works [1, 21,
which introduced the concept of a conditionally-effective estima-
tion and examined certain particular problems. Estimations which
are optimum for a given criterion in the case of given limitations
are called conditionally-effective. This approach is necessary
because, among the desired properties of the estimation -- along
with properties which characterize their accuracy (independence,
nondisplacement, effectiveness) -- there are others, such as
difficulty of .the estimation algorithms, their stability with
respect to deviations of the laws for error distribution from the
proposed one, etc. (see, for example, [31).
With respect to the stability of the estimation, several
studies have been devoted to this, beginning with the well-known
study of Huber [4]. Of the more recent works, we would only like
to mention [9). These studies, however, do not consider the diffi-
culty of the algorithms. On the other hand, the studies 11, ?_]
consider only the difficulty of the algorithms, but do not
consider the stability. Roth limitations are considered in this
article for a rather simple problem of estimation. The examined
method for obtainin g; the best estimation is sufficiently general
nd can he applied to more complex problems, and the result ob-
ained for a specific problem has already been used in practice.
)Numbers in the margin indicate pagination of original foreign
ext .
1
Previously, when considering this problem, we considered
the simplest case of the direct measurement, which is greatly
influenced by the action of the limitations upon the selection
of the conditionally-effective estimation.
Section 2. Study of the Case of the Direct Measurement
Let us assume there are N measured values X1 of the con-
stant C, containing independent measurement errors of 4`
distributed according to the Laplace law, i.e.,
(1)
It is necessary to find the conditionally-effective estimation
of the quantity C, i.e.,
txr, •••^+^ j n A^	 (2)
so that
	 j^^ C	 67-o", 	 in	 under t: e Condit s on
that the computational time on a computer is'^^ ^'o
Thus, we con.j.der only the limit ption on the difficulty.
If t o is very large as compared with N, so that the limita-
tions on the difficulty are insigrificant, then, as is known
[61, we must set #1 1V and as	 we use the median of the sampl-
ing x` , i.e., we use the method of the least moduli. If the
limitation with respect to t o
 is great, then it is inadequate
to consider only the estimation methods, and we must turn to
specific algorithms. To determine the median, we may use several
algorithms, for example, the following.
A1 . Let us formulate a shortened variational series, i.e., we
find the following (in increasing order) from a part of the
sampling with the volume
V
2
Then	 ex. (for odd n) or	 N«6x^ (for eves; n).
A2 . We use the dichotomy method described in [6].
For comparison, let us examine the method of least squares,
reduced to the algorithm A 3 , i.e., the determination of the
arithmetic mean part of the sampling with the volume n (as is
known, this corresponds to an effective estimation for a normal
law governing the error distribution).
Let us use fl(A3 to designate the computational time on a 	 /5
computer for the algorithm A i for the volume of the sampling n.
The function rzo depends cn the computer used, the language,
the translator, etc. Therefore, they may be o'"cained by the
method of statistical tests. In several cases, we may reach
the conclusions analytically. Thus, let us compare the al-
gorithms A l
 and A 2 . We use m i (n) to designate the mathematical
expectation of the number of comparison operations when using
the algorithm Ai.
We have
	2^
where m.,(n) satisfies the recurrence relationship
Let us prove that
in YL 4•	 (6 )
Let us set
1'1
Then we have the following from [5]
We may prove by induction that (gA)
	 From [ ] , we have
V_
Let us use f^(p) to designate the following function:
^,(M)= mitt	 tp(^) .	 t )
Then we have	 isJ-t
T	 L	
7	
(10)
If tP n^^ ^ SP^p^ , then `i l lgtY2`^^ 	and f (p) will be mono-
tonically nondecreasing. But the condition 	 gives
(taking into account [101)
+h
Let us consider two possibilities: a) beginning with a certain
large n, the condition (11) is always satisfied; b) there is,
an arbitrarily large number of n, for which the condition (11)
is not satisfied.
In the first case, beginning with any n, 1^A) does not de-
crease monotonically, and consequently we have
M Cf,{h)= a	 (12)
A-1aw
i.e., beginning with a certain T)z Y1j, Vm)O t for an arbitrarily
small E" . Using (8) , for any $10 1^ ( for which ^^/ s t^ r^
	 t ^
we obtain
(13)
Passing to the limit at t.
", 
)o:+ao , we obtain
tw#a lt ,	 bt4-	 clu)
u
However, we have	 consequently,
t=4	 (15)
Thus, in this case (6) is proven.
Let us consider the second case. If the condition (11) is
not satisfied for a!W n, then for	 J..^^ ^, ` .,.^	 we have
^w*	 jr Ph)	 (16)
n: (Y}3}
In addition
	 T at yl"+as
Thus, 
,W either lies below A) and does not decrease
monotonical
	
	 Let us consider one ofy, or it lies above f VV
the values of ni.
	
^^pp^j 1^^^,^ ^ tM^j i ^^+'1tt+^ ^ T^ ^nl ^•
	
(17)	 /7
If Tltljj4+1 * tj^ ♦ ^^ , we may not be interested in this case.
Thus, in addition to (17), let us assume we have
T+ ( r4+ +) 'd Pot.+ ').	 (18)
Then, beginning with the value of 41#1	 ^(it)	 again increases.
Let us estimate the difference
	 ItIt is
clear that ^^^lrla ^tA^-i^ and Y^(hW) , if it does not equal
f(II I) , but equals ((#I#) (according to the definition of the
function tP^t^+l) Thus,
(no) -e tv(0 4- 1)-Y(md ; Y(ht- +) —ifth,) >0.	 (19 )
wever,
bil
r ^ i ^ i •
should note that, depending on the evenness of n l , the form of
r,
(20) changes, but we make the estima^ion for the worst ease.
Considering that the left part is positive, we may again discard
the negative terms and we obtain the estimate (we should recall
that WN 6 40
M, ( 21)
1 2< y tg t ---n,-i no	 n,-I
Thus, in every case
We thus find that in this case (6) holds.
A comparison of (4) and (6) shows that for rather large n,
the algorithm A2 is best, even if the noncomputational operations
(which are basically connected with the organization of the cycles) /8
comprise a larger part in the algorithm A 2 than in the algorithm
A1.
When comparing the algorithms A 2 and A 3 , we must consider
that the algorithm A3 is best under the condition
*2b) t VA)	 (23)
(since the dispersions of the estimations are equal, if the
algorithm A 3 uses a sampling volume which is twice as large [61).
The algorithm A 3 for the sampling volume 2n includes 2n-1
additions and one division, and A2 includes 4n comparisons fo r
the sampling volume n. Considering the identical order of these
quantities and the dependence of the computational time on several
factors which are not considered, we may see that the final conclu-
sion may be reached only on the basis of a numerical experiment
with a computer. This experiment was done on the BESM-6 computer,
6
and for the corresponding sampling volumes, gave the ratio of
the computational time of 2.8, in favor of the algorithm A3.
Thus, although the method of least moduli gives an effective
estimation, a conditionally-effective estimation is obtained
with the use of an algorithm employing the method of least
squares.
However, if we consider the stability of the estimation,
then the picture greatly changes. Tt is known that the median
is much less sensitive to "lost" points than the arithmetic mean
(see, for example, 151)• If we pass from the simplest problem
to a more complex one, even to problems of linear regression
analysis, then the picture is greatly complicated. However, the
statements made in this section show that to obtain stable, condi-
tionally-effective estimations, we must use combined algorithms
which use both the method of least squares and the method of
least moduli. Let us consider the corresponding problem.
Section 3. Study of the Problem of Linear Regression
We shall use the model
YK = Q # &;X&' ` ^ ` j Z = lo 2 ,,  IV	 (24)
where y  are the measurement results, A -- the values of the
independent variable, and the measurement errors 
rk are inde-
pendent and have a distribution which is similar to the normal
distribution, more precisely, the density of the distribution
probability ?z has the form:
/ CX	 (X) +	 tic)	 (25)
in formula (25)
xZ
/=(z) s a	 (26)
^x^ o	 fxt^a.
i
/9
7
At a - 0, we obtain the normal law for the error distribution.
The following algorithms are considered (estimations).
Al . The customary algorithm for the method of least squares
(MLS).
A2 . The algorithm for the method of the least squares with
preliminary grouping. Thus, N values of the independent variable
(we assume that the measurements are performed for equal
values of x) form n groups with respect to m = N/n points. We
have
x
It is clear that, instead of (24), we obtain the problem
MO	
(28)
and the estimations C ^^ , obtained by the regular MLS, may be
used as the estimation for a and 8 , i.e.,
as st ; 4.2
Below, we shall use 
^9 ty to designate the estimations of a
and 6 obtained using the algorithm Ay. For greater determinacy, /10
we introduce the second index m for estimations with grouping.
This index designates the number of points in the group, for ex-
ample, we shall designate definite estimations as follows
,► 	 ^
C^^ p, : C s ^^,^ (29)
A 3 , Combined algorithm with preliminary grouping. In this
	
case, in contrast to A 2 , in (27)
	
is replaced by the median
of the corresponding group of values y i , and we shall designate
It by y;. Then, instead of (28), we obtaino
^^• V.e f	 t^t f 	 n.	 (30)
and, consequently,
8
rA
where ,C and
	 -- estimations of a and f from (30), obtained
by the regular MLS.
Thus, we have ^ h ^following set of estimations:
6^^^i^l^^^^Ql,*S^tw (the second index assumes several
values).
To compare the estimations, the method of statistical model-
ing was used. We set 429 a; A994 in the model of (24) . In addi-
tion, the case of a - 0; 0.05; 0.25 was considered in (25). In
formula (26), we have a - 30. In addition, at N - 15, we assume
m - 3; 5 and at N	 45 m - 5, 5, 15. For each estimation, using
the results of M
	
100 realizations, we obtain the average com-
putational time on the computer f (in seconds per 100 realiza-
tions), the average displacement p, and the average dispersion
Q 1 . The results of the experiment are given in Table 1-2.
Section 4. Conclusions and Problems of Further Research
1. At a - 0, grouping is not advantageous, as would be ex-
pected from theoretical considerations.
2. For the linear model, the individual algorithms differ
very little based on the computational time. 	 /13
3. For deviations of the normal distribution (a > 0), the
algorithm A 3 is more advantageous than A2 , since it leads to in-
creased accuracy. Thus, for large values of a, it is better to
use algorithms with large values of m (for great deviations from
normal distribution, the estimation in the form of the median is
more effective than large groups).
Considering that in real problems we may expect smaller values
of a, on the basis of the conclusions given above, we may recommend
the algorithm A 3 with the value MZ^
9
11TABLE 1
QUALITY INDICES OF ESTIMATIONS AT N a 15
Algorithm(estimation) AL 3
0 0505 0 9 25 0 Uyl^S	 .. 0^%a
I	 ^ 7yI ^
p q,, io90987,0? 1310 ? 7307
t^-o ^r 0,.0942 _ 1 1103 —090030
4-9996112292970!f (a,^ I	 0 S 2335 ;
AD 0,00371 0,0789 +1 0,35581
i
I T`	 t 1 7.,52 1 7,89
a 0,v892 ^--0^,_t150 C.v3 ;7 0i50. -^7^C-371 099I33i	 3 ! 0,"0G^7
j
0,.I+47 0,C06I C,0667 Ii 0 9 1033 1--0,0067
V 9 21Z4;0 ^ .,7712 124 s8 40 1 0 t ,0S2 , C 9 8221
I
99V670
Qt l^ ( C, 003?; C,0705 1 0,40uI^ 0 9 0049 0 5,C088 09,I4
t ?;2= 7,39
c 	 w w
^J 1 A M[
n	 r _Uyt.:.
f. r '
-., yviJt: ^	 7C,' 5S0 ,j I"1^s^,i Lyv^ ;J ^ C/y^call
f G wU v^+ .,'' 0i s1 1J^$ 1J^O^CV{ O,OS, 5v 0 ) oc ^5 OpCi..,.
s,V2.z 't 0^ C,-1 ".V7^ 0 ;1 7vi v:,,2
I Qt ;L,^ 00401 G s C9"00 0.,4520 C.COG3I 0,.00'rI
10
TABLE 2	 /12
QUALITY INDICES OF ESTIMATIONS AT N 45
AL A3
V. 0-ar ear
0, cs^ 0 V5	
06
mzl 0 9.0-V1 0% 0 9 i llou"! 0, 037
(A#) ^700	 98 655 73093
T o o 016 0 ,. 0022 0 9 C-1-10
!^ ^ ^	 i : ,may 	 ^ ^ i ^^^
0 13 a I 0, 0?2,d6,1 -0 ci'^-'	 C43
0 ,0 :"TO 0 2 TO"2 ' 07	 iz5l
al zncz: ^..296 , 2 ^0 9 :92	 .	 a
tp)
LiS VA. v 0002
o"r. 
vuw,	 uy
r7
0	 8 ^4; --0 q 0.-656 !--C ,9-107 .6 2.1. 71 -0 C C -i09T
7
f A,
7 S -B,3 0,, .r,. 5 0, 4 %J E'	 T	 -'72
W o p t% c
0 V: 02 3	 o c
Is ":-7
nW
CC- =
Aw	 & -4
of V t C -21
0RIGINAL PAGE 18
f)V POOR QUALM
1. Estimations of nonlinear models are considered. In
this case, there must be a greater difference between individual
algorithms in terms of difficulty.
2. For the problem considered of linear regression, al-
gorithms are additionally studied which are based on excluding
the lost points I7, 81.
REFERENCES
1. Gurin, L. S., Tsoy, K. A. Use of statistical modeling in
the development and comparative analysis of methods for
estimating parameters. In the Collection: Obrabotka
kosmicheskoy informatsii (.Processing of space informa-
tion). Moscow, Nauka press, 1976, pp. 36-57•
2. Gurin, L. S., Tsoy, K. A. Conditionally-effective estima-
tions of parameters of nonlinear models. Preprint.
Moscow, IKI, AN SSSR, Pr-257, 1975, 34 pages.
3. Statisticheskiye metody - eksperimental'noy fizike (Statis-
tical methods - experimental physics). Moscow, Atomizdat,
19769 336 PP•
4. Huber, P. J. Robust estimation of a location parameter.
Ann. Math. Stat., 1964, 35, No. 1, 73-101.
5. Polyak, B. T., Tsypkin, Ya. Z. Noise-resistant identifica-
tion. In the Collection: Reports of the IV Symposium of
the IFAK "Identification and estimation of system parame-
ters," Part I. Tbilisi, Izdatel'stvo Metsniereba, 1976,
PP. 190-213.
6. Mudrov, V. I., Kushko, V. L. Metody obrabotki izmereniy
(Methods of processing measurements), Moscow, Sovetskoye
radio, 1976, 192 pp.
7. Thompson, W. R. Ann. Math. Stat., 1962, 6 1 214.
8. Dixon, W. J. Biometrics, 1953, 9 9 74.
/14
12


Steady (robust) conditionally effective estimation of parameters

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server