Komenda, Stanislav

Lukš, Antonín

Institute of Mathematics AS CR, v. v. i.

KybernetikaAntonín Lukš; Stanislav KomendaVariance of estimator of a discrete parameterKybernetika, Vol. 16 (1980), No. 3, (273)--284Persistent URL: http://dml.cz/dmlcz/125115Terms of use:© Institute of Information Theory and Automation AS CR, 1980Institute of Mathematics of the Academy of Sciences of the Czech Republic provides access to digitizeddocuments strictly for personal use. Each copy of any part of this document must contain theseTerms of use.This paper has been digitized, optimized for electronic delivery and stamped withdigital signature within the project DML-CZ: The Czech Digital Mathematics Libraryhttp://project.dml.czK Y B E R N E T I K A — V O L U M E 16 (1980), N U M B E R 3 Variance of Estimator of a Discrete Parameter ANTONÍN LUKŠ, STANISLAV KOMENDA Studying statistical properties of a simple probabilistic model of the school achievement test authors faced the problem of expressing the amount of information about a parameter contained in an empirically observable quantity in the case when the parametric space is to be considered as discrete. In this paper a measure of information is proposed, which can be applied in this situation. 1. A PROBABILISTIC MODEL OF THE SCHOOL ACHIEVEMENT TEST It is supposed that the matter the student is obliged to master represents a mea-surable space \Q, S4\ where Q is an abstract set and si is a er-algebra of its subsets. After a period of education this universe can be partitioned into two classes A, A' e e si. A is a set of the facts mastered by the student and A' a set of the facts unmastered by him. We suppose that the measurable space \Q, si] can be turned into a measure space [£2, si, P] in such a way that it is meaningful to speak about the proportion 7i = P(A') of the matter unmastered by the student. The measure space [£3, si, P] is identified in a natural way with the Kolmogorov probability space. The student is put to a school achievement test which consists of questions s;, i = 1, . . . , n, and some offered answers r y , i = = 1, . . . , n, j — 1 , . . . , q (q _ 2) to these questions. For every i it holds that the concept represented by the answer riy to the question s ; is a fact for just one j , i.e. just one offered answer to a given question is correct. We assume that the school achievement test is made and used in such a way that a student, having unmastered n 274 of the matter required, knows the correct answer to a given question with the proba­bility 1 — ?r, and does not know it with the probability %. See [2, 3]. The number Y of questions of the test the student does not know is, accordingly, a random quantity which has the binomial distribution, P(y | rO = Q ^ ( l -«)-'• (See Fig. 1.) However, the student can guess the correct answer to a given question from q offered answers. Thus the probability p of the student not answering the given question correctly, is given by the formula q - 1 p = %. 1 The number X of questions of the test the student does not answer correctly, is, accordingly, a random quantity, which has the binomial distribution, P(x\p) = (^p*(l-pГ *l->-Q( -- 1 - q - í We state yet the conditional distribution of the number X of questions of the test the student does not answer correctly, given the number y of questions of the test the student does not know, P(yKr) I P(x\y) PЫp) Fig. 1. Scheme of conditioning, dependence on parameters and forming the mixed model. (1) P(x I >) = (For the statistical inference about parameters, see Fig. 2.) If a value x of X is known, x\n is an estimate of the quantity p. The estimator X\n is the unique unbiased estimator for p, as can be proved. The relation n = [q\(q — l)] P implies that [q\(q — 1)] (X\n) is a reasonable estimator for the parameter %, although for x ^ (q — 1)1 q it takes on values greater than 1. The estimator [q\(q — 1)] (X\n) is unbiased. The quantity y unknown by us can be considered a parameter as well. As will be proved in what follows, [q\(q — l)] X is the unique unbiased estimator for this quantity. The situation is rather unusual in that this parameter is discrete. Since Y\n would be the unique unbiased estimator for the parameter n, if Y were observable, we conclude that the estimator (ijn) [q\(q — tf\X will be a reasonable estimator for the parameter n. But 1 Q „ Q X n q — 1 X = q - 1 so that the two ways of reasoning lead to the same estimator. (See Fig. 2.) -Jrx -I Fig. 2. Scheme of inference. For the estimator of n in the mixed model see text. We can find easily that Fisher's measure of information about the parameter n yielded by the quantity X [4] is given by the formula •G-V-)' 276 As for the rest of the problems of the estimation theory, as indicated above, see Fig. 3. As for the estimation of the discrete parameter y, Fisher's measure of information cannot be applied. The purpose of this paper is to introduce a simple measure of information, suitable as a substitute in this situation for that commonly used. f ? > У see text / чrц-r) P(1-p) Fig. 3. Information scheme. Our problem is indicated by a question mark. For Fisher's measure of information in the mixed model see text. Let us show that in the case of the distribution \q\(q — 1)] X is the unique unbiased estimator of y. Proof, (a) Above all, it is unbiased, for q - 1 Í - 2 - X\y) = - « _ E(X\y) ш-Я-y^. шy (b) If any estimator t(X) is unbiased, then it satisfies the relation (2) Mf-™ for every y. We shall provide an inductive proof that it is necessary for this estimator to satisfy the relation (3) t(x) = [ql(q-l)-]x for every x = 0, 1, 2, Substituting 0 for y in (2) we obtain that t(0) = 0 or that (3) is valid for x = 0. Let x = 0, 1, 2, . . . . If (3) holds for every integer x, 0 ^ x ^ x, 277 then substituting 3c + 1 for y in (2) we have X+1 = 1 = o q — 1 which yields the following equation = o a — l \ x j \ ^ / \ t j j \ q -f-4-.í' ,+v-^Yíir-+-Lc+')rt t=o q — 1 \ x j \ q j \<zj <2 — 1 \ ? whence, necessarily, So (3) holds for x, 0 g x :£ X + 1, which proves the proposition. ť(- + l) = -J (* + l). « - 1 2. A MEASURE OF INFORMATION It is well known [4] that the ability of a random quantity to yield information about the value of a parameter can be judged on what change of distribution of the quantity will be brought about by a change of the value of the parameter. In the case of a continuous parameter, when infinitesimal change is possible, using various metrics [4, 5] one always arrives at the same, Fisher's measure of information. It cannot be expected that this will be preserved in the case of a discrete parameter. Now, we shall apply some formulae from [5] for the mean information for discri­mination between hypotheses to assess the change of distribution (2) induced by augmenting y by 1. Denote ny the probability measure determined by the distribution (1). We shall see to the fact that fiy_1 < fiy (read: fiy^± is absolutely continuous with respect to fiy), but not fiy <g /*,_«.. The distribution (1) is discrete, ny < X, where A is the counting measure of the countable set {0, 1, 2,...}. The validity of ny-i -4 (iy suffices for the quantity /(y - 1 : y) = £ P(x j y - l)iogff*J y ~ ^ , p(x|y-i)>o P{x | y) y = 1, 2, ..., to be well-defined. This quantity can be interpreted, but solely for il-lustrative use in this section, as the information J(y) yielded by the random quantity X about a value of the parameter y. Unfortunately, this interpretation is ambiguous. We can také as well as J(y) = í(y - 1 : y ) , y = 1, 2, J ( y ) = l ( y : y + 1), y = 0 , 1 , 2 , . . . (For this "ambiguity" and numerical values, see Tab. I.) 9 = 2 9 = 3 9 = 4 9 = 5 7(0: 1) 0.6932 1.0986 1.3863 1.6094 7(1 : 2) 0.3466 0.6365 0.8664 1.0549 1(2: 3) 0.2158 0.4301 0.6163 0.7766 / ( 3 : 4) 0.1521 0.3159 0.4686 0.6059 7(4: 5) 0.1161 0.2453 0.3723 0.4907 / ( 5 : 6) 0.0937 0.1985 0.3055 0.4084 K6: 7) 0.0785 0.1658 0.2572 0.3472 / ( 7 : 8) 0.0676 0.1420 0.2209 O.ЗOOЗ / ( 8 : 9) 0.0594 0.1240 0.1930 0.2635 1(9 : 10) 0.0530 0.1100 0.1710 0.2341 3. AN ANALOGY OF THE RAO-CRAMER INEQUALITY The theory of the efficiency of estimation is known only for a continuous para­meter. It concentrates around the Rao-Cramer theorem, furnishing Fisher's definition of the measure of information with a solid basis. We should like to demonstrate an analogy of this theory for a discrete parameter, which is very transparent, includes an analogy of the Rao-Cramer inequality and is a basis, too, we believe, that a defini­tion of the measure of information can rest on. Let (a) X be a discrete random quantity which can assume nonnegative integral values x = 0, 1, 2 , . . . . Consider a discrete system of distributions of the random quantity X dependent on a discrete parameter y = 0,1, ...,N. This system of distributions of the random quantity X is described by the probabilities P(x\y), x - 0 , 1 , 2 , ' . . . , y = 0, 1, ...,ІV : where co 279 Y _ P ( x | y ) = l , y = 0 , l , . . . , N . Suppose (b) P(x | y - 1) > 0 => P(x | y) > 0 for every x, y = 1, 2, 3, ... . Denote P(-/.)>o y = 1, 2, ..., JV, supposing (c) that the right-hand sums are all finite and positive. Example 1. (School achievement test.) Let the random quantity X have the binomial distribution, ^-©eřTO where q is a known integer, 9 = 2 , 3 , 4 Note that P(x j y) > 0 if and only if 0 = x = y. Now, P(x I y - 1) y - x q .__fek_^____(ł_1)-_(__i__ţЛ, *>-£®"(I-1т1'),'W' )-ï Y D ( X | , ) - f í Y , Î ^ Ì . Ł ^ , , - ,,2,...,N. Thus, (4) j(y) = «-^l, y = l,2,...,JV. y Note that this quantity takes on only finite positive values. Example 2. Let a random quantity X have the hypergeometric distribution 0 O ^ x ^ n , n S ^ N - n, where rc and N are known integers. The algebraic expres­sion for J(y) is found to be irreducible, appropriate only to numerical computations. -80 Theorem. Let T = t(X) be an estimator for y that has finite second moment for y = 1, 2,..., N. Let b(y) = £(T) - y, y - 0, 1, . . . , N, be the bias of the estimator T Let the assumptions (a), (b), (c) hold. Then (s) E{T-yf^l + bw-b(y-w, J(y) y = l,2,...,N. Proof. The function b(y) is defined implicitely by the formula tt(x)P(x I y) = y + b(y), x = 0 y = 0, 1, ...,N. Therefore (6) I t(x) [P(x | y) - P(x | y - 1)] = 1 H- %) - K>' - 1), 1 = 0 , -= 1, 2 , . . . , N. Moreover, (7) I y[P(x \y)-P(x\y-1)1 = 0, x = 0 y = 1, 2, ..., JV. Subtracting (7) from (6) and after slight modification we obtain E [<*) - y] h - ^ r y ^ i p(x \y) =1 + b(y) - b(y - «• P(*b)>0 ' On the left-hand side there is essentially a mean value of a product of two mea-surable functions. According to the Schwarz inequality [ l ] , (8) [1 + b(y) - b(y - I ) ] 2 = = Z [t(x)-y]2P(x|>) I r - - * l l y | " 1 ) T - f r l J 0 -x x P(x y) P(x\y)>0 P(x\y»0 ~ V | / _• It proves the inequality [l + b(y)-b(y-l)Y^E(T-yyj(y). It is equivalent to the assertion (5) by virtue of the assumption that the value of J(y), for y = 1,2, ...,N, be finite and positive. Q.E.D. Now, we shall investigate into when, under the assumptions of the theorem, the equality in (5) is attained. As can be seen, this equality is attained if and only if the equality in (8) is valid. But the Schwarz inequality turns into an equality, if either (9) l[t(x)-yYP(x\y) = 0, or there exists such a function K(y), y = 1, 2, .... 2V, that (io) 1 _ M _ > L ^1 ) = K(,)[<x)- J] P(x | y) for every x, y, P(x | y) > 0. The first case is an absurd case of an exact estimator T of the parameter y, which need not be taken into considerations. (By the way, this case can be eliminated on a purely logical basis, too. [l].) In the second case, we constrain ourselves to x, y's such that P(x j y) > 0. If x, y's are such that, moreover, P(x I y — 1) > 0, then the equality (10) can be modified to the form (ii) P(* I y) = : - -.*—-r^ fy \y - i) • 1 + K(y) [y - t(x)] If y is the least of the values for which P(x I y — 1) > 0 then P(x I y) is not deter-mined by the relation (10) and can be chosen arbitrarily, but £P(x I y) — 1 for y = 0 ,1 , 2 , . . . . So the recurrent relation (11) characterizes essentially the type of distribution for which there exists such estimator T = t(X) for y for which the equali-ty in (5) holds. Example 3. Let us return to the case we have treated in Example 1. Here P(x | y) > > 0 if and only if x ^ y. The equality (11) runs as follows: Weput P(x | y) = - J L . i P(x | y - 1). y - x q l+K(y)ty-t(x)-] = &—ÊЛ У [y - t(x)] K(y) = W ~ qX -1 = ІL-VУ-Æ =<Ľl±(y-^L - 1 g — I y Accordingly, the equality in (5) is attained for the estimator T — [q\(q — 1)] X of the parameter y. Example 4. Let us return to the case treated in Example 2. In a similar way as in Example 3 we arrive at the conclusion that the equality (11) runs here as follows: Put P(X \ y ) - - J - N-n + l-(y-X) , } V i y y - x JV + 1 - y V 1 / ì + вдь-^)]-'-* N + í - y y N - n + 1 - (y - x ) ' (.V - n + 1) y - (y - x) y Apparently, it is not possible to find K(y) and t(x) such that this equality be satisfied. The equality in (5) is, accordingly, not attained for any estimator Tof the parameter y. From the theorem it follows immediately that for every unbiased estimator T with finite variance of the parameter y it holds D(T) > — , for y = l,2,...,N. The efficiency e of an unbiased estimator T with finite variance is defined as J(y) D(T) Notice that this is, in fact, a system of N numbers. It holds, of course, 0 ^ e :S 1. In the case when e = 1 for every y = 1, 2, ..., 7V, call the estimator Tefficient. Example 5. Let us continue the treatment we have begun in Examples 1 and 3. Since E(X\y) = y^-±, q. the estimator r - - i - j f q - \ of the parameter y is unbiased. We know that the equality in (5) is attained for it, or that it is efficient. Independently of it we can compute its variance as follows: m - (-S-Y op) . (_!_)', !^ i 1 . -_L \ 4 - V \ « - V 4 4 4 - 1 Then 1 1 J(y) D(T) ҙj-1 ^ ^ y q-І = 1 . If one believes that an estimator T is efficient, he can define J(y) by the equality 283 J(y) = 1/D(T). This has led one of the authors (A. L.) to all the theory, rather trivial perhaps, for the estimator treated is the only unbiased one. Example 6. In the case we have treated in Examples 2 and 4, apparently there exists no efficient estimator of the parameter y. 4. THE MEASURE OF INFORMATION The function J(y) defined by the following formula P(x\y)>0 u v l J ' y = 1, 2, ..., N, is now considered as a measure of information about the parameter y, contained in a random quantity X. In practice, we need not stick to the condition 0 < J(y) < oo and can define the value J(0), too. Example 7. The definition of the measure of information, which is expressed by the formula (4), can be extended to y = 0, J(0) = +oo. The formula j , y ) = y [P(x\y)-P(x\y-l)y P(x I y) is equivalent to the foregoing one. 5. DISCUSSION In our paper, we have been motivated primarily by the effort to find a suitable solution in the case of school achievement test introduced above. However, the analogy of the Rao-Cramer formula presented here, can be generalized in many directions. First of all, one need assume that the univariate random quantity is discrete; it can be continuous, too. Into the formulae, some generalized densities can be introduced, similarly as -'n Kullback's theory [5]. Further, even the assumption that the parameter takes on positive integral values is not necessary, although appropriate. Thus, we may derive an inequality for discrimination between two states of our knowledge, between two hypotheses, which can be applied even in the continuous case. Here, however, the equality may be a simple consequence of the well-known Rao-Cramer inequality. The analysis of the effect of change of parameter on the distribution of a random quantity leads to analogies of Fisher's measure of information, the appropriate choice of metric being decisive in the discrete case. The two metrics treated here are statistically related to the two types of tests of good fit: the likelihood-ratio test (Kullback's metric) and the %2-test (our metric), the latter having proved to be the handier one. (Received April 18, 1979.) REFERENCES  [1] J. Anděl: Matematická statistika. SNTL, Praha 1978. [2] S. Komenda, J. Klementa: Modelování a analýza vlivu náhody v didaktickém testu. Matema­tika a fyzika ve škole 9 (1978/79), 311—318. [3] S. Komenda, J. Klementa: Ein wahrscheinlichkeitstheoretisches Modellfureinen didaktischen Test und seine statistische Analyse. Wiss. Z. (Halle/S.) XVI (1978), 3—9. [4] C. R. Rao: Lineární metody statistické indukce a jejich aplikace. Academia, Praha 1978. [5] S. Kullback: Teorija informacii i statistika. Moskva 1967. RNDr. Antonín Lukš, Ústav pro výzkum vyšší nervové činnosti Univerzity Palackého v Olomouci (Institute of Higher Nervous Activity — Palacký University), I. P. Pavlova 13, 775 20 Olomouc. Czechoslovakia. RNDr. Stanislav Komenda, CSc, Katedra lékařské biofyziky a nukleární medicíny Lékařské fakulty University Palackého v Olomouci (Institute of Medical Biophysics and Nuclear Medicíně — Palacký University), Dr. S. Allende 3, 775 15 Olomouc. Czechoslovakia. 

English

Variance of estimator of a discrete parameter

Czech digital mathematics library

http://dml.cz/bitstream/handle/10338.dmlcz/125115/Kybernetika_16-1980-3_6.pdf

Variance of estimator of a discrete parameter

Abstract

Similar works

Full text

Available Versions

Institute of Mathematics AS CR, v. v. i.

Czech digital mathematics library