In the context of recapture sampling design for debugging experiments the problem of estimating the error or hitting rate of the faults remaining in a system is considered. Moment estimators are derived for a family of models in which the rate parameters are assumed proportional to the tail probabilities of a discrete distribution on the positive integers. The estimators are shown to be asymptotically normal and fully efficient. Their fixed sample properties are compared, through simulation, with those of the conditional maximum likelihood estimators

Gupta, Rajan

Lee, Larry D.

English

NASA Technical Reports Server

4 
DEPARTMENT OF MATHEMATICS AND STATISTICS 
COLLEGE OF SCIENCES 
OLD DOMINION UNIVERSITY 
NORFOLK, VIRGINIA 23529 
ESTIMATION IN A DISCRETE TAIL RATE FAMILY 
OF RECAPTURE SAMPLING MODELS 
BY 
Rajan Gupta, Graduate Research Assistant 
and 
Larry D. Lee, Principal Investigator 
Final Report 
For the period ended February 28, 1990 
Prepared for 
National Aeronautics and Space Administration 
Langley Research Center 
Hampton, VA 23665 
Under 
Research Grant NAG-1-835 
Dr. Dave E. Eckhardt Jr., Technical Monitor 
ISD-Systems Architecture Branch 
Submitted by the 
Old Dominion University Research Foundation 
P.O. Box 6369 
Norfolk, Virginia 23508-0369 
February 1990 
( N A S A - C R - Z Y 0 4 " L )  tSTIMATTdN I% A U T S C R F T F  N90- 1 R 19 7 
T d I L  R A T E  FlMllY OF KFCAPTURt LAMPLING 
Y?UFLC F i n d l  R e p o r t ,  p e r i o d  endinq 7 8  FpD. 
1Q9n ( f ? l d  Sopinion univ,) 14 D C S C L  1 2 A  
&79/ 
uncl a s  
6 3 / 6 4  0264dQ)a 
https://ntrs.nasa.gov/search.jsp?R=19900008881 2020-03-20T00:05:30+00:00Z
ESTIMATION I N  A DISCRETE TAIL RATE FAMILY 
O F  R E C A P T U R E  SAMPLING MODELS 
Rajan Gupta and Larry Lee 
Department of Mathematics and Statistics 
Old Dominion University 
Norfolk, VA 23529-0077 
ABSTRACT 
In the context of a recapture sampling design for debugging experiments we 
consider the problem of estimating the error or hitting rate of the faults remaining 
in a system. Moment estimators are derived for a family of models in which the 
rate parameters are assumed proportional to the tail probabilities of a discrete 
distribution on the positive integers. The estimators are shown to be asymptotically 
normal and fully efficient. Their fixed sample properties are compared, through 
simulation, with those of the conditional maximum likelihood estimators. 
Key words and phrases: software reliability; asymptotic eficienc y ;  conditional 
likelihood; average information; interval truncated sampling. 
1 
1. INTRODUCTION 
Nayak (1988) recently proposed a recapture sampling design for estimating 
the number of errors or faults remaining in a system. As is common in debugging 
experiments, a system is tested for a time period of length 7 ,  the failure (i.e., error 
detection) times Tl,T2,. . ., T, are observed, and repair takes place immediately 
after a fault produces an error. By using standard error detection techniques (e.g., 
Avizienis and Chen, 1977) the hitting frequency M; = M;(T;, 7 )  of the fault detected 
at time T; is observed as the number of times the region (i.e., path in software) 
containing the fault is accessed during the interval (Ti, 7 ) .  Nayak’s (1988) discussion 
concerns the Jelinski-Moranda (1972) model A i  = (v-i+l)r$, 4 > 0, i = 1,2,. . . , v 
where v, a parameter, is the initial number of faults in a system. 
The purpose of the present paper is to study estimation procedures related to 
the following model. The spacings yi = T; - Ti-1 (To = 0), i = 1,2,. . . are assumed 
independent and exponentially distributed with rate parameters A; given by 
A; = a! q i  - l,+), CY > 0, E; = a! g( i , f$ )  (1) 
That is, A; is the rate of encountering the remaining faults after i - 1 faults have 
been removed and [; = A; - X;+l are the hitting rates of the first, second, etc., 
detected faults. In (l), G(z,t$) = 1 - G(z,$), is the distribution function of a 
discrete positive random variable and g(z, 4) is the density function or probability 
mass function of G(z,$). The quantity ti can be interpreted as the amount by 
which A i  decreases when repairing the fault detected at time Ti. Counts {M;( t ) }  
of repeated error occurrences are assumed to be independent homogeneous Poisson 
processes with rate parameters &. 
In this context the J-M model is given by a discrete uniform distribution with 
mass at 1,2,. . . ,v. This model, however, assumes that faults have a common 
rate e; = 4 whereas experimental investigations (e.g., Nagel, Scholz, and Skrivan, 
2 
1984) indicate that faults may have different hitting rates. The log linear rate 
model A; = cxe-P(;-l) (Cox and Lewis 1966) corresponds to a geometric distribution 
and describes the case €1 > €2 > ... in which faults having the highest hitting 
rates are detected early in the debugging process. Other models that seem to 
be related to (1) are those of Sandland and Cormack (1984) and Miller (1986). 
For our purpose it suffices to take g ( i , t $ )  to be the discrete exponential family of 
densities g ( i , t $ )  = exp[a;t$ - $(t$) + ai]. These models yield a sufficient statistic of 
smaller dimension than obtained in general, although for most families the likelihood 
function (Section 2) does not have exponential family structure. 
The main problem we consider is that of estimating the error (or hitting) rate 
A r + l  for a system in its final state; i.e., a system for which R = r faults have been 
removed. Moment estimators of (a,t$) are presented in Section 3. Their bias and 
asymptotic variances are compared with those of the maximum likelihood estimators 
in Section 4. In Section 3 we show that functions of the form r-1/2 ln(&+l/Ar+i) 
have a limiting (r  + 00) normal distribution under various models. The conditional 
likelihood function given in Section 2 defines the setting of our discussion. 
2 .  A CONDITIONAL IKELIHOOD 
We assume that a system is tested until no errors are detected for a time 
period of length s. Data is obtained through interval truncated sampling by which 
we observe Tl,Tz,. . . ,TR and R = t providing Y; 2 s , i  = 1,2,. . . , r and Yr+l > s. 
With Y1, Y2,. . ., being independent exponential random variables, the condi- 
tional density of Y1, Y2,. .. , YR given R = r ,  is 
3 
The total test time 7 = 
is a fixed quantity. 
yi + s is random while in Nayak's (1988) discussion, T 
The full data vector can be represented in terms of the vector quantities zk 
defined by 
Here k f i k , i  < k, is the number of times the system encounters the ith detected 
fault during the interval (Tk-l,Tk]. The last interval (Tr,7] has fixed length s while 
the remaining intervals ( T k - 1 ,  Tk], k 5 r ,  have random length Y k .  For notational 
convenience, we let Yt+l = s .  
Our earlier assumption that {Mi ( t ) }  are independent homogeneous Poisson 
processes together with Y1, Y2,. . . , Y,. being independent implies that Z 1 , Z 2 , .  . ., 
Zt+l  are conditionally, given R = r ,  independent with densities 
Substituting A i  = aG(i - 1,4) and ti = ag(i,4) in (4), the log likelihood is 
I ,  = xi+' Ik where 
Here Yt+l = s ,  C does not depend upon (a,4), and 
ck(a,4) = ln[Xk(l - k = 1,2, .  . . , r  
= Xr+ls, k = r + 1 
4 
Let vk = ( V l k ,  V 2 k ,  V 3 k ) ,  k = 1 ,2 , .  . . , r + 1 (prime denotes vector transpose) be 
defined by 
k-1 k- 1 
i=l i= 1 
where a; are constants. 
The following moments are needed to obtain the average information matrix 
and also in Section 3, to study the asymptotic distribution of S, = E;=, v k .  
These moments can be obtained by noting that Y k  has an exponential distri- 
bution truncated over the interval (0,s) and that {&} are conditionally, given 
Yk, independent Poisson random variable with means & Y k .  Since V 3 k  is a h e a r  
function of & , z  < k, these moment are similar to those given in [3]. 
By taking derivatives and expectations, the Fisher information matrix A k  = 
( a i j k ) ,  based on f k ,  can be obtained as follows: 
5 
Since Yk converges (as k + 00 ) in distribution to a uniform distribution on the 
interval (0,s) the moments of Yk converge to the corresponding moments of the 
limiting uniform distribution (Serfling, 1980, p.14 ). We thus have limplk = s/2, 
limallk = s2/12, and lim Xk = 0 as k + 00 Assuming that g(i, 4) is a regular family 
with support not depending upon 4, the limits 7; = lim7ik, i = 1,2 are given by 
where 7 2  is the Fisher information about the parameter 4 based on a single ob- 
servation from g(i,4). Thus 71 = 0 and the limiting average information matrix 
A = lim(l/t)(Al + A2 + . . . + Ar+1) is A = (a i j )  where all = s/2, a12 = 0, and 
a 2 2  = a72s/2. 
3.  ESTIMATION 
We now consider exponential family rate models given by 
f i  = aexp[b ( i )  - $($) + b( i ) ] , i  = 1,2, .  . . . (7) 
where q5 varies over the natural parameter set {q5 : xi"=, exp(c$a(i) + b ( i ) ]  < 00). 
This family includes the Poisson (ti = afli-lc8/(i - l ) ! ,O > 0 , i  = 1,2, .  . . .) and 
log linear model as well as other models. 
Let v k  be defined as in (6) where ai is the coefficient of 4 in (7). In reference 
to I ,  = XI;" Lk defined by ( 5 ) ,  we have the following: 
(i) VI, Vz, . . . , Vr+l are independent. 
(ii) Sr = E);+' v k  is a sufficient statistic for the family defined by 1,. 
(iii) S: = (Slr ,S2r,S3r) is given by SI, = 7, S2r = C ) ; M ; ,  S3r = ajM; 
6 
THEOREM 2. Under (7) where a; 2 0 is nondecreasing in i = 1,2,. . ., (l/r)Sr has 
a limiting (as r + 00) normal distribution with mean vector p’ = ( p l , p Z , p g )  and 
covariance matrix ( l / r )C given by 
p1 = s/2 p2 = a 4 2  p3 = as$’ 
c11 = 2 / 1 2  ~ 1 2  = as2/12 ~ 1 3  = ( ~ ~ 2 / 1 2 ) + ’  
~ 2 2  = C Y S / ~  + a2s2/12 ~ 2 3  = ~ 2 2 t , V  ~ 3 3  = (cYs/~)$” + ~ 2 2 ( $ ‘ ) ~  
The proof is given in Section 5.  
Note that a =  $7l(Pl,P2,P3) and 9’(4) = $72(p1,C12,C13) where 91(21,22,23) = 
22/21, 92(21,22,23) = 23/22 ,  Applying the &method, the estimates (&, 4) given by 
r r t 
i= 1 i= 1 i= 1 
have a limiting normal distribution with mean vector (a, 4), and are asymptotically 
independent with variances u~~ = 2a/rs , ai2 = 2[ras$”(4)]-’. 
In estimating X r + l  by i r + 1  = & G ( r ; J )  we must account for the fact that 
r increases as r1/2(& - a) and r1l2(J - 4) converge to their limiting distribu- 
tions. For the log linear rate model X i  = ae-fl(i-l) with 4 = -p,p > 0, we have 
~-1 /2  ln(ir+ 1 / A r +  1) = r1/2( 4 - 4) so that by Theorem 2, r-lj2 ln(&+i/Ar+1) has a 
limiting ( r  + 00) normal distribution with mean zero and variance 2[as$~”(4)]-’ = 
2e4(e-4 - I ) ~ ( C Y S ) - ~ .  
To deal with the other models in Table 1, we apply Taylor’s formula to H ( 4 )  = 
- In G(r;  4) and obtain 
where I+* - $1 < 14 - 41. The first term of (8) converges in probability to zero. 
Under the Poisson and logarithmic series models, r-11H’(q5*)l converges to 1, and 
7 
r-3/2H"(4*) converges in probability to zero. Thus for all of the models of Table 
1, the limiting distribution of r - l i 2  ln(Xr+l/Xr+l)is identical to that of r1I2(4 - 6) 
and is a normal distribution with mean zero and variance 2[a!s$~"(4))-~. 
4. EFFICIENCY AND BIAS 
Since = 2a!/(rs) and 0 4 ~  = 2[ra!s9!J"(4)]-' where 9!J"(4) = 7 2  is defined at 
the end of Section 2, it follows that ti and 6 are asymptotically fully efficient. 
To study the fixed sample properties of ti and 6,  we simulated their values for 
the Poisson rate model under the conditional likelihood defined in (6). Th' 1s was 
done by generating 200 replicates of (2'1,2'2,. . . , 2'7, M I ,  M2,. . . , Mr) for the values 
of r shown in Table 2. In addition the conditional maximum likelihood estimates 
bC and 4, were calculated for each replicate by maximizing 1, = E';" lk where lk 
is defined in (5) . 
r 18, " a! 4 
15 Bias 
MSE 
20 Bias 
MSE 
25 Bias 
MSE 
30 Bias 
MSE 
- .012464 
.OOO 155 
-.010878 
.OOOO18 
.000074 
.000050 
-.008613 
-.007036 
.020852 
.0004 35 
.004578 
.000021 
-.028581 
BO08 17 
.012544 
.OOO 157 
- .001241 
.000001 
-.002542 
.000006 
- .OO 1379 
.000002 
-.001484 
.000002 
-.050126 
.002512 
-.025457 
.000648 
-.046221 
.002136 
.022037 
.0004 8 5 
Table 2. Bias and mean square error (MSE) of the conditional maximum likelihood 
estimators &, and 6, and moment estimators & and 4 based on 200 simulations 
with a! = .10 and 
Table 2 shows the bias and mean square error (MSE) for & and 6 and also the 
bias and MSE for 6, and 4,. Although the conditional MLE 4, has smaller bias 
than 4, the moment estimator & seems to generally have smaller bias than 6,. 
= -2.00. 
8 
5 .  PROOF OF THEOREM 2 
To prove Theorem 2, note that the elements of p and C are given by p; = 
lim(l/r) xk=l p;k, i = 1,2,3 and C i j  = lim(l/t) E;:', a i j k  as r tends to infinity, 
where the terms in these sums are the moments given in Theorem 1. Since p;k 
and a i j k  converge to finite limits as k tends to infinity, we have p; = 1imp;k and 
c;, = limaijk. Thus the calculations are similar to those discussed at the end of 
Section 2. The remainder of the proof requires showing (Serfling, 1980, p.30) that 
r + l  
r+l 
k= 1 
where 
hkr = E[UkI(Uk > € 2 4 1  
3 
a= 1 
and I(.) is the indicator function. Since hkr 5 (c2r)-'E(U;), the limit in (9) can 
be established by examining the fourth moments of the Q k , i  = 1,2,3. 
To obtain bounds for these moments, we replace zk by 
where Nk = ctzt M;k and X j k  takes the value X j k  = i if the j t h  event occurring 
in the interval (Tk-1,Tk) corresponds to the occurrence of the ith detected fault, 
i = 1,2,. . . , k - 1. Given that Nk = n, X l k ,  X 2 k , .  . . ,Xnk are i.i.d with truncated 
density g ( i  ; 4) /G(k  - 1 ; d), i = 1,2, . . . , k - 1. 
In terms of Zi ,  the vectors Vk defined in (6) can be written 
9 
where u ( i )  = a;. 
Since the distribution of Nk is conditionally, given Y k ,  Poisson with mean 
(XI - &)Yk  and since Y k  I s ,  it follows that Nk is stochastically smaller than the 
Poisson random variable N that has mean X l s  = as. Thus for any positive integer 
p we have E ( Y [ )  5 sp and E(Nkp)  5 E ( N p )  < 00. 
For any nonnegative quantities w1, wq, . . . , wn and positive integer p we have 
n n 
i= 1 i= 1 
By applying (12) to the form of V3k given in ( l l ) ,  we obtain 
j =  1 
where N has a Poisson distribution with mean as and X has the density g(i;q5).  
All positive moments of a(%) exist for the family of densities in (7). In summary 
we have E(VA) 5 Bi, where Bip, i = 1,2,3 do not depend on k. 
To complete the proof of Theorem 2, we again use (12) to obtain u k 2  5 9 cy=l 
(Vik - ~ i k ) ~ .  Since (Kk - &k)4 5 (Kk + p ; k ) 4 ,  the binomial expansion can be used 
to show that E(Uz)  5 B where B is finite and does not depend on k. Thus the 
limit in (9) is zero, which proves Theorem 2. 
6 .  FINAL REMARKS 
Software testing counters (Huang, 1977) will tend to over count the number of 
times a fault produces an error in the output. An alternative method, which will 
10 
accurately give the fault hitting frequencies, has been described in the literature on 
multiversion programming (Avizienis and chen, 1977). To describe this method in 
the context of error recapture experiments, let PI, P2,. . . denote successive versions 
of the original program PO, where Pi is the result of correcting the fault detected in 
Pi-1 at time Ti. A copy of Pi-1 is made before correcting this fault and all of the 
versions (Po, PI,. . . , Pi) are run on the same input series during the interval (Ti, 7 ) .  
To determine the hitting frequency of the fault detected at time Ti, the outputs of 
Pi-l are compared with those of Pi. Any difference in the outputs is due to the 
fault that resides in Pi-l which has been corrected in Pi. Similarly, comparing the 
outputs of all pairs (P;- l ,Pi) ,  i = 1,2, .  . . , r  will yield the total set of fault hitting 
frequencies. 
ACKNOWLEDGMENT 
This research was supported by the National Aeronautics and Space Adminis- 
tration, under contract NAG-1-835. 
11 
Model Estimator Variance 
Poisson, 
r 
a > 0,  -00 < 4 < 00, +(4) = e@ r r 
e+ 6 = l n ( x ( i  - 1)M;/ Mi) 
= a l  u'-le-"du/(r - l ) !  i= 1 a= 1 
Geometric, r 
i= 1 
2a( r s) - ' 
Logarithmic Series, 
r 
& = M;/T 
i=l  
Table 1. Moment estimators of (a,4) for the Poisson, Geometric and Logarithmic series rate 
models. 
12 
REFERENCES 
[l] Avizienis and Chen (1977. On the implementation of N-version programming for 
software fault tolerance during program execution. Proc. COMPSAC 77. 149-55. 
[2] Cox, D.R. and Lewis, P.A.W. (1966). The Statistical Analysis of Series of Events 
Methuen, London. 
[3] Gupta, R. & Lee, L. (1989), Estimating Fault Hitting rates by Recapture Sampling. 
Communication in Statistics, 18 (6) 2139-53. 
[4] Huang, J.C. (1977), Error detection through program testing. In Current Trends 
in Programming Methodology, 2, Ed. R.T. Yeh, pp. 1643. Prentice Hall, Engelwood 
Cliffs, N.J. 
[5] Miller (1986), Exponential order statistics models of software reliability growth. IEEE 
Trans. Software Engrg., SE-12, 12-24. 
[6] Jelinski, Z. and Moranda, P.M. (1972). Software reliability research. In Statistical 
Computer Performance Evaluation, Ed. W. Freiberger, pp. 465-84. Academic Press, 
New York. 
[7] Nagel, P.M., Scholz, F.W., and Skrivan, J.A. (1984). Software reliabilitv; additional 
investinations into modeling with replicated experiments. NASA Langley Research 
Center, Hampton, VA, NASA Contractor Report 172378, June. 
[8] Nayak, T.K. (1988). Estimating population size by recapture sampling. Biometrika, 
75, 113-120. 
[9] Sandland, R.L. and Cormack R.M. (1984), Statistical inference for Poisson and multi- 
nomial models for capture-recapture experiments. Biometrika 71, 27-33. 
[ 101 Serfling, R.J. (1980) .Approximation Theorems of Mathematical Statistics. John Wiley 
& Sons, New York. 
13 


Estimation in a discrete tail rate family of recapture sampling models

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server