One of the major obstacles in identifying and interpreting a trend is the small number of data points. Future trending reports will begin with 1983 data. As the problem/failure data are aggregated by year, there are just seven observations (1983 to 1989) for the 1990 reports. Any statistical inferences with a small amount of data will have a large degree of uncertainty. Consequently, a regression technique approach to identify a trend is limited. Though trend determination by failure mode may be unrealistic, the data may be explored for consistency or stability and the failure rate investigated. Various alternative data analysis procedures are briefly discussed. Techniques that could be used to explore problem/failure data by failure mode are addressed. The data used are taken from Section One, Space Shuttle Main Engine, of the Calspan Quarterly Report dated April 2, 1990

Church, Curtis K.

English

NASA Technical Reports Server

N91-18975
1990
NASA/ASEE SUMMER FACULTY FELLOWSHIP PROGRAM
MARSHALL SPACE FLIGHT CENTER
THE UNIVERSITY OF ALABAMA
IS IT POSSIBLE TO IDENTIFY A TREND IN PROBLEM/FAILURE DATA ?
Prepared By:
Academic Rank:
Institution:
Curtis K. Church, Ph.D.
Associate Professor
Middle Tennessee State
University, Department of
Mathematics and Statistics
NASA/MSFC:
Office:
Division:
Branch:
MSFC Colleague:
Contract No.:
System Safety & Reliability
Reliability &
Maintainability Engineering
Problem Assessment
Raymond Dodd, Ph.D.
Frank Pizzano
NGT-01-002-099
The University of Alabama
VIII
https://ntrs.nasa.gov/search.jsp?R=19910009662 2020-03-19T18:42:48+00:00Z

In many scientific endeavors, researchers want to determine
whether a sequence of observations taken over time exhibits some
type of trend. The NASA Standard, "Trend Analysis Techniques"
(NASA-STD-8070.5), describes a variety of statistical methods that
could be applied to time series data. Generally, trend is regarded
as a smooth broad motion of the system over a "long" term. Several
techniques are currently being used. The result of these efforts is
reported in "Quarterly Problem Trending Report for MSFC Shuttle
Elements and Payloads" prepared by the Calspan Corporation.
However, the nature of the problem/failure data poses difficulty in
identifying a trend.
One of the major obstacles in identifying and interpreting a
trend is the small number of data points. Future trending reports
will begin with 1983 data. As the problem/failure data is
aggregated by year, there are just seven observations (1983-1989)
for the 1990 reports. Any statistical inferences with a small
amount of data will have a large degree of uncertainty.
Consequently, a regression technique approach to identify a trend
is limited. Though trend determination by failure mode may be
unrealistic, the data may be explored for consistency or stability
and the failure rate investigated. In what follows, various
alternative data analysis procedures are briefly discussed.
Techniques that could be used to explore problem/ failure data by
failure mode. The data used is taken from Section One, Space
Shuttle Main Engine, of the Calspan Quarterly Report dated April 2,
1990.
There were four set of observations in the Quarterly Report
SSME Section that had a statistically significant downward trend
based on a regression analysis. There were a total of 36 data sets
trended in the SSME Section. These significant trends were based on
data from 1979 through 1989. Reconsidering these sets of data from
1983 on, not one of them has a significant regression fit at the
.01 level, which is the level used in the Quarterly Report. If we
begin with data in 1983, a significant downward trend at the .01
level of significance requires a Pearson product moment correlation
coefficient of r= -.875, that is r' = .765. In addition to the
Pearson correlation coefficient with the corresponding test of
hypothesis based on the normal distribution, there are numerous
nonparametric measures of association. Two widely used ones in
connection with regression and trending are Spearman's rank
correlation coefficient and Kendall's tau. These two procedures use
the rank order of the observations rather than the actual observed
value. The four sets of data that were previously fit with a
regression model are summarized below with the data beginning in
1983.
VIII-I
Correlations and Observed Siunific_nce _evels
Pearson Spearman Kendall
Fuel preburner injector
erosion/wear
dents/etc
contamination
Controller hardware
unexplained anomalies
-.618(linear) -.607 -.429
p=.139 p=.137 p=.177
-.765(power) -.857 -.714
p=.047 p=.036 p=.024
-.675(linear) -.685 -.488
p=.096 p=.094 p=.111
-.805(linear) -.786 -.714
p=.029 p=.054 p=.024
Another approach to exploring the data is from the
perspective of consistency. That is, does the failure rate
fluctuate from year to year or is it relatively stable? While this
approach is not to identify a trend it may provide the
experimenter with insight into the failure process. Assuming a
Poisson model for the number of failures, a chi-square goodness of
fit test, assuming a constant failure rate from 1983-1989, a
Kolmogorov-Smirnov goodness of fit test with the same assumption,
or a level of performance chart could be applied.
The chi-square goodness of fit test utilizes an assumed
probability model and estimates the expected number of failures
per year and then compares the observed number of failures with
these expected frequencies. To specify the procedure, let Xi be
the number of failures in year i and let t i be the number of test
seconds (or test starts) in year i. The expected number of
failures, under the assumption of a constant failure rate, for
year i is given by _i = 8ti, where e is the probability of a
failure in a sm_11 interval. The constant failure rate estimate of
8 is given by 0 = Z Xi/Z ti, where the summation ranges through
the years 1983 to 1989. The chi-square test statistic is given by
T= Z(Xi-_i)'/_ i which is approximately distributed as a chi-square
random variable with 5 degrees of freedom. The table below displays
observed and expected frequencies for the 3 fuel preburner injector
failure modes plus 2 other randomly selected data sets.
The .01 critical value for a chi-square distribution with 5
degrees of freedom is 15.1. The value of the test statistic T
gives some idea of the agreement between the observed results and
the assumption of a constant failure rate. The value of T must be
cautiously viewed in cases of small expected frequencies. Consult
the reference by Lawal and Upton for this consideration.
VIII-2
V-
vQbserved and Expected Freuuencies
Fuel preburner injector
eroslon/wear
83 84 85 86
3 4 5 1
2.73 2.52 3.83 1.21
dents/etc 13
3.72
3 7 1
3.44 5.23 1.65
contamination 3 5 4 0
1.86 1.72 2.61 .82
LPOTP contamination 6
6.45
10 2 3
5.96 9.06 2.85
HPOTPturbine nozzle
second stage crack
87 88 89
2 4 3
3.95 4.30 3.5
T=2.33
1 3 2
5.39 5.86 4.7
T=30.60
1 0 2
2.70 2.93 2.4
T=I2.57
15 13 3
9.35 10.2 8.2
T=15.76
1 1 1 1 0 7 2
1.61 1.49 2.27 .71 2.34 2.54 2.0
T=II.39
Another technique to explore the stability of the failure
rate is a level of performance chart. It is constructed and used
as a control chart. Using the Poisson probability model, the
oyerall_rate estimate _ is used to compute limits of
+ 3_[e /Z t i . The yearly failure rate estimates e i are th_n
compared, often graphically, with these limits. Values of e i
outside the limits point to extreme fluctuation of the failure
rates. The table below gives the yearly estimates of e with limits
based on the pooled estimate for 3 of the failure modes previously
considered.
Estimates of 8 by Year ( x 10 -3 )
Fuel preburner injector
erosion/wear
83 84 8_ 8_ 87 88 89 Limits
.10 .15 .12 .08 .05 .09 .08 (.03,.16)
dents/etc .45 .11 .17 .08 .02 .07 .05 (.06,.20)
HPOTP turbine nozzle
second stage crack .03 .04 .02 .08 0 .15 .05 (.01, .10)
An alternative process control technique that would monitor
the yearly failure rate relative to a specified target failure
rate is a cumulative sum (CUSUM) procedure. The CUSUM procedure is
a sequence of Wald sequential probability ratio tests used to
detect a change in the distribution of the number of problems. As
before, a Poisson distribution for the number of problems is used.
The CUSUM procedure is often enhanced by a fast initial response
(FIR) feature. The reference by Lucas discusses a Poisson CUSUM
procedure.
A cumulative sum procedure cumulates the difference between
VIII-3 i
an observed value Yi, some normalized value of Xi, and reference
value k. If this cumulation equals or exceeds the decision value
h, then conclude the failure rate for that year is greater than
the target rate. To detect an increase in counts, the CUSUM
statistic is Si= max(0, Yi-k+Si_l). The FIR CUSUM typically uses a
starting value S O = h/2. The CUSUM is restarted after indicating an
out of target value.
CUSUM procedures are evaluated by calculating their average
run length (ARL). The ARL should be large when the failure rate is
at the target level and short when the rate is at an undesireable
level.
The parameter k is the reference value for the CUSUM. Its
value will be chosen to be between the acceptable failure rate
(_a) and an unacceptable level (_d) that is to be detected.
Although the desired level of _a is zero, it is usually not used
since any occurence of a failure will then glve a signal. The
reference value for the Poisson CUSUM should be selected to be
close to k= (_ -_a)/(in _d -In _a )-
After k zs selected, the decision value h is chosen using a
table look-up procedure. There are tables given in the article by
Lucas. The value of h should give an appropriately large ARL when
the failure rate is on target and an appropriately small ARL value
when the rate is too high.
Use of a CUSUM takes more involvement from the analyst than
do the goodness of fit test or the level of performance chart. The
CUSUM, however, combines looking at the data for stability and
checking agreement with a target value. A CUSUM procedure with an
acceptable failure rate of 1 per 50,000 seconds and an
unacceptable rate of 1 per 20,000 seconds has been applied to the
problem/failure data. When a CUSUM value exceeds h then it is
restarted with next data value. The results are comparable to
those from the goodness of fit test and rate performance chart.
Along with the chi-square goodness of fit test and the level of
performance chart, even though simple, these techniques offer some
insight to supplement the regression approach.
Trend fitting and trend estimation are very far, particularly
with small samples, from being a purely mechanial process. There is
great scope, even necessity, for personal judgement. Exploring the
data for patterns can be a very difficult, delicate issue.
REFERENCES
1. Lawal, H. B., and Upton, G. J. G., "An approximation to the
distribution of the chi-square goodness of fit statistic for
use with small expectations," _m_tJL_, 67 (1980) 447-453
2. Lucas, J. M., "Counted Data CUSUM's," Technometrics, 27 (1985)
129-144
3. Quarterly Problem Trending Report for MSFC Shuttle Elements
and Payloads, Special Study c43a-3, Calspan Corporation, April
2, 1990
4. Trend Analysis Techniques, NASA Standard 8070.5, October 1988
VIII-4


Is it possible to identify a trend in problem/failure data

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server