Applying Bayes linear methods to support reliability procurement decisions

Bayesian methods are common in reliability and risk assessment, however, such methods often demand a large amount of specification and can be computationally intensive. Because of this, many practitioners are unable to take advantage of many of the benefits found in a Bayesian-based approach. The Bayes linear methodology is similar in spirit to a Bayesian approach but offers an alternative method of making inferences. Bayes linear methods are based on the use of expected values rather than probabilities, and updating is carried out by linear adjustment rather than by Bayes Theorem. The foundations of the method are very strong, based as they are in work of De Finetti and developed further by Goldstein. A Bayes linear model requires less specification than a corresponding probability model and for a given amount of model building effort, one can model a more complex situation quicker. The Bayes linear methodology has the potential to allow us to build ''broad-brush' models that enable us, for example, to explore different test setups or analysis methods and assess the benefits that they can give. The output a Bayes linear model is viewed as an approximation to 'traditional' probabilistic models. The methodology has been applied to support reliability decision making within a current United Kingdom Ministry of Defence (MOD) procurement project. The reliability decision maker had to assess different contractor bids and assess the reliability merit of each bid. Currently the MOD assess reliability programmes subjectively using expert knowledge - for a number of reasons, a quantitative method of assessment in some projects is desirable. The Bayes linear methodology was used to support the decision maker in quantifying his assessment of the reliability of each contractor's bid and determining the effectiveness of each contractor's reliability programme. From this, the decision maker was able to communicate to the project leader and contractors, why a specific contractor was chosen. The methodology has been used in other MOD projects and is considered by those within the MOD as a useful tool to support decision making. The paper will contain the following. The paper will introduce the Bayes linear methodology and briefly discuss some of the philosophical implications of adopting a Bayes linear methodology within the context of a reliability programme analysis. The paper will briefly introduce the reliability domain and the reasons why it is believed that the Bayes linear methodology can offer support to decision makers. An in-depth analysis of the problem will then be given documenting the steps taken in the project and how future decision makers can apply the methodology. A brief summary will then be given as to possible future work for those interested in the Bayes linear methodology

Evaluation of elicitation methods to quantify Bayes linear models

The Bayes linear methodology allows decision makers to express their subjective beliefs and adjust these beliefs as observations are made. It is similar in spirit to probabilistic Bayesian approaches, but differs as it uses expectation as its primitive. While substantial work has been carried out in Bayes linear analysis, both in terms of theory development and application, there is little published material on the elicitation of structured expert judgement to quantify models. This paper investigates different methods that could be used by analysts when creating an elicitation process. The theoretical underpinnings of the elicitation methods developed are explored and an evaluation of their use is presented. This work was motivated by, and is a precursor to, an industrial application of Bayes linear modelling of the reliability of defence systems. An illustrative example demonstrates how the methods can be used in practice

Confidence intervals for reliability growth models with small sample sizes

Fully Bayesian approaches to analysis can be overly ambitious where there exist realistic limitations on the ability of experts to provide prior distributions for all relevant parameters. This research was motivated by situations where expert judgement exists to support the development of prior distributions describing the number of faults potentially inherent within a design but could not support useful descriptions of the rate at which they would be detected during a reliability-growth test. This paper develops inference properties for a reliability-growth model. The approach assumes a prior distribution for the ultimate number of faults that would be exposed if testing were to continue ad infinitum, but estimates the parameters of the intensity function empirically. A fixed-point iteration procedure to obtain the maximum likelihood estimate is investigated for bias and conditions of existence. The main purpose of this model is to support inference in situations where failure data are few. A procedure for providing statistical confidence intervals is investigated and shown to be suitable for small sample sizes. An application of these techniques is illustrated by an example

Empirical bayes estimates of development reliability for one shot devices

This article describes a method for estimating the reliability of a system under development that is an evolution of previous designs. We present an approach to making effective use of heritage data from similar operational systems to estimate reliability of a design that is yet to realise any data. The approach also has a mechanism to adjust initial estimates in the light of sparse data that becomes available in early stages of test. While the estimation approach, known as empirical Bayes is generic, we focus on one shot devices as this was the type of system which provided the practical motivation for this work and for which we illustrate an application

Expert Elicitation for Reliable System Design

This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.Comment: This paper commented in: [arXiv:0708.0285], [arXiv:0708.0287], [arXiv:0708.0288]. Rejoinder in [arXiv:0708.0293]. Published at http://dx.doi.org/10.1214/088342306000000510 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Incorporating collateral information using an adaptive management framework for the regulation of transgenic crops:

"A lack of data often makes biological management decisions difficult and has been an area of contention in the debate over the approval of transgenic crops. Our knowledge of agricultural and natural systems is limited and our ability to gain additional information, quickly and effectively, is often handicapped by statistical complexity. To adequately cope with this requires new approaches and models that integrate decision-making and management. This paper describes one possible approach to the integration of decision-making and management, which may have application for the regulatory approval of transgenic crops. In many situations countries wishing to approve transgenic crops will have limited data on the environmental performance of the crop. The approach outlined in this paper looks at how related information, possibly collected from other countries, might be used to help inform decisions about the approval of transgenic crops. This is done within an integrated decision-making and management framework." Authors' AbstractTransgenic plants, Collateral data, Bayesian theory, Inference,

Multivariate reliability modelling with empirical Bayes inference

Recent developments in technology permit detailed descriptions of system performance to be collected and stored. Consequently, more data are available about the occurrence, or non-occurrence, of events across a range of classes through time. Typically this implies that reliability analysis has more information about the exposure history of a system within different classes of events. For highly reliable systems, there may be relatively few failure events. Thus there is a need to develop statistical inference to support reliability estimation when there is a low ratio of failures relative to event classes. In this paper we show how Empirical Bayes methods can be used to estimate a multivariate reliability function for a system by modelling the vector of times to realise each failure root cause

Optimal discrete stopping times for reliability growth tests

Often, the duration of a reliability growth development test is specified in advance and the decision to terminate or continue testing is conducted at discrete time intervals. These features are normally not captured by reliability growth models. This paper adapts a standard reliability growth model to determine the optimal time for which to plan to terminate testing. The underlying stochastic process is developed from an Order Statistic argument with Bayesian inference used to estimate the number of faults within the design and classical inference procedures used to assess the rate of fault detection. Inference procedures within this framework are explored where it is shown the Maximum Likelihood Estimators possess a small bias and converges to the Minimum Variance Unbiased Estimator after few tests for designs with moderate number of faults. It is shown that the Likelihood function can be bimodal when there is conflict between the observed rate of fault detection and the prior distribution describing the number of faults in the design. An illustrative example is provided