Empirical bayes estimates of development reliability for one shot devices

This article describes a method for estimating the reliability of a system under development that is an evolution of previous designs. We present an approach to making effective use of heritage data from similar operational systems to estimate reliability of a design that is yet to realise any data. The approach also has a mechanism to adjust initial estimates in the light of sparse data that becomes available in early stages of test. While the estimation approach, known as empirical Bayes is generic, we focus on one shot devices as this was the type of system which provided the practical motivation for this work and for which we illustrate an application

Modelling and managing reliability growth during the engineering design process

[This is a keynote speech presented at the 2nd International Conference on Design Engineering and Science, discussing modelling and managing reliability growth during the engineering process.] Reliability is vital for safe and efficient operation of systems. Decisions about the configuration and selection of parts within a system, and the development activities to prove the chosen design, will influence the inherent reliability. Modelling provides a mechanism for explicating the relationship between the engineering activities and the statistical measures of reliability so that useful estimates of reliability can be obtained. Reliability modelling should be aligned to support the decisions taken during design and development. We examine why and how a reliability growth model can be structured, the type of data required and available to populate them, the selection of relevant summary measures, the process for updating estimates and feeding back into design to support planning decisions. The modelling process described is informed by our theoretical background in management science and our practical experience of working with UK industry

UK Foot and Mouth disease: a systemic risk assessment of existing controls

This article details a systemic analysis of the controls in place and possible interventions available to further reduce the risk of a foot and mouth disease (FMD) outbreak in the United Kingdom. Using a research-based network analysis tool, we identify vulnerabilities within the multibarrier control system and their corresponding critical control points (CCPs). CCPs represent opportunities for active intervention that produce the greatest improvement to United Kingdom's resilience to future FMD outbreaks. Using an adapted ‘features, events, and processes’ (FEPs) methodology and network analysis, our results suggest that movements of animals and goods associated with legal activities significantly influence the system's behavior due to their higher frequency and ability to combine and create scenarios of exposure similar in origin to the U.K. FMD outbreaks of 1967/8 and 2001. The systemic risk assessment highlights areas outside of disease control that are relevant to disease spread. Further, it proves to be a powerful tool for demonstrating the need for implementing disease controls that have not previously been part of the system

Expert Elicitation for Reliable System Design

This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.Comment: This paper commented in: [arXiv:0708.0285], [arXiv:0708.0287], [arXiv:0708.0288]. Rejoinder in [arXiv:0708.0293]. Published at http://dx.doi.org/10.1214/088342306000000510 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Incorporation of genuine prior information in cost-effectiveness analysis of clinical trial data

The Bayesian approach to statistics has been growing rapidly in popularity as an alternative to the frequentist approach in the appraisal of heathcare technologies in clinical trials. Bayesian methods have significant advantages over classical frequentist statistical methods and the presentation of evidence to decision makers. A fundamental feature of a Bayesian analysis is the use of prior information as well as the clinical trial data in the final analysis. However, the incorporation of prior information remains a controversial subject that provides a potential barrier to the acceptance of practical uses of Bayesian methods. The pur pose of this paper is to stimulate a debate on the use of prior information in evidence submitted to decision makers. We discuss the advantages of incorporating genuine prior information in cost-effectiveness analyses of clinical trial data and explore mechanisms to safeguard scientific rigor in the use of such prior information

Stated versus inferred beliefs: A methodological inquiry and experimental test

If asking subjects their beliefs during repeated game play changes the way those subjects play, using those stated beliefs to evaluate and compare theories of strategic behavior is problematic. We experimentally verify that belief elicitation can alter paths of play in a repeated asymmetric matching pennies game. In this setting, belief elicitation improves the goodness of fit of structural models of belief learning, and the prior beliefs implied by such structural models are both stronger and more realistic when beliefs are elicited than when they are not. These effects are, however, confined to the player type who sees a strong asymmetry between payoff possibilities for her two strategies in the game. We also find that “inferred beliefs” (beliefs estimated from past observed actions of opponents) can be better predictors of observed actions than the “stated beliefs” resulting from belief elicitation.beliefs; stated beliefs; belief elicitation; inferred beliefs; estimated beliefs; belief updating; repeated games; experimental methods