Engineering judgement in reliability and safety and its limits: what can we learn from research in psychology?

Engineering judgement has an important role in safety or reliability assessment. This paper focuses on the use of engineering judgement for integrating diverse evidence into an assessment of the safety or reliability of a product. In many cases of stringent safety requirements, this form of engineering (or "expert") judgement, i.e., "informal inference from complex evidence", is the crucial resource for the decision maker, for lack of more solid, objective evidence. This dependence on judgement is especially evident in the assessment of the unreliability due to possible design faults in complex products, and computer software in particular. Although engineering judgement plays an essential role in the assessment, there are good reasons to doubt the ability of experts in some of the judgement tasks in which they are usually employed. Experimental research both about the way humans think and integrate evidence, and about the performance of experts in tasks similar to engineering judgement, support the idea that the ability of experts may be overrated. This paper summarises some literature about common fallacies and ways to guard against them, and argues for a more disciplined use of expert judgement

Elicitation of structured engineering judgement to inform a focussed FMEA

The practical use of Failure Mode and Effects Analysis (FMEA) has been criticised because it is often implemented too late and in a manner that does not allow information to be fed-back to inform the product design. Lessons learnt from the use of elicitation methods to gather structured expert judgement about engineering concerns for a new product design has led to an enhancement of the approach for implementing design and process FMEA. We refer to this variant as a focussed FMEA since the goal is to enable relevant engineers to contribute to the analysis and to act upon the outcomes in such a way that all activities focus upon the design needs. The paper begins with a review of the proposed process to identify and quantify engineering concerns. The pros and cons of using elicitation methods, originally designed to support construction of a Bayesian prior, to inform a focussed FMEA are analysed and a comparison of the proposed process in relation to the existing standards is made. An industrial example is presented to illustrate customisation of the process and discuss the impact on the design process

The effect of radio frequency interference on the 136- to 138-MHz return link and 400.5- to 401.5-MHz forward link of the Tracking and Data Relay Satellite System

The purpose is to update the RFI estimates in the 136- to 138-MHz VHF band and to make estimates for the first time for the 400.5- to 401.5-MHz UHF band. These preliminary predictions are based on primarily ITU frequency-registration data, with missing data bridged by engineering judgement

Expert Elicitation for Reliable System Design

This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.Comment: This paper commented in: [arXiv:0708.0285], [arXiv:0708.0287], [arXiv:0708.0288]. Rejoinder in [arXiv:0708.0293]. Published at http://dx.doi.org/10.1214/088342306000000510 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Predicting with sparse data

It is well known that effective prediction of project cost related factors is an important aspect of software engineering. Unfortunately, despite extensive research over more than 30 years, this remains a significant problem for many practitioners. A major obstacle is the absence of reliable and systematic historic data, yet this is a sine qua non for almost all proposed methods: statistical, machine learning or calibration of existing models. In this paper we describe our sparse data method (SDM) based upon a pairwise comparison technique and Saaty's Analytic Hierarchy Process (AHP). Our minimum data requirement is a single known point. The technique is supported by a software tool known as DataSalvage. We show, for data from two companies, how our approach — based upon expert judgement — adds value to expert judgement by producing significantly more accurate and less biased results. A sensitivity analysis shows that our approach is robust to pairwise comparison errors. We then describe the results of a small usability trial with a practising project manager. From this empirical work we conclude that the technique is promising and may help overcome some of the present barriers to effective project prediction

PENERAPAN MODEL ASESMEN PORTOFOLIO PADA PERKULIAHAN PRAKTIK BATU BETON DI PRODI PENDIDIKAN TEKNIK BANGUNAN JPTS FPTK UPI

Portofolio asessment model had been used for concrete practice course which can evaluate study result not only at the end of the given time with focusing on cognitive aspect, a little of psychomotoric, and almost untouched affective aspect but the whole aspect instead. Which hoped can help the lecturer to judge as evaluate conceptual that can be viewed not to exceed or harm the studenst. This research using descriptive qualitative approximation, with indepth interview. Sample in this research is concrete practice course lecturers, its acquaintance and 2013 building engineering of education students. Research subject is five persons. And using purposive sampling as sample withdrawal. From result of the reseach can be assumed that in concrete practice course, portofolio asessment model can be used to see student development graphic and evaluate study result through 3 judgement aspect (cognitive, affective and psychomotoric) on process and product of practice using jobsheet. Judgement taken by criteria and itegrity of competence. However its still have obstacles which include individual judgement caused by practice implementation is as a team due to variety restriction, and product judgement is more precise than process judgement. Because of that lecturers intuition to judge on process and judgement details when filing the archive is very important so the judgement value will be objective

A case of conceptualisation: using a grounded theory approach to further explore how professionals define engineering judgement for use in engineering education

Students are expected to have developed their engineering judgement throughout the course of their studies as part of their accreditation requirements (as stipulated by the Accreditation Board of Engineering and Technology for example), and yet conceptually it is often ill-defined and therefore difficult to teach. This work was carried out in an attempt to better conceptualise engineering judgement for use in higher education. As such, semi-structured interviews were conducted with established members of academic staff who additionally had extensive industrial experience – who were asked to define engineering judgement and which aspects students ought to develop in their studies. A pragmatic grounded theory approach was used, based on the assumption that a theoretical idea/framework could be developed, enabling us to refer to previous literature and the emerging categories from our data set to help clarify engineering judgement. Several terms help define engineering judgement, including accumulated experience, fundamental theoretical knowledge, and imagination/intuition. Essential criteria for developing judgement includes students’ ability to identify and reduce complex problems, and embrace failure. A theoretical framework has been proposed accommodating a more enhanced definition and conceptualisation of engineering judgement which can be applied and adapted for use within engineering education for students’ ultimate benefit

PENGEMBANGAN MULTIMEDIA ANIMASI PENGUJIAN TARIK

Penelitian ini bertujuan untuk mengetahui pengembangan media animasi pada materi pengujian tarik (tensile test). Metode pengembangan yang digunakan adalah “ADDIE” (Analysis, Design, Development, Implementation, Evaluation). Peneliti mengambil subyek terhadap 20 orang mahasiswa Departemen Pendidikan Teknik Mesin yang telah mengontrak mata kuliah Material Teknik untuk mengetahui respon pengguna terhadap multimedia. Selain itu, dilakukan judgement ahli media dan judgement ahli materi oleh dosen yang ahli di bidangnya. Hasil penelitian untuk persentase nilai total respon pengguna adalah 87,6% masuk dalam kategori menarik untuk penggunaan multimedia animasi pengujian tarik pada proses pembelajaran. Hasil judgement ahli media didapatkan persentase kelayakan media sebesar 82% dimana untuk acuan rating scale, multimedia animasi pengujian tarik dikatakan layak untuk dipakai dalam pembelajaran. Hasil judgement ahli materi didapatkan persentase sebesar 80%, sehingga untuk materi yang terdapat di multimedia animasi pengujian tarik sudah relevan dan layak. Kategori tersebut diinterpretasikan terhadap hasil yang didapat dari judgement ahli media dan judgement ahli materi bahwa media animasi pengujian tarik layak dan menarik untuk digunakan dalam proses pembelajaran pada materi pengujian tarik. ;--This study aims to determine the development of animation media on the material tensile test. The development method used was “ADDIE” (Analysis, Design, Development, Implementation, Evaluation). The researcher take 20 students at Mechanical Engineering Department of Education as subject who has been take Engineering of Material lessons for know the responce of multimedia users. In addition, media expert judgement and material expert judgement are conducted by lecturers who are experts in their focus. The result shows that the multimedia animation of tensile test were well develope and attractive with 87,6% student responser, its included in the interesting category for using multimedia animation tensile testing in learning process. The results of media experts judgement obtained a percentage of the feasibility media by 82% where for the reference is rating scale, multimedia animation tensile testing is said to be suitable for use in learning. The material expert judgment results obtained a percentage of 80%, so for the material contained in the tensile animation animation testing is relevant and feasible. The category is interpreted towards the results obtained from the judgment of media experts and judgment of material experts that the animation media tensile testing is feasible and interesting to be used in the learning process in tensile testing material

An automated procedure for material parameter evaluation for viscoplastic constitutive models

An automated procedure is presented for evaluating the material parameters in Walker's exponential viscoplastic constitutive model for metals at elevated temperature. Both physical and numerical approximations are utilized to compute the constants for Inconel 718 at 1100 F. When intermediate results are carefully scrutinized and engineering judgement applied, parameters may be computed which yield stress output histories that are in agreement with experimental results. A qualitative assessment of the theta-plot method for predicting the limiting value of stress is also presented. The procedure may also be used as a basis to develop evaluation schemes for other viscoplastic constitutive theories of this type