Assimilating Non-Probabilistic Assessments of the Estimation of Uncertainty Bias in Expert Judgment Elicitation Using an Evidence Based Approach in High Consequence Conceptual Designs
One of the major challenges in conceptual designs of complex systems is the identification of uncertainty embedded in the information due to lack of historic data. This becomes of increased concern especially in high-risk industries. This document reports a developed methodology that allows for the cognitive bias, estimation of uncertainty, to be elucidated to improve the quality of elicited data. It consists of a comprehensive literature review that begins by defining a \u27High Consequence Conceptual Engineering Environment\u27 and identifies the high-risk industries in which these environments are found. It proceeds with a discussion that differentiates risk and uncertainty in decision-making in these environments. An argument was built around the identified epistemic category of uncertainty, the impact on hard data for decision-making, and from whom we obtain this data.
The review shifts to defining and selecting the experts, the elicitation process in terms of the components, the process phases and steps involved, and an examination of a probabilistic and a fuzzy example. This sets the stage for this methodology that uses evidence theory for the mathematical analysis after the data is elicited using a tailored elicitation process. Yager\u27s combination rule is used to combine evidence and fully recognize the ignorance without ignoring available information.
Engineering and management teams from NASA Langley Research Center were the population from which the experts for this study were identified. NASA officials were interested in obtaining uncertainty estimates, and a comparison of these estimates, associated with their Crew Launch Vehicle (CLV) designs; the existing Exploration Systems Architecture Study Crew Launch Vehicle (ESAS CLV) and the Parallel-Staged Crew Launch Vehicle (P-S CLV) which is currently being worked.
This evidence-based approach identified that the estimation of cost parameters uncertainty is not specifically over or underestimated in High Consequence Conceptual Engineering Environments; rather, there is more uncertainty present than what is being anticipated. From the perspective of maturing designs, it was concluded that the range of cost parameters\u27 uncertainty at different error-state-values were interchangeably larger or smaller when compared to each other even as the design matures
