Graph theory approach to quantify uncertainty of performance measures

In this work, the performance measurement process is studied to quantify the uncertainty induced in the resulting performance measure (PM). To that end, the causes of uncertainty are identified, analysing the activities undertaken in the three following stages of the performance measurement process: design and implementation, data collection and record, and determination and analysis. A quantitative methodology based on graph theory and on the sources of uncertainty of the performance measurement process is used to calculate an uncertainty index to evaluate the level of uncertainty of a given PM or (key) performance indicator. An application example is presented. The quantification of PM uncertainty could contribute to better represent the risk associated with a given decision and also to improve the PM to increase its precision and reliability.FCT – Fundação para a Ciência e Tecnologia within the Project Scope: PEst- OE/EEI/UI0319/2014

A Methodology for the Range Ordering of Alternatives using a Bayesian Decision Model with Applications to the Space Program

The primary objective of this paper is to provide a reasonably general and essentially unified approach to those problems involving value judgments and subjective decision making, without regard to excessive rigor. The principle areas and methods of attack developed are: (1) The selection of a value measure which emphasizes the fact that the criterion of optimum performance is quite arbitrary, its merits reflecting only the constraints on the problem and the objectives sought, (2) The utilization of statistical decision theory as a basis for the solution and subsequent ev aluation of a class of problems in which a priori value judgments must be assigned by an individual or committee under uncertainty, and (3) The application of the methodology to those areas in which the relative uncertainty level of a decision need be assessed in terms of a cost or penalty incurred in reaching the conclusion. A parti cularly important application is the selection of alternatives (ioe., projects by corporate executives) and the subsequent sensitivity analysis of the decision

A Methodology for the Range Ordering of Alternatives using a Bayesian Decision Model with Applications to the Space Program

The primary objective of this paper is to provide a reasonably general and essentially unified approach to those problems involving value judgments and subjective decision making, without regard to excessive rigor. The principle areas and methods of attack developed are: (1) The selection of a value measure which emphasizes the fact that the criterion of optimum performance is quite arbitrary, its merits reflecting only the constraints on the problem and the objectives sought, (2) The utilization of statistical decision theory as a basis for the solution and subsequent ev aluation of a class of problems in which a priori value judgments must be assigned by an individual or committee under uncertainty, and (3) The application of the methodology to those areas in which the relative uncertainty level of a decision need be assessed in terms of a cost or penalty incurred in reaching the conclusion. A parti cularly important application is the selection of alternatives (ioe., projects by corporate executives) and the subsequent sensitivity analysis of the decision

Evaluating high risks in large-scale projects using an extended VIKOR method under a fuzzy environment

The complexity of large-scale projects has led to numerous risks in their life cycle. This paper presents a new risk evaluation approach in order to rank the high risks in large-scale projects and improve the performance of these projects. It is based on the fuzzy set theory that is an effective tool to handle uncertainty. It is also based on an extended VIKOR method that is one of the well-known multiple criteria decision-making (MCDM) methods. The proposed decision-making approach integrates knowledge and experience acquired from professional experts, since they perform the risk identification and also the subjective judgments of the performance rating for high risks in terms of conflicting criteria, including probability, impact, quickness of reaction toward risk, event measure quantity and event capability criteria. The most notable difference of the proposed VIKOR method with its traditional version is just the use of fuzzy decision-matrix data to calculate the ranking index without the need to ask the experts. Finally, the proposed approach is illustrated with a real-case study in an Iranian power plant project, and the associated results are compared with two well-known decision-making methods under a fuzzy environment

Quantitative Measures of Regret and Trust in Human-Robot Collaboration Systems

Human-robot collaboration (HRC) systems integrate the strengths of both humans and robots to improve the joint system performance. In this thesis, we focus on social human-robot interaction (sHRI) factors and in particular regret and trust. Humans experience regret during decision-making under uncertainty when they feel that a better result could be obtained if chosen differently. A framework to quantitatively measure regret is proposed in this thesis. We embed quantitative regret analysis into Bayesian sequential decision-making (BSD) algorithms for HRC shared vision tasks in both domain search and assembly tasks. The BSD method has been used for robot decision-making tasks, which however is proved to be very different from human decision-making patterns. Instead, regret theory qualitatively models human\u27s rational decision-making behaviors under uncertainty. Moreover, it has been shown that joint performance of a team will improve if all members share the same decision-making logic. Trust plays a critical role in determining the level of a human\u27s acceptance and hence utilization of a robot. A dynamic network based trust model combing the time series trust model is first implemented in a multi-robot motion planning task with a human-in-the-loop. However, in this model, the trust estimates for each robot is independent, which fails to model the correlative trust in multi-robot collaboration. To address this issue, the above model is extended to interdependent multi-robot Dynamic Bayesian Networks

Monopoly Pricing in a Vertical Market with Demand Uncertainty

We study a vertical market with an upsteam supplier and multiple downstream retailers. Demand uncertainty falls to the supplier who acts first and sets a uniform wholesale price before the retailers observe the realized demand and engage in retail competition. Our focus is on the supplier's optimal pricing decision. We express the price elasticity of expected demand in terms of the mean residual demand (MRD) function of the demand distribution. This allows for a closed form characterization of the points of unitary elasticity that maximize the supplier's profits and the derivation of a mild unimodality condition for the supplier's objective function that generalizes the widely used increasing generalized failure rate (IGFR) condition. A direct implication is that optimal prices between different markets can be ordered if the markets can be stochastically ordered according to their MRD functions or equivalently to their elasticities. Based on this, we apply the theory of stochastic orders to study the response of the supplier's optimal price to various features of the demand distribution. Our findings challenge previously established economic insights about the effects of market size, demand transformations and demand variability on wholesale prices and indicate that the conclusions largely depend on the exact notion that will be employed. We then turn to measure market performance and derive a distribution free and tight bound on the probability of no trade between the supplier and the retailers. If trade takes place, our findings indicate that ovarall performance depends on the interplay between demand uncertainty and level of retail competition

Estimating Risks in Emerging Soil Remediation Technologies

The Department of Energy is focusing a long-term development effort on producing cheaper, safer, and faster state-of-the-art soil remediation technologies. To assist with the management of these innovative technology development projects, ways of quantifiably measuring technical risk were investigated through a detailed literature review. \u27Technical risk\u27 was defined in this study as the combination of the consequences of undesired events and their likelihood. Careful design of the inputs into a technology selection decision support system accounted for the uncertainty in forecasting final characteristics of remediation technologies still in the early phases of R&D. Experts made subjective probability estimates of these cost, schedule, and performance factors. Examination of several measures of final cost and schedule risk focused on communicating the risks inherent in different technological alternatives to the technology manager for operational, not theoretical, use. These risk measures included subjective measures, using utility theory, and objective measures, using variation about an expected value. A new measure was developed, the expected unfavorable deviation, which is similar but superior to the semi-variance as a measure of downside risk. These simple risk measures can be used whenever uncertainty is expressed through probability distributions of cost, schedule, and performance characteristics

Generalizing Bayesian Optimization with Decision-theoretic Entropies

Bayesian optimization (BO) is a popular method for efficiently inferring optima of an expensive black-box function via a sequence of queries. Existing information-theoretic BO procedures aim to make queries that most reduce the uncertainty about optima, where the uncertainty is captured by Shannon entropy. However, an optimal measure of uncertainty would, ideally, factor in how we intend to use the inferred quantity in some downstream procedure. In this paper, we instead consider a generalization of Shannon entropy from work in statistical decision theory (DeGroot 1962, Rao 1984), which contains a broad class of uncertainty measures parameterized by a problem-specific loss function corresponding to a downstream task. We first show that special cases of this entropy lead to popular acquisition functions used in BO procedures such as knowledge gradient, expected improvement, and entropy search. We then show how alternative choices for the loss yield a flexible family of acquisition functions that can be customized for use in novel optimization settings. Additionally, we develop gradient-based methods to efficiently optimize our proposed family of acquisition functions, and demonstrate strong empirical performance on a diverse set of sequential decision making tasks, including variants of top-$k$ optimization, multi-level set estimation, and sequence search.Comment: Appears in Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS 2022