Assessing Human Error Against a Benchmark of Perfection

An increasing number of domains are providing us with detailed trace data on human decisions in settings where we can evaluate the quality of these decisions via an algorithm. Motivated by this development, an emerging line of work has begun to consider whether we can characterize and predict the kinds of decisions where people are likely to make errors. To investigate what a general framework for human error prediction might look like, we focus on a model system with a rich history in the behavioral sciences: the decisions made by chess players as they select moves in a game. We carry out our analysis at a large scale, employing datasets with several million recorded games, and using chess tablebases to acquire a form of ground truth for a subset of chess positions that have been completely solved by computers but remain challenging even for the best players in the world. We organize our analysis around three categories of features that we argue are present in most settings where the analysis of human error is applicable: the skill of the decision-maker, the time available to make the decision, and the inherent difficulty of the decision. We identify rich structure in all three of these categories of features, and find strong evidence that in our domain, features describing the inherent difficulty of an instance are significantly more powerful than features based on skill or time.Comment: KDD 2016; 10 page

The decision-making entrepreneur; Literature review

This study provides a literature overview of the entrepreneurial decision-making process. The literature review is used as background information for a qualitative study, which investigates, by means of case studies, the decision-making process of small business enterpreneurs in The Netherlands (Gibcus and Van Hoesel, 2003). The literature overview is the starting point of a confrontation between the literature on decision-making and the empirical findings of the latter qualitive study. Firstly, this literature review gives an introduction to general decision theory. It discusses the classical rationality, the bounded rationality and the neoclassical rationality. The place of the entrepreneur in the general decision theory is also discussed. Next, an analytic framework of the strategic decision-making in SMEs is presented. The analytic framework consists of three elements: the entrepreneur, the environment and the strategic decision process. Each of these elements is critical. Finally, some earlier empirical findings on entrepreneurial strategic decision-making are discussed.

Pilot Your Life Decisively for Well-Being and Flourishing

I have been a pilot, aviation instructor and FAA Pilot Examiner for over 40 years. Aviation requires a “pilot in command” mindset consistent with the tenets of positive psychology. This paper explains and advocates for this daily empowered, adaptive decision making process used by pilots in aviation as a necessary life skill to eliminate mind wandering and disengagement and optimize human performance consistent with the goals of positive psychology. Exploring the concepts of “pilot-in-command” (decisive control and self-efficacy) and “situational awareness” (alert mental functioning) I will offer techniques and suggestions for developing and deploying these critical skills in everyday life. I will examine the heuristic-based, “fast and frugal” (time and data limited) decision-making used every day in aviation and apply this to life for optimal performance and flourishing for individual lives and organizational effectiveness

Constructive Criticism

Attempts to attain knowledge as certified true belief have failed to circumvent Hume’s injunction against induction. Theories must be viewed as unprovable, improbable, and undisprovable. The empirical basis is fallible, and yet the method of conjectures and refutations is untouched by Hume’s insights. The implications for statistical methodology is that the requisite severity of testing is achieved through the use of robust procedures, whose assumptions have not been shown to be substantially violated, to test predesignated range null hypotheses. Nonparametric range null hypothesis tests need to be developed to examine whether or not effect sizes or measures of association, as well as distributional assumptions underlying the tests themselves, meet satisficing criteria

Empirical Game Theoretic Models for Autonomous Driving: Methods and Applications

In recent years, there has been enormous public interest in autonomous vehicles (AV), with more than 80 billion dollars invested in self-driving car technology. However, for the foreseeable future, self-driving cars will interact with human driven vehicles and other human road users, such as pedestrians and cyclists. Therefore, in order to ensure safe operation of AVs, there is need for computational models of humans traffic behaviour that can be used for testing and verification of autonomous vehicles. Game theoretic models of human driving behaviour is a promising computational tool that can be used in many phases of AV development. However, traditional game theoretic models are typically built around the idea of rationality, i.e., selection of the most optimal action based on individual preferences. In reality, not only is it hard to infer diverse human preferences from observational data, but real-world traffic shows that humans rarely choose the most optimal action that a computational model suggests. The thesis makes a set of methodological contributions towards modelling sub-optimality in driving behaviour within a game theoretic framework. These include solution concepts that account for boundedly rational behaviour in hierarchical games, addressing challenges of bounded rationality in dynamic games, and estimation of multi-objective utility aggregation from observational data. Each of these contributions are evaluated based on a novel multi-agent traffic dataset. Building on the game theoretic models, the second part of the thesis demonstrates the application of the models by developing novel safety validation methodologies for testing AV planners. The first application is an automated generation of interpretable variations of lane change behaviour based on Quantal Best Response model. The proposed model is shown to be effective for generating both rare-event situations and to replicate the typical behaviour distribution observed in naturalistic data. The second application is safety validation of strategic planners in situations of dynamic occlusion. Using the concept of hypergames, in which different agents have different views of the game, the thesis develops a new safety surrogate metric, dynamic occlusion risk (DOR), that can be used to evaluate the risk associated with each action in situations of dynamic occlusion. The thesis concludes with a taxonomy of strategic interactions that maps complex design specific strategies in a game to a simpler taxonomy of traffic interactions. Regulations around what strategies an AV should execute in traffic can be developed over the simpler taxonomy, and a process of automated mapping can protect the proprietary design decisions of an AV manufacturer