Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods

The literature on Inverse Reinforcement Learning (IRL) typically assumes that humans take actions in order to minimize the expected value of a cost function, i.e., that humans are risk neutral. Yet, in practice, humans are often far from being risk neutral. To fill this gap, the objective of this paper is to devise a framework for risk-sensitive IRL in order to explicitly account for a human's risk sensitivity. To this end, we propose a flexible class of models based on coherent risk measures, which allow us to capture an entire spectrum of risk preferences from risk-neutral to worst-case. We propose efficient non-parametric algorithms based on linear programming and semi-parametric algorithms based on maximum likelihood for inferring a human's underlying risk measure and cost function for a rich class of static and dynamic decision-making settings. The resulting approach is demonstrated on a simulated driving game with ten human participants. Our method is able to infer and mimic a wide range of qualitatively different driving styles from highly risk-averse to risk-neutral in a data-efficient manner. Moreover, comparisons of the Risk-Sensitive (RS) IRL approach with a risk-neutral model show that the RS-IRL framework more accurately captures observed participant behavior both qualitatively and quantitatively, especially in scenarios where catastrophic outcomes such as collisions can occur.Comment: Submitted to International Journal of Robotics Research; Revision 1: (i) Clarified minor technical points; (ii) Revised proof for Theorem 3 to hold under weaker assumptions; (iii) Added additional figures and expanded discussions to improve readabilit

Decision-Making with Belief Functions: a Review

Approaches to decision-making under uncertainty in the belief function framework are reviewed. Most methods are shown to blend criteria for decision under ignorance with the maximum expected utility principle of Bayesian decision theory. A distinction is made between methods that construct a complete preference relation among acts, and those that allow incomparability of some acts due to lack of information. Methods developed in the imprecise probability framework are applicable in the Dempster-Shafer context and are also reviewed. Shafer's constructive decision theory, which substitutes the notion of goal for that of utility, is described and contrasted with other approaches. The paper ends by pointing out the need to carry out deeper investigation of fundamental issues related to decision-making with belief functions and to assess the descriptive, normative and prescriptive values of the different approaches

Making mathematics phenomenal : Based on an Inaugural Professorial Lecture delivered at the Institute of Education, University of London, on 14 March 2012

Mathematics is often portrayed as an 'abstract' cerebral subject, beyond the reach of many. In response, research with digital technology has led to innovative design in which mathematics can be experienced to some extent like everyday phenomena. I examine how careful design can 'phenomenalise' mathematics - that is to say create mathematical artefacts that can be directly experienced to support not only engagement but also focus on key ideas. I argue that mathematical knowledge gained through interaction with suitably designed tools can prioritise powerful reasons for doing mathematics, imbuing it with a sort of utility and offering learners hooks on which they can gradually develop fluency and connected understanding. Illustrative examples are taken from conventional topics such as number, algebra, geometry and statistics but also from novel situations where mathematical methods are juxtaposed with social values. The suggestion that prioritising utility supports a more natural way of learning mathematics emerges directly from constructionist pedagogy and inferentialist philosophy

A Conceptual Model of Investor Behavior

Based on a survey of behavioral finance literature, this paper presents a descriptive model of individual investor behavior in which investment decisions are seen as an iterative process of interactions between the investor and the investment environment. This investment process is influenced by a number of interdependent variables and driven by dual mental systems, the interplay of which contributes to boundedly rational behavior where investors use various heuristics and may exhibit behavioral biases. In the modeling tradition of cognitive science and intelligent systems, the investor is seen as a learning, adapting, and evolving entity that perceives the environment, processes information, acts upon it, and updates his or her internal states. This conceptual model can be used to build stylized representations of (classes of) individual investors, and further studied using the paradigm of agent-based artificial financial markets. By allowing us to implement individual investor behavior, to choose various market mechanisms, and to analyze the obtained asset prices, agent-based models can bridge the gap between the micro level of individual investor behavior and the macro level of aggregate market phenomena. It has been recognized, yet not fully explored, that these models could be used as a tool to generate or test various behavioral hypothesis.behavioral finance;financial decision making;agent-based artificial financial markets;cognitive modeling;investor behavior

Concepts for Decision Making under Severe Uncertainty with Partial Ordinal and Partial Cardinal Preferences

We introduce three different approaches for decision making under uncertainty if (I) there is only partial (both cardinally and ordinally scaled) information on an agent’s preferences and (II) the uncertainty about the states of nature is described by a credal set (or some other imprecise probabilistic model). Particularly, situation (I) is modeled by a pair of binary relations, one specifying the partial rank order of the alternatives and the other modeling partial information on the strength of preference. Our first approach relies on decision criteria constructing complete rankings of the available acts that are based on generalized expectation intervals. Subsequently, we introduce different concepts of global admissibility that construct partial orders between the available acts by comparing them all simultaneously. Finally, we define criteria induced by suitable binary relations on the set of acts and, therefore, can be understood as concepts of local admissibility. For certain criteria, we provide linear programming based algorithms for checking optimality/admissibility of acts. Additionally, the paper includes a discussion of a prototypical situation by means of a toy example