Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods

The literature on Inverse Reinforcement Learning (IRL) typically assumes that humans take actions in order to minimize the expected value of a cost function, i.e., that humans are risk neutral. Yet, in practice, humans are often far from being risk neutral. To fill this gap, the objective of this paper is to devise a framework for risk-sensitive IRL in order to explicitly account for a human's risk sensitivity. To this end, we propose a flexible class of models based on coherent risk measures, which allow us to capture an entire spectrum of risk preferences from risk-neutral to worst-case. We propose efficient non-parametric algorithms based on linear programming and semi-parametric algorithms based on maximum likelihood for inferring a human's underlying risk measure and cost function for a rich class of static and dynamic decision-making settings. The resulting approach is demonstrated on a simulated driving game with ten human participants. Our method is able to infer and mimic a wide range of qualitatively different driving styles from highly risk-averse to risk-neutral in a data-efficient manner. Moreover, comparisons of the Risk-Sensitive (RS) IRL approach with a risk-neutral model show that the RS-IRL framework more accurately captures observed participant behavior both qualitatively and quantitatively, especially in scenarios where catastrophic outcomes such as collisions can occur.Comment: Submitted to International Journal of Robotics Research; Revision 1: (i) Clarified minor technical points; (ii) Revised proof for Theorem 3 to hold under weaker assumptions; (iii) Added additional figures and expanded discussions to improve readabilit

Relational Contracts and Inequity Aversion

We study the effects of envy on the feasibility of relational contracts in a standard moral hazard setup with two agents. Performance is evaluated via an observable, but non-contractible signal which reflects the agent´s individual contribution to firm value. Both agents exhibit disadvantageous inequity aversion. In contrast to the literature, we find that inequity aversion may be beneficial: In the presence of envy, for a certain range of interest rates relational contracts may be more profitable. Furthermore, for some interest rates reputational equilibria exist only with inequity averse agents.Principal-Agent, Relational Contract, Inequity Aversion, Envy

On behavioral Arrow Pratt risk process with applications to risk pricing, stochastic cash flows, and risk control

We introduce a closed form behavioural stochastic Arrow-Pratt risk process, decomposed into discrete asymmetric risk seeking and risk averse components that run on different local times in ϵ-disks centered at risk free states. Additionally, we embed Arrow-Pratt (“AP”) risk measure in a simple dynamic system of discounted cash flows with constant volatility, and time varying drift. Signal extraction of Arrow-Pratt risk measure shows that it is highly nonlinear in constant volatility for cash flows. Robust identifying restrictions on the system solution confirm that even for small time periods constant volatility is not a measure of AP risk. By contrast, time-varying volatility measures aspects of embedded AP risk. Whereupon maximal AP risk measure is obtained from a convolution of input volatility and idiosyncratic shocks to the system. We provide four applications for our theory. First, we find that Engle, Ng and Rothschild (1990) Factor-ARCH model for risk premia is misspecified because the factor price of risk is time varying and unstable. Our theory predicts that a hyper-ARCH correction factor is required to remove the Factor-ARCH specification. Second, when applied to analysts beliefs about interest rates and volatility, we find that AP risk measure is a feedback control over stochastic cash flows. Whereupon increased risk aversion to negative shocks to earnings increases volatility. Third, we use an oft cited example of Benes, Shepp and Witsenhausen (1980) to characterize a controlled AP diffusion for a conservative investor who wants to minimize the AP risk process for an asset. Fourth, we recover stochastic differential utility functional from the AP risk process and show how it is functionally equivalent to Duffie and Epstein’s (1992) parametrization.behavioural Arrow-Pratt risk process; asymmetric risk decomposition; asset pricing; Markov process; local martingale; local time change

Shape invariant modelling pricing kernels and risk aversion

Pricing kernels play a major role in quantifying risk aversion and investors' preferences. Several empirical studies reported that pricing kernels exhibit a common pattern across dierent markets. Mostly visual inspection and occasionally numerically summarise are used to make comparison. With increasing amount of information updated every day, the empirical pricing kernels can be viewed as an object evolving over time. We propose a systematic modelling approach to describing the evolution of the empirical pricing kernels. The approach is based on shape invariant models. It captures the common features contained in the shape of the functions and at the same time characterises the variability between the pricing kernels based on a few interpretable parameters. The method is demonstrated with the European options and returns values of DAX index.pricing kernels, risk aversion, risk neutral density

Management Information, Decision Sciences, and Financial Economics : a connection

The paper provides a brief review of the connecting literature in management information, decision sciences, and financial economics, and discusses some research that is related to the three cognate disciplines. Academics could develop theoretical models and subsequent econometric models to estimate the parameters in the associated models, and analyze some interesting issues in the three related disciplines