Additive utility in prospect theory

Prospect theory is currently the main descriptive theory of decision under uncertainty. It generalizes expected utility by introducing nonlinear decision weighting and loss aversion. A difficulty in the study of multiattribute utility under prospect theory is to determine when an attribute yields a gain or a loss. One possibility, adopted in the theoretical literature on multiattribute utility under prospect theory, is to assume that a decision maker determines whether the complete outcome is a gain or a loss. In this holistic evaluation, decision weighting and loss aversion are general and attribute-independent. Another possibility, more common in the empirical literature, is to assume that a decision maker has a reference point for each attribute. We give preference foundations for this attribute-specific evaluation where decision weighting and loss aversion are depending on the attributes

Cumulative Prospect Theory for Parametric and Multiattribute Utilities

In cumulative prospect theory models, different behavior concerning gains and losses is per-mitted. For gains different decision weights are assigned than for losses, and the shape of utility can reveal loss aversion. Decision analyses concentrate on both, the capacities, which determine the decision weights, and the nature of utility. This paper focuses on linear/exponential, power and multilinear utility for decision models under uncertainty. Simple preference axioms are for-mulated for a representation by a cumulative prospect theory function. All models share the following axioms: weak ordering, continuity, monotonicity and tail independence. We first show that in their presence constant absolute (proportional) risk aversion implies linear/exponential (power) utility. Then, in the multiattribute case, considering (mutual) utility independence, it is shown that the utility function is (additive/multiplicative) multilinear.mathematical economics and econometrics ;

Common Mathematical Foundations of Expected Utility and Dual Utility Theories

We show that the main results of the expected utility and dual utility theories can be derived in a unified way from two fundamental mathematical ideas: the separation principle of convex analysis, and integral representations of continuous linear functionals from functional analysis. Our analysis reveals the dual character of utility functions. We also derive new integral representations of dual utility models

Decision by sampling: the role of the decision environment in risky choice

Decision by sampling (DbS) is a theory about how our environment shapes the decisions that we make. Here, I review the application of DbS to risky decision making. According to classical theories of risky decision making, people make stable transformations between outcomes and probabilities and their subjective counterparts using fixed psychoeconomic functions. DbS offers a quite different account. In DbS, the subjective value of an outcome or probability is derived from a series of binary, ordinal comparisons with a sample of other outcomes or probabilities from the decision environment. In this way, the distribution of attribute values in the environment determines the subjective valuations of outcomes and probabilities. I show how DbS interacts with the real-world distributions of gains, losses, and probabilities to produce the classical psychoeconomic functions. I extend DbS to account for preferences in benchmark data sets. Finally, in a challenge to the classical notion of stable subjective valuations, I review evidence that manipulating the distribution of attribute values in the environment changes our subjective valuations just as DbS predicts

Fair social decision under uncertainty and belief disagreements

This paper aims to address two issues related to simultaneous aggregation of utilities and beliefs. The first one is related to how to integrate both inequality and uncertainty considerations into social decision making. The second one is related to how social decision should take disagreements in beliefs into account. To accomplish this, whereas individuals are assumed to abide by Savage model’s of subjective expected utility, society is assumed to prescribe, either to each individual when the ex ante individual well-being is favored or to itself when the ex post individual well-being is favored, acting in accordance with the maximin expected utility theory of Gilboa and Schmeidler (J Math Econ 18:141–153, 1989). Furthermore, it adapts an ex ante Pareto-type condition proposed by Gayer et al. (J Legal Stud 43:151–171, 2014), which says that a prospect Pareto dominates another one if the former gives a higher expected utility than the latter one, for each individual, for all individuals’ beliefs. In the context where the ex ante individual welfare is favored, our ex ante Pareto-type condition is shown to be equivalent to social utility taking the form of a MaxMinMin social welfare function, as well as to the individual set of priors being contained within the range of individual beliefs. However, when the ex post individual welfare is favored, the same Pareto-type condition is shown to be equivalent to social utility taking the form of a MaxMinMin social welfare function, as well as to the social set of priors containing only weighted averages of individual beliefs

Eliciting Utility for (Non)Expected Utility Preferences Using Invariance Transformations

This paper presents a methodology to determine the preferences of an individual facing risk in the framework of (non)-expected utility theory. When individual preference satisfies a given invariance property, his utility function is solution of a functional equation associated to a specific transformation. Conversely, there exist transformations characterizing any given utility function and its invariance property. More precisely, invariance with respect to two transformations uniquely determines the individual utility function. We provide examples of such transformations for CARA or CRRA utility, but also with any other utility specification and discuss the example of DARA and IRRA specifications.Utility theory; risk aversion, elicitation of preferences.

Rethinking the Discount Factor in Reinforcement Learning: A Decision Theoretic Approach

Reinforcement learning (RL) agents have traditionally been tasked with maximizing the value function of a Markov decision process (MDP), either in continuous settings, with fixed discount factor $\gamma < 1$, or in episodic settings, with $\gamma = 1$. While this has proven effective for specific tasks with well-defined objectives (e.g., games), it has never been established that fixed discounting is suitable for general purpose use (e.g., as a model of human preferences). This paper characterizes rationality in sequential decision making using a set of seven axioms and arrives at a form of discounting that generalizes traditional fixed discounting. In particular, our framework admits a state-action dependent "discount" factor that is not constrained to be less than 1, so long as there is eventual long run discounting. Although this broadens the range of possible preference structures in continuous settings, we show that there exists a unique "optimizing MDP" with fixed $\gamma < 1$ whose optimal value function matches the true utility of the optimal policy, and we quantify the difference between value and utility for suboptimal policies. Our work can be seen as providing a normative justification for (a slight generalization of) Martha White's RL task formalism (2017) and other recent departures from the traditional RL, and is relevant to task specification in RL, inverse RL and preference-based RL.Comment: 8 pages + 1 page supplement. In proceedings of AAAI 2019. Slides, poster and bibtex available at https://silviupitis.com/#rethinking-the-discount-factor-in-reinforcement-learning-a-decision-theoretic-approac