Hedging Bets in Markov Decision Processes

The classical model of Markov decision processes with costs or rewards, while widely used to formalize optimal decision making, cannot capture scenarios where there are multiple objectives for the agent during the system evolution, but only one of these objectives gets actualized upon termination. We introduce the model of Markov decision processes with alternative objectives (MDPAO) for formalizing optimization in such scenarios. To compute the strategy to optimize the expected cost/reward upon termination, we need to figure out how to balance the values of the alternative objectives. This requires analysis of the underlying infinite-state process that tracks the accumulated values of all the objectives. While the decidability of the problem of computing the exact optimal strategy for the general model remains open, we present the following results. First, for a Markov chain with alternative objectives, the optimal expected cost/reward can be computed in polynomial-time. Second, for a single-state process with two actions and multiple objectives we show how to compute the optimal decision strategy. Third, for a process with only two alternative objectives, we present a reduction to the minimum expected accumulated reward problem for one-counter MDPs, and this leads to decidability for this case under some technical restrictions. Finally, we show that optimal cost/reward can be approximated up to a constant additive factor for the general problem

Optimized Bacteria are Environmental Prediction Engines

Experimentalists have observed phenotypic variability in isogenic bacteria populations. We explore the hypothesis that in fluctuating environments this variability is tuned to maximize a bacterium's expected log growth rate, potentially aided by epigenetic markers that store information about past environments. We show that, in a complex, memoryful environment, the maximal expected log growth rate is linear in the instantaneous predictive information---the mutual information between a bacterium's epigenetic markers and future environmental states. Hence, under resource constraints, optimal epigenetic markers are causal states---the minimal sufficient statistics for prediction. This is the minimal amount of information about the past needed to predict the future as well as possible. We suggest new theoretical investigations into and new experiments on bacteria phenotypic bet-hedging in fluctuating complex environments.Comment: 7 pages, 1 figure; http://csc.ucdavis.edu/~cmg/compmech/pubs/obepe.ht

Ambiguity in asset pricing and portfolio choice: a review of the literature

A growing body of empirical evidence suggests that investors’ behavior is not well described by the traditional paradigm of (subjective) expected utility maximization under rational expectations. A literature has arisen that models agents whose choices are consistent with models that are less restrictive than the standard subjective expected utility framework. In this paper we conduct a survey of the existing literature that has explored the implications of decision-making under ambiguity for financial market outcomes, such as portfolio choice and equilibrium asset prices. We conclude that the ambiguity literature has led to a number of significant advances in our ability to rationalize empirical features of asset returns and portfolio decisions, such as the empirical failure of the two-fund separation theorem in portfolio decisions, the modest exposure to risky securities observed for a majority of investors, the home equity preference in international portfolio diversification, the excess volatility of asset returns, the equity premium and the risk-free rate puzzles, and the occurrence of trading break-downs.Capital assets pricing model ; Investments

Learning Under Ambiguity

This paper considers learning when the distinction between risk and ambiguity (Knightian uncertainty) matters. Working within the framework of recursive multiple-priors utility, the paper formulates a counterpart of the Bayesian model of learning about an uncertain parameter from conditionally i.i.d. signals. Ambiguous signals capture responses to information that cannot be captured by noisy signals. They induce nonmonotonic changes in agent confidence and prevent ambiguity from vanishing in the limit. In a dynamic portfolio choice model, learning about ambiguous returns leads to endogenous stock market participation costs that depend on past market performance. Hedging of ambiguity provides a new reason why the investment horizon matters for portfolio choice.ambiguity, learning, noisy signals, ambiguous signals, quality information, portfolio choice, portfolio diversification, Ellsberg Paradox

Why Stocks May Disappoint

Recently much progress has been made in developing optimal portfolio choice models accomodating time-varying opportunity sets, but unless investors are unreasonably risk averse, optimal holdings include unreasonably large equity positions. One reason is that most studies assume investors behave as expected utility maximizers with power utility. In this article, we provide a formal treatment of both static and dynamic portfolio choice using the Disappointment Aversion preferences of Gul (1991). While different from the Kahneman-Tversky (1979) loss aversion utility, these preferences imply asymmetric aversion to gains versus losses and are consistent with the tendency of some people to like lottery-type gambles but dislike stock in-vestments. By calibrating a number of data generating processes to actual US data on stock and bond returns, we find very reasonable portfolios for moderately disappointment averse investors with utility functions exhibiting low curvature. Disappointment aversion preferences affect intertemporal hedging demands and the state dependence of asset allocation in such a way as to not be replicable by standard expected utility functions with higher curvature. Furthermore, it is easy to reconcile the large equity premium observed in the data with disappointment aversion utility of low curvature and reasonable disappointment aversion.

Cephalon, Inc. Taking Risk Management Theory Seriously

We study a firm that justifies its novel use of equity derivatives as a cash-flow hedging strategy. Our purpose is to understand the challenge of translating risk management theory into managerial action. Cephalon Inc., a biotech firm, bought a large block of call options on its own stock. If the FDA approved the firm's new drug, the firm would have large cash needs, which the options were designed to meet. We analyze this stated rationale for the firm's choice, applying the cash flow hedging concepts articulated by Froot, Scharfstein and Stein (1993). In applying the theory to practice, there are lessons for both managers and theorists. Managers consider deadweight costs of financing and of risk management, whereas theory tends to ignore the latter costs. While theory is driven by costs of external financing, managers must measure these costs to arrive at decisions and this measurement problem is severe. Cephalon's risk management decisions seem motivated as much by fluctuations in the availability and cost of external financing and by accounting considerations as by fluctuations in operating cash flows or desired investment. Finally, even a field-based examination of this strategy cannot reject the conclusion that the transaction was motivated by goals other than risk management.

Bi-directional grid absorption barrier constrained stochastic processes with applications in finance and investment

Whilst the gambler’s ruin problem (GRP) is based on martingales and the established probability theory proves that the GRP is a doomed strategy, this research details how the semimartingale framework is required for the grid trading problem (GTP) of financial markets, especially foreign exchange (FX) markets. As banks and financial institutions have the requirement to hedge their FX exposure, the GTP can help provide a framework for greater automation of the hedging process and help forecast which hedge scenarios to avoid. Two theorems are adapted from GRP to GTP and prove that grid trading, whilst still subject to the risk of ruin, has the ability to generate significantly more profitable returns in the short term. This is also supported by extensive simulation and distributional analysis. We introduce two absorption barriers, one at zero balance (ruin) and one at a specified profit target. This extends the traditional GRP and the GTP further by deriving both the probability of ruin and the expected number of steps (of reaching a barrier) to better demonstrate that GTP takes longer to reach ruin than GRP. These statistical results have applications into finance such as multivariate dynamic hedging (Noorian, Flower, & Leong, 2016), portfolio risk optimization, and algorithmic loss recovery