56 research outputs found
Computational Methods for Risk-Averse Undiscounted Transient Markov Models
Cataloged from PDF version of article.The total cost problem for discrete-time controlled transient Markov models is considered. The objective functional is a
Markov dynamic risk measure of the total cost. Two solution methods, value and policy iteration, are proposed, and their
convergence is analyzed. In the policy iteration method, we propose two algorithms for policy evaluation: the nonsmooth
Newton method and convex programming, and we prove their convergence. The results are illustrated on a credit limit
control problem
Computational methods for risk-averse undiscounted transient markov models
The total cost problem for discrete-time controlled transient Markov models is considered. The objective functional is a Markov dynamic risk measure of the total cost. Two solution methods, value and policy iteration, are proposed, and their convergence is analyzed. In the policy iteration method, we propose two algorithms for policy evaluation: the nonsmooth Newton method and convex programming, and we prove their convergence. The results are illustrated on a credit limit control problem. © 2014 INFORMS
Risk Aversion in Finite Markov Decision Processes Using Total Cost Criteria and Average Value at Risk
In this paper we present an algorithm to compute risk averse policies in
Markov Decision Processes (MDP) when the total cost criterion is used together
with the average value at risk (AVaR) metric. Risk averse policies are needed
when large deviations from the expected behavior may have detrimental effects,
and conventional MDP algorithms usually ignore this aspect. We provide
conditions for the structure of the underlying MDP ensuring that approximations
for the exact problem can be derived and solved efficiently. Our findings are
novel inasmuch as average value at risk has not previously been considered in
association with the total cost criterion. Our method is demonstrated in a
rapid deployment scenario, whereby a robot is tasked with the objective of
reaching a target location within a temporal deadline where increased speed is
associated with increased probability of failure. We demonstrate that the
proposed algorithm not only produces a risk averse policy reducing the
probability of exceeding the expected temporal deadline, but also provides the
statistical distribution of costs, thus offering a valuable analysis tool
Risk-averse control of undiscounted transient Markov models
We use Markov risk measures to formulate a risk-averse version of the undiscounted total cost problem for a transient controlled Markov process. Using the new concept of a multikernel, we derive conditions for a system to be risk transient, that is, to have finite risk over an infinite time horizon. We derive risk-averse dynamic programming equations satisfied by the optimal policy and we describe methods for solving these equations. We illustrate the results on an optimal stopping problem and an organ transplantation problem. © 2014 Society for Industrial and Applied Mathematic
Markov Decision Processes with Risk-Sensitive Criteria: An Overview
The paper provides an overview of the theory and applications of
risk-sensitive Markov decision processes. The term 'risk-sensitive' refers here
to the use of the Optimized Certainty Equivalent as a means to measure
expectation and risk. This comprises the well-known entropic risk measure and
Conditional Value-at-Risk. We restrict our considerations to stationary
problems with an infinite time horizon. Conditions are given under which
optimal policies exist and solution procedures are explained. We present both
the theory when the Optimized Certainty Equivalent is applied recursively as
well as the case where it is applied to the cumulated reward. Discounted as
well as non-discounted models are reviewe
Theory of Stochastic Optimal Economic Growth
This paper is a survey of the theory of stochastic optimal economic growth.International Development,
- …