3 research outputs found
Inverse Risk-Sensitive Reinforcement Learning
We address the problem of inverse reinforcement learning in Markov decision
processes where the agent is risk-sensitive. In particular, we model
risk-sensitivity in a reinforcement learning framework by making use of models
of human decision-making having their origins in behavioral psychology,
behavioral economics, and neuroscience. We propose a gradient-based inverse
reinforcement learning algorithm that minimizes a loss function defined on the
observed behavior. We demonstrate the performance of the proposed technique on
two examples, the first of which is the canonical Grid World example and the
second of which is a Markov decision process modeling passengers' decisions
regarding ride-sharing. In the latter, we use pricing and travel time data from
a ride-sharing company to construct the transition probabilities and rewards of
the Markov decision process.Comment: v3 (comments regarding updates): We significantly extended the theory
(Theorem 2, 3, 5 and Proposition 3). We also correct some minor typos
throughout the document; v2 (comments regarding updates): We corrected some
notational typos and made clarifications in the proof. We also added
clarifying remarks regarding reference points and acceptance levels which
were previously conflate
Inverse Risk-Sensitive Reinforcement Learning
This work addresses the problem of inverse reinforcement learning in Markov decision processes where the decision-making agent is risk-sensitive. In particular, a risk-sensitive reinforcement learning algorithm with convergence guarantees that makes use of coherent risk metrics and models of human decision-making which have their origins in behavioral psychology and economics is presented. The risk-sensitive reinforcement learning algorithm provides the theoretical underpinning for a gradient-based inverse reinforcement learning algorithm that seeks to minimize a loss function defined on the observed behavior. It is shown that the gradient of the loss function with respect to the model parameters is well defined and computable via a contraction map argument. Evaluation of the proposed technique is performed on a Grid World example, a canonical benchmark problem
Gradient-based inverse risk-sensitive reinforcement learning
We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risksensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of models of human decision-making having their origins in behavioral psychology and economics. We propose a gradient-based inverse reinforcement learning algorithm that minimizes a loss function defined on the observed behavior. We demonstrate the performance of the proposed technique on two examples, the first of which is the canonical Grid World example and the second of which is an MDP modeling passengers' decisions regarding ride-sharing. In the latter, we use pricing and travel time data from a ride-sharing company to construct the transition probabilities and rewards of the MDP