356 research outputs found
Max-Plus Matching Pursuit for Deterministic Markov Decision Processes
We consider deterministic Markov decision processes (MDPs) and apply max-plus
algebra tools to approximate the value iteration algorithm by a
smaller-dimensional iteration based on a representation on dictionaries of
value functions. The setup naturally leads to novel theoretical results which
are simply formulated due to the max-plus algebra structure. For example, when
considering a fixed (non adaptive) finite basis, the computational complexity
of approximating the optimal value function is not directly related to the
number of states, but to notions of covering numbers of the state space. In
order to break the curse of dimensionality in factored state-spaces, we
consider adaptive basis that can adapt to particular problems leading to an
algorithm similar to matching pursuit from signal processing. They currently
come with no theoretical guarantees but work empirically well on simple
deterministic MDPs derived from low-dimensional continuous control problems. We
focus primarily on deterministic MDPs but note that the framework can be
applied to all MDPs by considering measure-based formulations
Autonomous Driving: A Multi-Objective Deep Reinforcement Learning Approach
Autonomous driving is a challenging domain that entails multiple aspects: a vehicle should be able to drive to its destination as fast as possible while avoiding collision, obeying traffic rules and ensuring the comfort of passengers. It's representative of complex reinforcement learning tasks humans encounter in real life. The aim of this thesis is to explore the effectiveness of multi-objective reinforcement learning for such tasks characterized by autonomous driving. In particular, it shows that:
1. Multi-objective reinforcement learning is effective at overcoming some of the difficulties faced by scalar-reward reinforcement learning, and a multi-objective DQN agent based on a variant of thresholded lexicographic Q-learning is successfully trained to drive on multi-lane roads and intersections, yielding and changing lanes according to traffic rules.
2. Data efficiency of (multi-objective) reinforcement learning can be significantly improved by exploiting the factored structure of a task. Specifically, factored Q functions learned on the factored state space can be used as features to the original Q function to speed up learning.
3. Inclusion of history-dependent policies enables an intuitive exact algorithm for multi-objective reinforcement learning with thresholded lexicographic order
- …