Reinforcement learning is an essential paradigm for solving sequential
decision problems under uncertainty. Despite many remarkable achievements in
recent decades, applying reinforcement learning methods in the real world
remains challenging. One of the main obstacles is that reinforcement learning
agents lack a fundamental understanding of the world and must therefore learn
from scratch through numerous trial-and-error interactions. They may also face
challenges in providing explanations for their decisions and generalizing the
acquired knowledge. Causality, however, offers a notable advantage as it can
formalize knowledge in a systematic manner and leverage invariance for
effective knowledge transfer. This has led to the emergence of causal
reinforcement learning, a subfield of reinforcement learning that seeks to
enhance existing algorithms by incorporating causal relationships into the
learning process. In this survey, we comprehensively review the literature on
causal reinforcement learning. We first introduce the basic concepts of
causality and reinforcement learning, and then explain how causality can
address core challenges in non-causal reinforcement learning. We categorize and
systematically review existing causal reinforcement learning approaches based
on their target problems and methodologies. Finally, we outline open issues and
future directions in this emerging field.Comment: 48 pages, 10 figure