Traffic signal control (TSC) is a high-stakes domain that is growing in
importance as traffic volume grows globally. An increasing number of works are
applying reinforcement learning (RL) to TSC; RL can draw on an abundance of
traffic data to improve signalling efficiency. However, RL-based signal
controllers have never been deployed. In this work, we provide the first review
of challenges that must be addressed before RL can be deployed for TSC. We
focus on four challenges involving (1) uncertainty in detection, (2)
reliability of communications, (3) compliance and interpretability, and (4)
heterogeneous road users. We show that the literature on RL-based TSC has made
some progress towards addressing each challenge. However, more work should take
a systems thinking approach that considers the impacts of other pipeline
components on RL.Comment: 26 pages; accepted version, with shortened version published at the
12th International Workshop on Agents in Traffic and Transportation (ATT '22)
at IJCAI 202