23,190 research outputs found
Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures
Reinforcement learning (RL) constitutes a promising solution for alleviating
the problem of traffic congestion. In particular, deep RL algorithms have been
shown to produce adaptive traffic signal controllers that outperform
conventional systems. However, in order to be reliable in highly dynamic urban
areas, such controllers need to be robust with the respect to a series of
exogenous sources of uncertainty. In this paper, we develop an open-source
callback-based framework for promoting the flexible evaluation of different
deep RL configurations under a traffic simulation environment. With this
framework, we investigate how deep RL-based adaptive traffic controllers
perform under different scenarios, namely under demand surges caused by special
events, capacity reductions from incidents and sensor failures. We extract
several key insights for the development of robust deep RL algorithms for
traffic control and propose concrete designs to mitigate the impact of the
considered exogenous uncertainties.Comment: 8 page
A Case Study on Formal Verification of Self-Adaptive Behaviors in a Decentralized System
Self-adaptation is a promising approach to manage the complexity of modern
software systems. A self-adaptive system is able to adapt autonomously to
internal dynamics and changing conditions in the environment to achieve
particular quality goals. Our particular interest is in decentralized
self-adaptive systems, in which central control of adaptation is not an option.
One important challenge in self-adaptive systems, in particular those with
decentralized control of adaptation, is to provide guarantees about the
intended runtime qualities. In this paper, we present a case study in which we
use model checking to verify behavioral properties of a decentralized
self-adaptive system. Concretely, we contribute with a formalized architecture
model of a decentralized traffic monitoring system and prove a number of
self-adaptation properties for flexibility and robustness. To model the main
processes in the system we use timed automata, and for the specification of the
required properties we use timed computation tree logic. We use the Uppaal tool
to specify the system and verify the flexibility and robustness properties.Comment: In Proceedings FOCLASA 2012, arXiv:1208.432
Adaptive traffic signal control using approximate dynamic programming
This paper presents a study on an adaptive traffic signal controller for real-time operation. The controller aims for three operational objectives: dynamic allocation of green time, automatic adjustment to control parameters, and fast revision of signal plans. The control algorithm is built on approximate dynamic programming (ADP). This approach substantially reduces computational burden by using an approximation to the value function of the dynamic programming and reinforcement learning to update the approximation. We investigate temporal-difference learning and perturbation learning as specific learning techniques for the ADP approach. We find in computer simulation that the ADP controllers achieve substantial reduction in vehicle delays in comparison with optimised fixed-time plans. Our results show that substantial benefits can be gained by increasing the frequency at which the signal plans are revised, which can be achieved conveniently using the ADP approach
Feedback Control Goes Wireless: Guaranteed Stability over Low-power Multi-hop Networks
Closing feedback loops fast and over long distances is key to emerging
applications; for example, robot motion control and swarm coordination require
update intervals of tens of milliseconds. Low-power wireless technology is
preferred for its low cost, small form factor, and flexibility, especially if
the devices support multi-hop communication. So far, however, feedback control
over wireless multi-hop networks has only been shown for update intervals on
the order of seconds. This paper presents a wireless embedded system that tames
imperfections impairing control performance (e.g., jitter and message loss),
and a control design that exploits the essential properties of this system to
provably guarantee closed-loop stability for physical processes with linear
time-invariant dynamics. Using experiments on a cyber-physical testbed with 20
wireless nodes and multiple cart-pole systems, we are the first to demonstrate
and evaluate feedback control and coordination over wireless multi-hop networks
for update intervals of 20 to 50 milliseconds.Comment: Accepted final version to appear in: 10th ACM/IEEE International
Conference on Cyber-Physical Systems (with CPS-IoT Week 2019) (ICCPS '19),
April 16--18, 2019, Montreal, QC, Canad
Recommended from our members
A Survey on Cooperative Longitudinal Motion Control of Multiple Connected and Automated Vehicles
- …