1,881 research outputs found
Event-Triggered Action-Delayed Reinforcement Learning Control of a Mixed Autonomy Signalised Urban Intersection
We propose an event-triggered framework for deciding the traffic light at each lane in a mixed autonomy scenario. We deploy the decision after a suitable delay, and events are triggered based on the satisfaction of a predefined set of conditions. We design the trigger conditions and the delay to increase the vehicles’ throughput. This way, we achieve full exploitation of autonomous vehicles (AVs) potential. The ultimate goal is to obtain vehicle-flows led by AVs at the head. We formulate the decision process of the traffic intersection controller as a deterministic delayed Markov decision process, i.e., the action implementation and evaluation are delayed. We propose a Reinforcement Learning based model-free algorithm to obtain the optimal policy. We show - by simulations - that our algorithm converges, and significantly reduces the average wait-time and the queues length as the fraction of the AVs increases. Our algorithm outperforms our previous work [1] by a quite significant amount
Control of a Mixed Autonomy Signalised Urban Intersection: An Action-Delayed Reinforcement Learning Approach
We consider a mixed autonomy scenario where the traffic intersection
controller decides whether the traffic light will be green or red at each lane
for multiple traffic-light blocks. The objective of the traffic intersection
controller is to minimize the queue length at each lane and maximize the
outflow of vehicles over each block. We consider that the traffic intersection
controller informs the autonomous vehicle (AV) whether the traffic light will
be green or red for the future traffic-light block. Thus, the AV can adapt its
dynamics by solving an optimal control problem. We model the decision process
of the traffic intersection controller as a deterministic delay Markov decision
process owing to the delayed action by the traffic controller. We propose
Reinforcement-learning based algorithm to obtain the optimal policy. We show -
empirically - that our algorithm converges and reduces the energy costs of AVs
drastically as the traffic controller communicates with the AVs.Comment: Accepted for Publication at 24th IEEE International Conference on
Intelligent Transportation (ITSC'2021
Can ChatGPT Enable ITS? The Case of Mixed Traffic Control via Reinforcement Learning
The surge in Reinforcement Learning (RL) applications in Intelligent
Transportation Systems (ITS) has contributed to its growth as well as
highlighted key challenges. However, defining objectives of RL agents in
traffic control and management tasks, as well as aligning policies with these
goals through an effective formulation of Markov Decision Process (MDP), can be
challenging and often require domain experts in both RL and ITS. Recent
advancements in Large Language Models (LLMs) such as GPT-4 highlight their
broad general knowledge, reasoning capabilities, and commonsense priors across
various domains. In this work, we conduct a large-scale user study involving 70
participants to investigate whether novices can leverage ChatGPT to solve
complex mixed traffic control problems. Three environments are tested,
including ring road, bottleneck, and intersection. We find ChatGPT has mixed
results. For intersection and bottleneck, ChatGPT increases number of
successful policies by 150% and 136% compared to solely beginner capabilities,
with some of them even outperforming experts. However, ChatGPT does not provide
consistent improvements across all scenarios
Communication-Efficient Cooperative Multi-Agent PPO via Regulated Segment Mixture in Internet of Vehicles
Multi-Agent Reinforcement Learning (MARL) has become a classic paradigm to
solve diverse, intelligent control tasks like autonomous driving in Internet of
Vehicles (IoV). However, the widely assumed existence of a central node to
implement centralized federated learning-assisted MARL might be impractical in
highly dynamic scenarios, and the excessive communication overheads possibly
overwhelm the IoV system. Therefore, in this paper, we design a communication
efficient cooperative MARL algorithm, named RSM-MAPPO, to reduce the
communication overheads in a fully distributed architecture. In particular,
RSM-MAPPO enhances the multi-agent Proximal Policy Optimization (PPO) by
incorporating the idea of segment mixture and augmenting multiple model
replicas from received neighboring policy segments. Afterwards, RSM-MAPPO
adopts a theory-guided metric to regulate the selection of contributive
replicas to guarantee the policy improvement. Finally, extensive simulations in
a mixed-autonomy traffic control scenario verify the effectiveness of the
RSM-MAPPO algorithm
- …