10,539 research outputs found
Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning
Recent advances in combining deep neural network architectures with
reinforcement learning techniques have shown promising potential results in
solving complex control problems with high dimensional state and action spaces.
Inspired by these successes, in this paper, we build two kinds of reinforcement
learning algorithms: deep policy-gradient and value-function based agents which
can predict the best possible traffic signal for a traffic intersection. At
each time step, these adaptive traffic light control agents receive a snapshot
of the current state of a graphical traffic simulator and produce control
signals. The policy-gradient based agent maps its observation directly to the
control signal, however the value-function based agent first estimates values
for all legal control signals. The agent then selects the optimal control
action with the highest value. Our methods show promising results in a traffic
network simulated in the SUMO traffic simulator, without suffering from
instability issues during the training process
Modeling and Verification of Agent based Adaptive Traffic Signal using Symbolic Model Verifier
This paper addresses the issue of modeling and verification of a Multi Agent
System (MAS) scenario. We have considered an agent based adaptive traffic
signal system. The system monitors the smooth flow of traffic at intersection
of two road segment. After describing how the adaptive traffic signal system
can efficiently be used and showing its advantages over traffic signals with
predetermined periods, we have shown how we can transform this scenario into
Finite State Machine (FSM). Once the system is transformed into a FSM, we have
verified the specifications specified in Computational Tree Logic(CTL) using
NuSMV as a model checking tool. Simulation results obtained from NuSMV showed
us whether the system satisfied the specifications or not. It has also showed
us the state where the system specification does not hold. Using which we
traced back our system to find the source, leading to the specification
violation. Finally, we again verified the modified system with NuSMV for its
specifications.Comment: 13 pages, 6 figures, Submitted to International Journal of Computer
Application (IJCA
Learning and Management for Internet-of-Things: Accounting for Adaptivity and Scalability
Internet-of-Things (IoT) envisions an intelligent infrastructure of networked
smart devices offering task-specific monitoring and control services. The
unique features of IoT include extreme heterogeneity, massive number of
devices, and unpredictable dynamics partially due to human interaction. These
call for foundational innovations in network design and management. Ideally, it
should allow efficient adaptation to changing environments, and low-cost
implementation scalable to massive number of devices, subject to stringent
latency constraints. To this end, the overarching goal of this paper is to
outline a unified framework for online learning and management policies in IoT
through joint advances in communication, networking, learning, and
optimization. From the network architecture vantage point, the unified
framework leverages a promising fog architecture that enables smart devices to
have proximity access to cloud functionalities at the network edge, along the
cloud-to-things continuum. From the algorithmic perspective, key innovations
target online approaches adaptive to different degrees of nonstationarity in
IoT dynamics, and their scalable model-free implementation under limited
feedback that motivates blind or bandit approaches. The proposed framework
aspires to offer a stepping stone that leads to systematic designs and analysis
of task-specific learning and management schemes for IoT, along with a host of
new research directions to build on.Comment: Submitted on June 15 to Proceeding of IEEE Special Issue on Adaptive
and Scalable Communication Network
- …