32,858 research outputs found
Deep Reinforcement Learning for Event-Triggered Control
Event-triggered control (ETC) methods can achieve high-performance control
with a significantly lower number of samples compared to usual, time-triggered
methods. These frameworks are often based on a mathematical model of the system
and specific designs of controller and event trigger. In this paper, we show
how deep reinforcement learning (DRL) algorithms can be leveraged to
simultaneously learn control and communication behavior from scratch, and
present a DRL approach that is particularly suitable for ETC. To our knowledge,
this is the first work to apply DRL to ETC. We validate the approach on
multiple control tasks and compare it to model-based event-triggering
frameworks. In particular, we demonstrate that it can, other than many
model-based ETC designs, be straightforwardly applied to nonlinear systems
Optimal Network Control in Partially-Controllable Networks
The effectiveness of many optimal network control algorithms (e.g.,
BackPressure) relies on the premise that all of the nodes are fully
controllable. However, these algorithms may yield poor performance in a
partially-controllable network where a subset of nodes are uncontrollable and
use some unknown policy. Such a partially-controllable model is of increasing
importance in real-world networked systems such as overlay-underlay networks.
In this paper, we design optimal network control algorithms that can stabilize
a partially-controllable network. We first study the scenario where
uncontrollable nodes use a queue-agnostic policy, and propose a low-complexity
throughput-optimal algorithm, called Tracking-MaxWeight (TMW), which enhances
the original MaxWeight algorithm with an explicit learning of the policy used
by uncontrollable nodes. Next, we investigate the scenario where uncontrollable
nodes use a queue-dependent policy and the problem is formulated as an MDP with
unknown queueing dynamics. We propose a new reinforcement learning algorithm,
called Truncated Upper Confidence Reinforcement Learning (TUCRL), and prove
that TUCRL achieves tunable three-way tradeoffs between throughput, delay and
convergence rate
Event-triggered Pulse Control with Model Learning (if Necessary)
In networked control systems, communication is a shared and therefore scarce
resource. Event-triggered control (ETC) can achieve high performance control
with a significantly reduced amount of samples compared to classical, periodic
control schemes. However, ETC methods usually rely on the availability of an
accurate dynamics model, which is oftentimes not readily available. In this
paper, we propose a novel event-triggered pulse control strategy that learns
dynamics models if necessary. In addition to adapting to changing dynamics, the
method also represents a suitable replacement for the integral part typically
used in periodic control.Comment: Accepted final version to appear in: Proc. of the American Control
Conference, 201
A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning
Automatic decision-making approaches, such as reinforcement learning (RL),
have been applied to (partially) solve the resource allocation problem
adaptively in the cloud computing system. However, a complete cloud resource
allocation framework exhibits high dimensions in state and action spaces, which
prohibit the usefulness of traditional RL techniques. In addition, high power
consumption has become one of the critical concerns in design and control of
cloud computing systems, which degrades system reliability and increases
cooling cost. An effective dynamic power management (DPM) policy should
minimize power consumption while maintaining performance degradation within an
acceptable level. Thus, a joint virtual machine (VM) resource allocation and
power management framework is critical to the overall cloud computing system.
Moreover, novel solution framework is necessary to address the even higher
dimensions in state and action spaces. In this paper, we propose a novel
hierarchical framework for solving the overall resource allocation and power
management problem in cloud computing systems. The proposed hierarchical
framework comprises a global tier for VM resource allocation to the servers and
a local tier for distributed power management of local servers. The emerging
deep reinforcement learning (DRL) technique, which can deal with complicated
control problems with large state space, is adopted to solve the global tier
problem. Furthermore, an autoencoder and a novel weight sharing structure are
adopted to handle the high-dimensional state space and accelerate the
convergence speed. On the other hand, the local tier of distributed server
power managements comprises an LSTM based workload predictor and a model-free
RL based power manager, operating in a distributed manner.Comment: accepted by 37th IEEE International Conference on Distributed
Computing (ICDCS 2017
- …