15,404 research outputs found
Online Reinforcement Learning for Dynamic Multimedia Systems
In our previous work, we proposed a systematic cross-layer framework for
dynamic multimedia systems, which allows each layer to make autonomous and
foresighted decisions that maximize the system's long-term performance, while
meeting the application's real-time delay constraints. The proposed solution
solved the cross-layer optimization offline, under the assumption that the
multimedia system's probabilistic dynamics were known a priori. In practice,
however, these dynamics are unknown a priori and therefore must be learned
online. In this paper, we address this problem by allowing the multimedia
system layers to learn, through repeated interactions with each other, to
autonomously optimize the system's long-term performance at run-time. We
propose two reinforcement learning algorithms for optimizing the system under
different design constraints: the first algorithm solves the cross-layer
optimization in a centralized manner, and the second solves it in a
decentralized manner. We analyze both algorithms in terms of their required
computation, memory, and inter-layer communication overheads. After noting that
the proposed reinforcement learning algorithms learn too slowly, we introduce a
complementary accelerated learning algorithm that exploits partial knowledge
about the system's dynamics in order to dramatically improve the system's
performance. In our experiments, we demonstrate that decentralized learning can
perform as well as centralized learning, while enabling the layers to act
autonomously. Additionally, we show that existing application-independent
reinforcement learning algorithms, and existing myopic learning algorithms
deployed in multimedia systems, perform significantly worse than our proposed
application-aware and foresighted learning methods.Comment: 35 pages, 11 figures, 10 table
Fast Reinforcement Learning for Energy-Efficient Wireless Communications
We consider the problem of energy-efficient point-to-point transmission of
delay-sensitive data (e.g. multimedia data) over a fading channel. Existing
research on this topic utilizes either physical-layer centric solutions, namely
power-control and adaptive modulation and coding (AMC), or system-level
solutions based on dynamic power management (DPM); however, there is currently
no rigorous and unified framework for simultaneously utilizing both
physical-layer centric and system-level techniques to achieve the minimum
possible energy consumption, under delay constraints, in the presence of
stochastic and a priori unknown traffic and channel conditions. In this report,
we propose such a framework. We formulate the stochastic optimization problem
as a Markov decision process (MDP) and solve it online using reinforcement
learning. The advantages of the proposed online method are that (i) it does not
require a priori knowledge of the traffic arrival and channel statistics to
determine the jointly optimal power-control, AMC, and DPM policies; (ii) it
exploits partial information about the system so that less information needs to
be learned than when using conventional reinforcement learning algorithms; and
(iii) it obviates the need for action exploration, which severely limits the
adaptation speed and run-time performance of conventional reinforcement
learning algorithms. Our results show that the proposed learning algorithms can
converge up to two orders of magnitude faster than a state-of-the-art learning
algorithm for physical layer power-control and up to three orders of magnitude
faster than conventional reinforcement learning algorithms
Cognition-Based Networks: A New Perspective on Network Optimization Using Learning and Distributed Intelligence
IEEE Access
Volume 3, 2015, Article number 7217798, Pages 1512-1530
Open Access
Cognition-based networks: A new perspective on network optimization using learning and distributed intelligence (Article)
Zorzi, M.a , Zanella, A.a, Testolin, A.b, De Filippo De Grazia, M.b, Zorzi, M.bc
a Department of Information Engineering, University of Padua, Padua, Italy
b Department of General Psychology, University of Padua, Padua, Italy
c IRCCS San Camillo Foundation, Venice-Lido, Italy
View additional affiliations
View references (107)
Abstract
In response to the new challenges in the design and operation of communication networks, and taking inspiration from how living beings deal with complexity and scalability, in this paper we introduce an innovative system concept called COgnition-BAsed NETworkS (COBANETS). The proposed approach develops around the systematic application of advanced machine learning techniques and, in particular, unsupervised deep learning and probabilistic generative models for system-wide learning, modeling, optimization, and data representation. Moreover, in COBANETS, we propose to combine this learning architecture with the emerging network virtualization paradigms, which make it possible to actuate automatic optimization and reconfiguration strategies at the system level, thus fully unleashing the potential of the learning approach. Compared with the past and current research efforts in this area, the technical approach outlined in this paper is deeply interdisciplinary and more comprehensive, calling for the synergic combination of expertise of computer scientists, communications and networking engineers, and cognitive scientists, with the ultimate aim of breaking new ground through a profound rethinking of how the modern understanding of cognition can be used in the management and optimization of telecommunication network
- …