37,402 research outputs found
Dynamic Spectrum Access in Time-varying Environment: Distributed Learning Beyond Expectation Optimization
This article investigates the problem of dynamic spectrum access for
canonical wireless networks, in which the channel states are time-varying. In
the most existing work, the commonly used optimization objective is to maximize
the expectation of a certain metric (e.g., throughput or achievable rate).
However, it is realized that expectation alone is not enough since some
applications are sensitive to fluctuations. Effective capacity is a promising
metric for time-varying service process since it characterizes the packet delay
violating probability (regarded as an important statistical QoS index), by
taking into account not only the expectation but also other high-order
statistic. Therefore, we formulate the interactions among the users in the
time-varying environment as a non-cooperative game, in which the utility
function is defined as the achieved effective capacity. We prove that it is an
ordinal potential game which has at least one pure strategy Nash equilibrium.
Based on an approximated utility function, we propose a multi-agent learning
algorithm which is proved to achieve stable solutions with dynamic and
incomplete information constraints. The convergence of the proposed learning
algorithm is verified by simulation results. Also, it is shown that the
proposed multi-agent learning algorithm achieves satisfactory performance.Comment: 13 pages, 10 figures, accepted for publication in IEEE Transactions
on Communication
Evolutionarily Stable Spectrum Access
In this paper, we design distributed spectrum access mechanisms with both
complete and incomplete network information. We propose an evolutionary
spectrum access mechanism with complete network information, and show that the
mechanism achieves an equilibrium that is globally evolutionarily stable. With
incomplete network information, we propose a distributed learning mechanism,
where each user utilizes local observations to estimate the expected throughput
and learns to adjust its spectrum access strategy adaptively over time. We show
that the learning mechanism converges to the same evolutionary equilibrium on
the time average. Numerical results show that the proposed mechanisms are
robust to the perturbations of users' channel selections.Comment: arXiv admin note: substantial text overlap with arXiv:1103.102
Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access
We consider the problem of dynamic spectrum access for network utility
maximization in multichannel wireless networks. The shared bandwidth is divided
into K orthogonal channels. In the beginning of each time slot, each user
selects a channel and transmits a packet with a certain transmission
probability. After each time slot, each user that has transmitted a packet
receives a local observation indicating whether its packet was successfully
delivered or not (i.e., ACK signal). The objective is a multi-user strategy for
accessing the spectrum that maximizes a certain network utility in a
distributed manner without online coordination or message exchanges between
users. Obtaining an optimal solution for the spectrum access problem is
computationally expensive in general due to the large state space and partial
observability of the states. To tackle this problem, we develop a novel
distributed dynamic spectrum access algorithm based on deep multi-user
reinforcement leaning. Specifically, at each time slot, each user maps its
current state to spectrum access actions based on a trained deep-Q network used
to maximize the objective function. Game theoretic analysis of the system
dynamics is developed for establishing design principles for the implementation
of the algorithm. Experimental results demonstrate strong performance of the
algorithm.Comment: This work has been accepted for publication in the IEEE Transactions
on Wireless Communication
Database-assisted Spectrum Access in Dynamic Networks: A Distributed Learning Solution
This paper investigates the problem of database-assisted spectrum access in
dynamic TV white spectrum networks, in which the active user set is varying.
Since there is no central controller and information exchange, it encounters
dynamic and incomplete information constraints. To solve this challenge, we
formulate a state-based spectrum access game and a robust spectrum access game.
It is proved that the two games are ordinal potential games with the (expected)
aggregate weighted interference serving as the potential functions. A
distributed learning algorithm is proposed to achieve the pure strategy Nash
equilibrium (NE) of the games. It is shown that the best NE is almost the same
with the optimal solution and the achievable throughput of the proposed
learning algorithm is very close to the optimal one, which validates the
effectiveness of the proposed game-theoretic solution.Comment: 7 pages, 6 figures, to appear in IEEE Acces
Distributed Learning Algorithms for Spectrum Sharing in Spatial Random Access Wireless Networks
We consider distributed optimization over orthogonal collision channels in
spatial random access networks. Users are spatially distributed and each user
is in the interference range of a few other users. Each user is allowed to
transmit over a subset of the shared channels with a certain attempt
probability. We study both the non-cooperative and cooperative settings. In the
former, the goal of each user is to maximize its own rate irrespective of the
utilities of other users. In the latter, the goal is to achieve proportionally
fair rates among users. Simple distributed learning algorithms are developed to
solve these problems. The efficiencies of the proposed algorithms are
demonstrated via both theoretical analysis and simulation results.Comment: 40 pages, 6 figures, accepted for publication in the IEEE
Transactions on Automatic Control, part of this work was presented at the
13th International Symposium on Modeling and Optimization in Mobile, Ad Hoc
and Wireless Networks (WiOpt), 201
Intelligent Wireless Communications Enabled by Cognitive Radio and Machine Learning
The ability to intelligently utilize resources to meet the need of growing
diversity in services and user behavior marks the future of wireless
communication systems. Intelligent wireless communications aims at enabling the
system to perceive and assess the available resources, to autonomously learn to
adapt to the perceived wireless environment, and to reconfigure its operating
mode to maximize the utility of the available resources. The perception
capability and reconfigurability are the essential features of cognitive radio
while modern machine learning techniques project great potential in system
adaptation. In this paper, we discuss the development of the cognitive radio
technology and machine learning techniques and emphasize their roles in
improving spectrum and energy utility of wireless communication systems. We
describe the state-of-the-art of relevant techniques, covering spectrum sensing
and access approaches and powerful machine learning algorithms that enable
spectrum- and energy-efficient communications in dynamic wireless environments.
We also present practical applications of these techniques and identify further
research challenges in cognitive radio and machine learning as applied to the
existing and future wireless communication systems
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
This paper presents a comprehensive literature review on applications of deep
reinforcement learning in communications and networking. Modern networks, e.g.,
Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become
more decentralized and autonomous. In such networks, network entities need to
make decisions locally to maximize the network performance under uncertainty of
network environment. Reinforcement learning has been efficiently used to enable
the network entities to obtain the optimal policy including, e.g., decisions or
actions, given their states when the state and action spaces are small.
However, in complex and large-scale networks, the state and action spaces are
usually large, and the reinforcement learning may not be able to find the
optimal policy in reasonable time. Therefore, deep reinforcement learning, a
combination of reinforcement learning with deep learning, has been developed to
overcome the shortcomings. In this survey, we first give a tutorial of deep
reinforcement learning from fundamental concepts to advanced models. Then, we
review deep reinforcement learning approaches proposed to address emerging
issues in communications and networking. The issues include dynamic network
access, data rate control, wireless caching, data offloading, network security,
and connectivity preservation which are all important to next generation
networks such as 5G and beyond. Furthermore, we present applications of deep
reinforcement learning for traffic routing, resource sharing, and data
collection. Finally, we highlight important challenges, open issues, and future
research directions of applying deep reinforcement learning.Comment: 37 pages, 13 figures, 6 tables, 174 reference paper
File Transfer Application For Sharing Femto Access
In wireless access network optimization, today's main challenges reside in
traffic offload and in the improvement of both capacity and coverage networks.
The operators are interested in solving their localized coverage and capacity
problems in areas where the macro network signal is not able to serve the
demand for mobile data. Thus, the major issue for operators is to find the best
solution at reasonable expanses. The femto cell seems to be the answer to this
problematic. In this work (This work is supported by the COMET project AWARE.
http://www.ftw.at/news/project-start-for-aware-ftw), we focus on the problem of
sharing femto access between a same mobile operator's customers. This problem
can be modeled as a game where service requesters customers (SRCs) and service
providers customers (SPCs) are the players.
This work addresses the sharing femto access problem considering only one SPC
using game theory tools. We consider that SRCs are static and have some similar
and regular connection behavior. We also note that the SPC and each SRC have a
software embedded respectively on its femto access, user equipment (UE).
After each connection requested by a SRC, its software will learn the
strategy increasing its gain knowing that no information about the other SRCs
strategies is given. The following article presents a distributed learning
algorithm with incomplete information running in SRCs software. We will then
answer the following questions for a game with SRCs and one SPC: how many
connections are necessary for each SRC in order to learn the strategy
maximizing its gain? Does this algorithm converge to a stable state? If yes,
does this state a Nash Equilibrium and is there any way to optimize the
learning process duration time triggered by SRCs software?Comment: 15 pages, 9 figures; extended version from Conference ISCIS 201
Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues
As a key technique for enabling artificial intelligence, machine learning
(ML) is capable of solving complex problems without explicit programming.
Motivated by its successful applications to many practical tasks like image
recognition, both industry and the research community have advocated the
applications of ML in wireless communication. This paper comprehensively
surveys the recent advances of the applications of ML in wireless
communication, which are classified as: resource management in the MAC layer,
networking and mobility management in the network layer, and localization in
the application layer. The applications in resource management further include
power control, spectrum management, backhaul management, cache management,
beamformer design and computation resource management, while ML based
networking focuses on the applications in clustering, base station switching
control, user association and routing. Moreover, literatures in each aspect is
organized according to the adopted ML techniques. In addition, several
conditions for applying ML to wireless communication are identified to help
readers decide whether to use ML and which kind of ML techniques to use, and
traditional approaches are also summarized together with their performance
comparison with ML based approaches, based on which the motivations of surveyed
literatures to adopt ML are clarified. Given the extensiveness of the research
area, challenges and unresolved issues are presented to facilitate future
studies, where ML based network slicing, infrastructure update to support ML
based paradigms, open data sets and platforms for researchers, theoretical
guidance for ML implementation and so on are discussed.Comment: 34 pages,8 figure
Echo State Networks for Self-Organizing Resource Allocation in LTE-U with Uplink-Downlink Decoupling
Uplink-downlink decoupling in which users can be associated to different base
stations in the uplink and downlink of heterogeneous small cell networks (SCNs)
has attracted significant attention recently. However, most existing works
focus on simple association mechanisms in LTE SCNs that operate only in the
licensed band. In contrast, in this paper, the problem of resource allocation
with uplink-downlink decoupling is studied for an SCN that incorporates LTE in
the unlicensed band (LTE-U). Here, the users can access both licensed and
unlicensed bands while being associated to different base stations. This
problem is formulated as a noncooperative game that incorporates user
association, spectrum allocation, and load balancing. To solve this problem, a
distributed algorithm based on the machine learning framework of echo state
networks is proposed using which the small base stations autonomously choose
their optimal bands allocation strategies while having only limited information
on the network's and users' states. It is shown that the proposed algorithm
converges to a stationary mixed-strategy distribution which constitutes a mixed
strategy Nash equilibrium for the studied game. Simulation results show that
the proposed approach yields significant gains, in terms of the sum-rate of the
50th percentile of users, that reach up to 60% and 78%, respectively, compared
to Q-learning and Q-learning without decoupling. The results also show that ESN
significantly provides a considerable reduction of information exchange for the
wireless network.Comment: Accepted in the IEEE Transactions on Wireless Communication
- …