Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Principles of Physical Layer Security in Multiuser Wireless Networks: A Survey

This paper provides a comprehensive review of the domain of physical layer security in multiuser wireless networks. The essential premise of physical-layer security is to enable the exchange of confidential messages over a wireless medium in the presence of unauthorized eavesdroppers without relying on higher-layer encryption. This can be achieved primarily in two ways: without the need for a secret key by intelligently designing transmit coding strategies, or by exploiting the wireless communication medium to develop secret keys over public channels. The survey begins with an overview of the foundations dating back to the pioneering work of Shannon and Wyner on information-theoretic security. We then describe the evolution of secure transmission strategies from point-to-point channels to multiple-antenna systems, followed by generalizations to multiuser broadcast, multiple-access, interference, and relay networks. Secret-key generation and establishment protocols based on physical layer mechanisms are subsequently covered. Approaches for secrecy based on channel coding design are then examined, along with a description of inter-disciplinary approaches based on game theory and stochastic geometry. The associated problem of physical-layer message authentication is also introduced briefly. The survey concludes with observations on potential research directions in this area.Comment: 23 pages, 10 figures, 303 refs. arXiv admin note: text overlap with arXiv:1303.1609 by other authors. IEEE Communications Surveys and Tutorials, 201

Joint relay scheduling, channel access, and power allocation for green cognitive radio communications

PublishedJournal Article© 1983-2012 IEEE. The capacity of cognitive radio (CR) systems can be enhanced significantly by deploying relay nodes to exploit the spatial diversity. However, the inevitable imperfect sensing in CR has vital effects on the policy of relay selection, channel access, and power allocation that play pivotal roles in the system capacity. The increase in transmission power can improve the system capacity, but results in high energy consumption, which incurs the increase of carbon emission and network operational cost. Most of the existing schemes for CR systems have not jointly considered the imperfect sensing scenario and the tradeoff between the system capacity and energy consumption. To fill in this gap, this paper proposes an energy-aware centralized relay selection scheme that takes into account the relay selection, channel access, and power allocation jointly in CR with imperfect sensing. Specifically, the CR system is formulated as a partially observable Markov decision process (POMDP) to achieve the goal of balancing the system capacity and energy consumption as well as maximizing the system reward. The optimal policy for relay selection, channel access, and power allocation is then derived by virtue of a dynamic programming approach. A dimension reduction strategy is further applied to reduce its high computation complexity. Extensive simulation experiments and results are presented and analysed to demonstrate the significant performance improvement compared to the existing schemes. The performance results show that the received reward increases more than 50% and the network lifetime increases more than 35%, but the system capacity is reduced less than 6% only.This work was supported by the National Natural Science Foundation of China under Grants 61201219, 61171111, 61472150, and 61173045 and in part by the Fundamental Research Funds for the Central Universities under Grant 2013QN122

Optimization and Communication in UAV Networks

UAVs are becoming a reality and attract increasing attention. They can be remotely controlled or completely autonomous and be used alone or as a fleet and in a large set of applications. They are constrained by hardware since they cannot be too heavy and rely on batteries. Their use still raises a large set of exciting new challenges in terms of trajectory optimization and positioning when they are used alone or in cooperation, and communication when they evolve in swarm, to name but a few examples. This book presents some new original contributions regarding UAV or UAV swarm optimization and communication aspects

Markov decision processes with applications in wireless sensor networks: A survey

Ministry of Education, Singapore under its Academic Research Funding Tier

Network coding-aided MAC protocols for cooperative wireless networks

The introduction of third generation (3G) technologies has caused a vast proliferation of wireless devices and networks, generating an increasing demand for high level Quality of Service (QoS). The wide spread of mobile applications has further reinforced the user need for communication, motivating at the same time the concepts of user cooperation and data dissemination. However, this trend towards continuous exchange of information and ubiquitous connectivity is inherently restricted by the energy-greedy functionalities of high-end devices. These limitations, along with the pressure exerted on the Information and Communications Technology (ICT) industry towards energy awareness, have induced the design of novel energy efficient schemes and algorithms. In this context, the Medium Access Control (MAC) layer plays a key role, since it is mainly responsible for the channel access regulation, the transmission scheduling and the resource allocation, thus constituting an appropriate point to effectively address energy efficiency issues that arise due to the users overcrowding. This dissertation provides a contribution to the design, analysis and evaluation of novel MAC protocols for cooperative wireless networks. In our attempt to design energy efficient MAC schemes, we were extensively assisted by the introduction of new techniques, such as Network Coding (NC), that intrinsically bring considerable gains in system performance. The main thesis contributions are divided into two parts. The first part presents NCCARQ, a novel NC-aided Cooperative Automatic Repeat reQuest (ARQ) MAC protocol for wireless networks. NCCARQ introduces a new access paradigm for cooperative ARQ schemes, exploiting NC benefits in bidirectional communication among wireless users. The NCCARQ performance in terms of QoS and energy efficiency is assessed by means of analytical probabilistic models and extensive computer-based simulations, revealing the significant gains we can achieve compared to standardized MAC solutions. In addition, the impact of realistic wireless channel conditions on the MAC protocol operation further motivated us to study the NCCARQ performance in wireless links affected by correlated shadowing, showing that the channel correlation may adversely affect the distributed cooperation benefits. The second part of the thesis is dedicated to the investigation of MAC issues in wireless data dissemination scenarios. In particular, the existence of multiple source nodes in such scenarios generates conflicting situations, considering the selfish behavior of the wireless devices that want to maximize their battery lifetime. Bearing in mind the energy efficiency importance, we propose game theoretic medium access strategies, applying energy-based utility functions which inherently imply energy awareness. In addition, Random Linear NC (RLNC) techniques are adopted to eliminate the need of exchanging excessive control packets, while Analog NC (ANC) is employed to efface the impact of collisions throughout the communication. During the elaboration of this thesis, two general key conclusions have been extracted. First, there is a fundamental requirement for implementation of new MAC protocols in order to effectively deal with state-of-the-art techniques (e.g., NC), recently introduced to enhance both the performance and the energy efficiency of the network. Second, we highlight the importance of designing novel energy efficient MAC protocols, taking into account that traditional approaches - designed mainly to assist the collision avoidance in wireless networks - tend to be obsolete.La presente tesis doctoral contribuye al diseño, análisis y evaluación de nuevos protocolos MAC cooperativos para redes inalámbricas. La introducción de nuevas técnicas, tales como la codificación de red (NC), que intrínsecamente llevan un considerable aumento en el rendimiento del sistema, nos ayudó ampliamente durante el diseño de protocolos MAC energéticamente eficientes. Las principales contribuciones de esta tesis se dividen en dos partes. La primera parte presenta el NCCARQ, un protocolo cooperativo de retransmisión automática (ARQ), asistido por NC para redes inalámbricas. La segunda parte de la tesis se centra en el diseño de protocolos de capa MAC en escenarios inalámbricos de difusión de datos. Teniendo en cuenta la importancia de la eficiencia energética, se proponen técnicas de acceso al medio basadas en teoría de juegos dónde las funciones objetivo están motivadas por el consumo energético. Las soluciones propuestas son evaluadas por medio de modelos analíticos y simulaciones por ordenador