Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

A Tutorial on Clique Problems in Communications and Signal Processing

Since its first use by Euler on the problem of the seven bridges of K\"onigsberg, graph theory has shown excellent abilities in solving and unveiling the properties of multiple discrete optimization problems. The study of the structure of some integer programs reveals equivalence with graph theory problems making a large body of the literature readily available for solving and characterizing the complexity of these problems. This tutorial presents a framework for utilizing a particular graph theory problem, known as the clique problem, for solving communications and signal processing problems. In particular, the paper aims to illustrate the structural properties of integer programs that can be formulated as clique problems through multiple examples in communications and signal processing. To that end, the first part of the tutorial provides various optimal and heuristic solutions for the maximum clique, maximum weight clique, and $k$-clique problems. The tutorial, further, illustrates the use of the clique formulation through numerous contemporary examples in communications and signal processing, mainly in maximum access for non-orthogonal multiple access networks, throughput maximization using index and instantly decodable network coding, collision-free radio frequency identification networks, and resource allocation in cloud-radio access networks. Finally, the tutorial sheds light on the recent advances of such applications, and provides technical insights on ways of dealing with mixed discrete-continuous optimization problems

Multi-objective optimization of massive MIMO 5G wireless networks towards power consumption, uplink and downlink exposure

Featured Application The proposed method serves for the network planning of massive Multiple Input Multiple Output (MIMO) based 5G networks. This will be beneficial for the mobile network operators who aim to deploy wireless networks that are cost-effective and electromagnetic (EMF) -aware while providing maximal coverage for the users. Abstract The rapid development of the number of wireless broadband devices requires that the induced uplink exposure be addressed during the design of the future wireless networks, in addition to the downlink exposure due to the transmission of the base stations. In this paper, the positions and power levels of massive MIMO-LTE (Multiple Input Multiple Output-Long Term Evolution) base stations are optimized towards low power consumption, low downlink and uplink electromagnetic exposure and maximal user coverage. A suburban area in Ghent, Belgium has been considered. The results show that the higher the number of BS antenna elements, the fewer number of BSs the massive MIMO network requires. This leads to a decrease of the downlink exposure (-12% for the electric field and -32% for the downlink dose) and an increase of the uplink exposure (+70% for the uplink dose), whereas both downlink and uplink exposure increase with the number of simultaneous served users (+174% for the electric field and +22% for the uplink SAR). The optimal massive MIMO network presenting the better trade-off between the power consumption, the total dose and the user coverage has been obtained with 37 64-antenna BSs. Moreover, the level of the downlink electromagnetic exposure (electric field) of the massive MIMO network is 5 times lower than the 4G reference scenario

Performance Evaluation of Energy Efficient Optimized Routing Protocol for WBANs Using PSO Protocol

A Wireless Body Area Network (WBAN) is a network that may be worn on the human body or implanted in the human body to transmit data, audio, and video in real-time to assess how vital organs are performing. A WBAN may be either an inter-WBAN or an intra-WBAN network. Intra-WBAN communication occurs when the various body sensors can share information. This is known as inter-WBAN communication, which occurs when two or more WBANs can exchange data with one another. One difficulty involves getting data traffic from wireless sensor nodes to the gateway with as little wasted energy, dropped packets, and downtime as possible. In this paper, the WBAN protocols have been compared with WBAN under Particle Swarm Optimization (PSO), and the performance of various parameters has been analysed for different simulation areas. The WBAN under the PSO protocol reduces the energy consumption by 43.2% against the SIMPLE protocoldue to the effective selection of forwarding nodes based on PSO optimization. In addition to that the experimental WBAN testbed is conducted in indoor environment to study the performance of the routing metrics towards energy and packet reception.

Combined ray-tracing/FDTD and network planner methods for the design of massive MIMO networks

The design of a massive MIMO network requires a channel model that captures the Spatio-temporal dimensions of the propagation environment. In this paper, we propose a novel method combining Hybrid Raytracing - Finite difference time domain (FDTD) and network planner tools to address this requirement. This method provides accurate and realistic EMF exposure models for the design of a massive MIMO network. Using this method, we proceed with the optimization of the BS's locations under the low power consumption and low EMF exposure constraints. Assuming equal preference of the optimization objectives, the simulations show that the uplink localized 10g dose appears to be the dominant factor of the localized 10g EMF exposure. Moreover, a massive MIMO network designed to serve 224 simultaneous active users at the same time-frequency resource is subject to an increase of the total whole-body dose (2 times higher in downlink and +18% in uplink), compared to a design with 14 active users. However, in the same conditions, the downlink localized 10g dose reduces (20 times lower) whereas the uplink localized 10g dose increases (+23%) in comparison with the scenario with fewer users (14). Besides, the electromagnetic field strength in all locations obtained with this new method is 2 times weaker compared to a 4G LTE network, while complying with the international guidelines