Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

ENERGY CONSERVATION FOR WIRELESS AD HOC ROUTING

Self-configuring wireless ad hoc networks have attracted considerable attention in the last few years due to their valuable civil and military applications. One aspect of such networks that has been studied insufficiently is the energy efficiency. Energy efficiency is crucial to prolong the network lifetime and thus make the network more survivable.Nodes in wireless ad hoc networks are most likely to be driven by battery and hence operate on an extremely frugal energy budget. Conventional ad hoc routing protocols are focused on handling the mobility instead of energy efficiency. Energy efficient routing strategies proposed in literature either do not take advantage of sleep modes to conserve energy more efficiently, or incur much overhead in terms of control message and computing complexity to schedule sleep modes and thus are not scalable.In this dissertation, a novel strategy is proposed to manage the sleep of the nodes in the network so that energy can be conserved and network connectivity can be kept. The novelty of the strategy is its extreme simplicity. The idea is derived from the results of the percolation theory, typically called gossiping. Gossiping is a convenient and effective approach and has been successfully applied to several areas of the networking. In the proposed work, we will developa sleep management protocol from gossiping for both static and mobile wireless ad hoc networks. Then the protocol will be extended to the asynchronous network, where nodes manage their own states independently. Analysis and simulations will be conducted to show thecorrectness, effectiveness and efficiency of the proposed work. The comparison between analytical and simulation results will justify them for each other. We will investigate the most important performance aspects concerning the proposed strategy, including the effect ofparameter tuning and the impacts of routing protocols. Furthermore, multiple extensions will be developed to improve the performance and make the proposed strategy apply to different network scenarios

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Energy sustainable paradigms and methods for future mobile networks: A survey

In this survey, we discuss the role of energy in the design of future mobile networks and, in particular, we advocate and elaborate on the use of energy harvesting (EH) hardware as a means to decrease the environmental footprint of 5G technology. To take full advantage of the harvested (renewable) energy, while still meeting the quality of service required by dense 5G deployments, suitable management techniques are here reviewed, highlighting the open issues that are still to be solved to provide eco-friendly and cost-effective mobile architectures. Several solutions have recently been proposed to tackle capacity, coverage and efficiency problems, including: C-RAN, Software Defined Networking (SDN) and fog computing, among others. However, these are not explicitly tailored to increase the energy efficiency of networks featuring renewable energy sources, and have the following limitations: (i) their energy savings are in many cases still insufficient and (ii) they do not consider network elements possessing energy harvesting capabilities. In this paper, we systematically review existing energy sustainable paradigms and methods to address points (i) and (ii), discussing how these can be exploited to obtain highly efficient, energy self-sufficient and high capacity networks. Several open issues have emerged from our review, ranging from the need for accurate energy, transmission and consumption models, to the lack of accurate data traffic profiles, to the use of power transfer, energy cooperation and energy trading techniques. These challenges are here discussed along with some research directions to follow for achieving sustainable 5G systems.Comment: Accepted by Elsevier Computer Communications, 21 pages, 9 figure

Energy-efficient multi-criteria packet forwarding in multi-hop wireless networks

Reliable multi-hop packet forwarding is an important requirement for the implementation of realistic large-scale wireless ad-hoc networks. However, packet forwarding methods based on a single criterion, such as the traditional greedy geographic forwarding, are not sufficient in most realistic wireless settings because perfect-reception-within-rangecannot be assumed. Furthermore, methods where the selection of intermediate relaying nodes is performed at the transmitter-side do not adapt well to rapidly changing network environments. Although a few link-aware geographic forwarding schemes have been reported in the literature, the tradeoffs between multiple decision criteria and their impact on network metrics such as throughput, delay and energy consumption have not been studied. This dissertation presents a series of strategies aimed at addressing the challenges faced by the choice of relay nodes in error-prone dynamic wireless network environments. First, a single-criterion receiver-side relay election (RSRE) is introduced as a distributed alternative to the traditional transmitter-side relay selection. Contrary to the transmitter- side selection, at each hop, an optimal node is elected among receivers to relay packets toward the destination. Next, a multi-criteria RSRE, which factors multiple decision criteria in the election process at lower overhead cost, is proposed. A general cost metric in the form of a multi-parameter mapping function aggregates decision criteria into a single metric used to rank potential relay candidates. A two-criteria RSRE case study shows that a proper combination of greedy forwarding and link quality leads to higher energy efficiency and substantial improvement in the end-to-end delay. Last, mesh multi-path forwarding methods are examined. A generalized mesh construction algorithm in introduced to show impact of a mesh structure on network performance