Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

QoS BASED ENERGY EFFICIENT ROUTING IN WIRELESS SENSOR NETWORK

A Wireless Sensor Networks (WSN) is composed of a large number of low-powered sensor nodes that are randomly deployed to collect environmental data. In a WSN, because of energy scarceness, energy efficient gathering of sensed information is one of the most critical issues. Thus, most of the WSN routing protocols found in the literature have considered energy awareness as a key design issue. Factors like throughput, latency and delay are not considered as critical issues in these protocols. However, emerging WSN applications that involve multimedia and imagining sensors require end-to-end delay within acceptable limits. Hence, in addition to energy efficiency, the parameters (delay, packet loss ratio, throughput and coverage) have now become issues of primary concern. Such performance metrics are usually referred to as the Quality of Service (QoS) in communication systems. Therefore, to have efficient use of a sensor node’s energy, and the ability to transmit the imaging and multimedia data in a timely manner, requires both a QoS based and energy efficient routing protocol. In this research work, a QoS based energy efficient routing protocol for WSN is proposed. To achieve QoS based energy efficient routing, three protocols are proposed, namely the QoS based Energy Efficient Clustering (QoSEC) for a WSN, the QoS based Energy Efficient Sleep/Wake Scheduling (QoSES) for a WSN, and the QoS based Energy Efficient Mobile Sink (QoSEM) based Routing for a Clustered WSN. Firstly, in the QoSEC, to achieve energy efficiency and to prolong network/coverage lifetime, some nodes with additional energy resources, termed as super-nodes, in addition to normal capability nodes, are deployed. Multi-hierarchy clustering is done by having super-nodes (acting as a local sink) at the top tier, cluster head (normal node) at the middle tier, and cluster member (normal node) at the lowest tier in the hierarchy. Clustering within normal sensor nodes is done by optimizing the network/coverage lifetime through a cluster-head-selection algorithm and a sleep/wake scheduling algorithm. QoSEC resolves the hot spot problem and prolongs network/coverage lifetime. Secondly, the QoSES addressed the delay-minimization problem in sleep/wake scheduling for event-driven sensor networks for delay-sensitive applications. For this purpose, QoSES assigns different sleep/wake intervals (longer wake interval) to potential overloaded nodes, according to their varied traffic load requirement defined a) by node position in the network, b) by node topological importance, and c) by handling burst traffic in the proximity of the event occurrence node. Using these heuristics, QoSES minimizes the congestion at nodes having heavy traffic loads and ultimately reduces end-to-end delay while maximizing the throughput. Lastly, the QoSEM addresses hot spot problem, delay minimization, and QoS assurance. To address hot-spot problem, mobile sink is used, that move in the network to gather data by virtue of which nodes near to the mobile sink changes with each movement, consequently hot spot problem is minimized. To achieve delay minimization, static sink is used in addition to the mobile sink. Delay sensitive data is forwarded to the static sink, while the delay tolerant data is sent through the mobile sink. For QoS assurance, incoming traffic is divided into different traffic classes and each traffic class is assigned different priority based on their QoS requirement (bandwidth, delay) determine by its message type and content. Furthermore, to minimize delay in mobile sink data gathering, the mobile sink is moved throughout the network based on the priority messages at the nodes. Using these heuristics, QoSEM incur less end-to-end delay, is energy efficient, as well as being able to ensure QoS. Simulations are carried out to evaluate the performance of the proposed protocols of QoSEC, QoSES and QoSEM, by comparing their performance with the established contemporary protocols. Simulation results have demonstrated that when compared with contemporary protocols, each of the proposed protocol significantly prolong the network and coverage lifetime, as well as improve the other QoS routing parameters, such as delay, packet loss ratio, and throughput

A Survey on Mobile Charging Techniques in Wireless Rechargeable Sensor Networks

The recent breakthrough in wireless power transfer (WPT) technology has empowered wireless rechargeable sensor networks (WRSNs) by facilitating stable and continuous energy supply to sensors through mobile chargers (MCs). A plethora of studies have been carried out over the last decade in this regard. However, no comprehensive survey exists to compile the state-of-the-art literature and provide insight into future research directions. To fill this gap, we put forward a detailed survey on mobile charging techniques (MCTs) in WRSNs. In particular, we first describe the network model, various WPT techniques with empirical models, system design issues and performance metrics concerning the MCTs. Next, we introduce an exhaustive taxonomy of the MCTs based on various design attributes and then review the literature by categorizing it into periodic and on-demand charging techniques. In addition, we compare the state-of-the-art MCTs in terms of objectives, constraints, solution approaches, charging options, design issues, performance metrics, evaluation methods, and limitations. Finally, we highlight some potential directions for future research

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Cross-layer design through joint routing and link allocation in wireless sensor networks

Both energy and bandwidth are scarce resources in sensor networks. In the past, the energy efficient routing problem has been extensively studied in efforts to maximize sensor network lifetimes, but the link bandwidth has been optimistically assumed to be abundant. Because energy constraint affects how data should be routed, link bandwidth affects not only the routing topology, but also the allowed data rate on each link, which in turn affects the lifetime. Previous research that focus on energy efficient operations in sensor networks with the sole objective of maximizing network lifetime only consider the energy constraint ignoring the bandwidth constraint. This thesis shows how infeasible these solutions can be when bandwidth does present a constraint. It provides a new mathematical model that address both energy and bandwidth constraints and proposes two efficient heuristics for routing and rate allocation. Simulation results show that these heuristics provide more feasible routing solutions than previous work, and significantly improve throughput. A method of assigning the time slot based on the given link rates is presented. The cross layer design approach improves channel utility significantly and completely solves the hidden terminal and exposed terminal problems --Abstract, page iii

QoS BASED ENERGY EFFICIENT ROUTING IN WIRELESS SENSOR NETWORK

A Wireless Sensor Networks (WSN) is composed of a large number of low-powered sensor nodes that are randomly deployed to collect environmental data. In a WSN, because of energy scarceness, energy efficient gathering of sensed information is one of the most critical issues. Thus, most of the WSN routing protocols found in the literature have considered energy awareness as a key design issue. Factors like throughput, latency and delay are not considered as critical issues in these protocols. However, emerging WSN applications that involve multimedia and imagining sensors require end-to-end delay within acceptable limits. Hence, in addition to energy efficiency, the parameters (delay, packet loss ratio, throughput and coverage) have now become issues of primary concern. Such performance metrics are usually referred to as the Quality of Service (QoS) in communication systems. Therefore, to have efficient use of a sensor node’s energy, and the ability to transmit the imaging and multimedia data in a timely manner, requires both a QoS based and energy efficient routing protocol. In this research work, a QoS based energy efficient routing protocol for WSN is proposed. To achieve QoS based energy efficient routing, three protocols are proposed, namely the QoS based Energy Efficient Clustering (QoSEC) for a WSN, the QoS based Energy Efficient Sleep/Wake Scheduling (QoSES) for a WSN, and the QoS based Energy Efficient Mobile Sink (QoSEM) based Routing for a Clustered WSN. Firstly, in the QoSEC, to achieve energy efficiency and to prolong network/coverage lifetime, some nodes with additional energy resources, termed as super-nodes, in addition to normal capability nodes, are deployed. Multi-hierarchy clustering is done by having super-nodes (acting as a local sink) at the top tier, cluster head (normal node) at the middle tier, and cluster member (normal node) at the lowest tier in the hierarchy. Clustering within normal sensor nodes is done by optimizing the network/coverage lifetime through a cluster-head-selection algorithm and a sleep/wake scheduling algorithm. QoSEC resolves the hot spot problem and prolongs network/coverage lifetime. Secondly, the QoSES addressed the delay-minimization problem in sleep/wake scheduling for event-driven sensor networks for delay-sensitive applications. For this purpose, QoSES assigns different sleep/wake intervals (longer wake interval) to potential overloaded nodes, according to their varied traffic load requirement defined a) by node position in the network, b) by node topological importance, and c) by handling burst traffic in the proximity of the event occurrence node. Using these heuristics, QoSES minimizes the congestion at nodes having heavy traffic loads and ultimately reduces end-to-end delay while maximizing the throughput. Lastly, the QoSEM addresses hot spot problem, delay minimization, and QoS assurance. To address hot-spot problem, mobile sink is used, that move in the network to gather data by virtue of which nodes near to the mobile sink changes with each movement, consequently hot spot problem is minimized. To achieve delay minimization, static sink is used in addition to the mobile sink. Delay sensitive data is forwarded to the static sink, while the delay tolerant data is sent through the mobile sink. For QoS assurance, incoming traffic is divided into different traffic classes and each traffic class is assigned different priority based on their QoS requirement (bandwidth, delay) determine by its message type and content. Furthermore, to minimize delay in mobile sink data gathering, the mobile sink is moved throughout the network based on the priority messages at the nodes. Using these heuristics, QoSEM incur less end-to-end delay, is energy efficient, as well as being able to ensure QoS. Simulations are carried out to evaluate the performance of the proposed protocols of QoSEC, QoSES and QoSEM, by comparing their performance with the established contemporary protocols. Simulation results have demonstrated that when compared with contemporary protocols, each of the proposed protocol significantly prolong the network and coverage lifetime, as well as improve the other QoS routing parameters, such as delay, packet loss ratio, and throughput