Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial

Optimal Compression and Transmission Rate Control for Node-Lifetime Maximization

We consider a system that is composed of an energy constrained sensor node and a sink node, and devise optimal data compression and transmission policies with an objective to prolong the lifetime of the sensor node. While applying compression before transmission reduces the energy consumption of transmitting the sensed data, blindly applying too much compression may even exceed the cost of transmitting raw data, thereby losing its purpose. Hence, it is important to investigate the trade-off between data compression and transmission energy costs. In this paper, we study the joint optimal compression-transmission design in three scenarios which differ in terms of the available channel information at the sensor node, and cover a wide range of practical situations. We formulate and solve joint optimization problems aiming to maximize the lifetime of the sensor node whilst satisfying specific delay and bit error rate (BER) constraints. Our results show that a jointly optimized compression-transmission policy achieves significantly longer lifetime (90% to 2000%) as compared to optimizing transmission only without compression. Importantly, this performance advantage is most profound when the delay constraint is stringent, which demonstrates its suitability for low latency communication in future wireless networks.Comment: accepted for publication in IEEE Transactions on Wireless Communicaiton

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Lifetime Estimation of Wireless Body Area Sensor Network for Patient Health Monitoring

Wireless Body Area Sensor Networks (WBASN) is an emerging technology which utilizes wireless sensors to implement real-time wearable health monitoring of patients to enhance independent living. These sensors can be worn externally to monitor multiple bio-parameters (such as blood oxygen saturation (SpO2), blood pressure and heart activity) of multiple patients at a central location in the hospital. In health monitoring, the loss of critical or emergency information is a serious issue so there is a concern for quality of service which needs to be addressed. It is important to have an estimate of the time the first node will fail in order to replace or recharge the battery. A common type of failure happens when a node runs out of energy and shuts down. In this work, Monte Carlo simulation is used to determine the lifetime of WBASN. The lifetime of the WBASN is defined in this work as the duration of time until the first sensor failure due to battery depletion. A parametric model of the health care network is created with sets of random input distributions. Probabilistic analysis is used to determine the timing and distributions of nodes\u27 failures in the health monitoring network

Resilient networking in wireless sensor networks

This report deals with security in wireless sensor networks (WSNs), especially in network layer. Multiple secure routing protocols have been proposed in the literature. However, they often use the cryptography to secure routing functionalities. The cryptography alone is not enough to defend against multiple attacks due to the node compromise. Therefore, we need more algorithmic solutions. In this report, we focus on the behavior of routing protocols to determine which properties make them more resilient to attacks. Our aim is to find some answers to the following questions. Are there any existing protocols, not designed initially for security, but which already contain some inherently resilient properties against attacks under which some portion of the network nodes is compromised? If yes, which specific behaviors are making these protocols more resilient? We propose in this report an overview of security strategies for WSNs in general, including existing attacks and defensive measures. In this report we focus at the network layer in particular, and an analysis of the behavior of four particular routing protocols is provided to determine their inherent resiliency to insider attacks. The protocols considered are: Dynamic Source Routing (DSR), Gradient-Based Routing (GBR), Greedy Forwarding (GF) and Random Walk Routing (RWR)

Probabilistic approaches to the design of wireless ad hoc and sensor networks

The emerging wireless technologies has made ubiquitous wireless access a reality and enabled wireless systems to support a large variety of applications. Since the wireless self-configuring networks do not require infrastructure and promise greater flexibility and better coverage, wireless ad hoc and sensor networks have been under intensive research. It is believed that wireless ad hoc and sensor networks can become as important as the Internet. Just as the Internet allows access to digital information anywhere, ad hoc and sensor networks will provide remote interaction with the physical world. Dynamics of the object distribution is one of the most important features of the wireless ad hoc and sensor networks. This dissertation deals with several interesting estimation and optimization problems on the dynamical features of ad hoc and sensor networks. Many demands in application, such as reliability, power efficiency and sensor deployment, of wireless ad hoc and sensor network can be improved by mobility estimation and/or prediction. In this dissertation, we study several random mobility models, present a mobility prediction methodology, which relies on the analysis of the moving patterns of the mobile objects. Through estimating the future movement of objects and analyzing the tradeoff between the estimation cost and the quality of reliability, the optimization of tracking interval for sensor networks is presented. Based on the observation on the location and movement of objects, an optimal sensor placement algorithm is proposed by adaptively learn the dynamical object distribution. Moreover, dynamical boundary of mass objects monitored in a sensor network can be estimated based on the unsupervised learning of the distribution density of objects. In order to provide an accurate estimation of mobile objects, we first study several popular mobility models. Based on these models, we present some mobility prediction algorithms accordingly, which are capable of predicting the moving trajectory of objects in the future. In wireless self-configuring networks, an accurate estimation algorithm allows for improving the link reliability, power efficiency, reducing the traffic delay and optimizing the sensor deployment. The effects of estimation accuracy on the reliability and the power consumption have been studied and analyzed. A new methodology is proposed to optimize the reliability and power efficiency by balancing the trade-off between the quality of performance and estimation cost. By estimating and predicting the mass objects\u27 location and movement, the proposed sensor placement algorithm demonstrates a siguificant improvement on the detection of mass objects with nearmaximal detection accuracy. Quantitative analysis on the effects of mobility estimation and prediction on the accuracy of detection by sensor networks can be conducted with recursive EM algorithms. The future work includes the deployment of the proposed concepts and algorithms into real-world ad hoc and sensor networks