Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

EC-CENTRIC: An Energy- and Context-Centric Perspective on IoT Systems and Protocol Design

The radio transceiver of an IoT device is often where most of the energy is consumed. For this reason, most research so far has focused on low power circuit and energy efficient physical layer designs, with the goal of reducing the average energy per information bit required for communication. While these efforts are valuable per se, their actual effectiveness can be partially neutralized by ill-designed network, processing and resource management solutions, which can become a primary factor of performance degradation, in terms of throughput, responsiveness and energy efficiency. The objective of this paper is to describe an energy-centric and context-aware optimization framework that accounts for the energy impact of the fundamental functionalities of an IoT system and that proceeds along three main technical thrusts: 1) balancing signal-dependent processing techniques (compression and feature extraction) and communication tasks; 2) jointly designing channel access and routing protocols to maximize the network lifetime; 3) providing self-adaptability to different operating conditions through the adoption of suitable learning architectures and of flexible/reconfigurable algorithms and protocols. After discussing this framework, we present some preliminary results that validate the effectiveness of our proposed line of action, and show how the use of adaptive signal processing and channel access techniques allows an IoT network to dynamically tune lifetime for signal distortion, according to the requirements dictated by the application

HEH-BMAC: hybrid polling MAC protocol for WBANs operated by human energy harvesting

This paper introduces human energy harvesting medium access control (MAC) protocol (HEH-BMAC), a hybrid polling MAC suitable for wireless body area networks powered by human energy harvesting. The proposed protocol combines two different medium access methods, namely polling (ID-polling) and probabilistic contention access, to adapt its operation to the different energy and state (active/inactive) changes that the network nodes may experience due to their random nature and the time variation of the energy harvesting sources. HEH-BMAC exploits the packet inter-arrival time and the energy harvesting rate information of each node to implement an efficient access scheme with different priority levels. In addition, our protocol can be applied dynamically in realistic networks, since it is adaptive to the topology changes, allowing the insertion/removal of wireless sensor nodes. Extensive simulations have been conducted in order to evaluate the protocol performance and study the throughput and energy tradeoffs.Peer ReviewedPostprint (author's final draft

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial

A group-based wireless body sensors network using energy harvesting for soccer team monitoring

[EN] In team-based sports, it is difficult to monitor physical state of each athlete during the match. Wearable body sensors with wireless connections allow having low-power and low-size devices, that may use energy harvesting, but with low radio coverage area but the main issue comes from the mobility. This paper presents a wireless body sensors network for soccer team players' monitoring. Each player has a body sensor network that use energy harvesting and each player will be a node in the wireless sensor network. This proposal is based on the zone mobility of the players and their dynamism. It allows knowing the physical state of each player during the whole match. Having fast updates and larger connection times to the gateways, the information can be routed through players of both teams, thus a secure system has been added. Simulations show that the proposed system has very good performance in high mobility.This work has been partially supported by the Instituto de Telecomunicacoes, Next Generation Networks and Applications Group (NetGNA), Portugal, by Government of Russian Federation, Grant 074-U01, by National Funding from the FCT - Fundacao para a Ciencia e a Tecnologia through the PEst-OE/EEI/LA0008/2013 Project.Lloret, J.; García Pineda, M.; Catala Monzo, A.; Rodrigues, JJPC. (2016). A group-based wireless body sensors network using energy harvesting for soccer team monitoring. International Journal of Sensor Networks. 21(4):208-225. https://doi.org/10.1504/IJSNET.2016.079172S20822521

Enlightening Network Lifetime based on Dynamic Time Orient Energy Optimization in Wireless Sensor Network

Mobile Ad-hoc Networks (MANET) are a set of Large-scale infrastructure and mobile device networks that build themselves without centralized control to provide various services through mobile. However, the quality of service of MANET is highly dependent on multiple parameters. Many routing schemes in literature use hop count, mobility speed, direction, etc. Similarly, the flow-based approach chooses long routes, which increases latency and reduces throughput efficiency. However, not all methods work well with all Quality of Service (QoS) parameters. To introduce a Dynamic Time Orient Energy Optimization (DTOEO) algorithm to construct the energy-based tree formation to achieve the minimum energy consumption network. Energy-based Dynamic Tree Routing to provide higher energy node and shortest route estimation that help to better transmission quality. In this proposed DTOEO method, perform three stages, there are i). Source node discovery process, ii). Time-orient density estimation, and iii). Energy-based Dynamic Tree Routing. In this stage, orient density estimation evaluates the data transmission size for each window period. To assess the consuming energy in the overall network. The proposed method of performance evaluation using various QoS matrices and its comparison to the existing process provides better performance

Combining distributed queuing with energy harvesting to enable perpetual distributed data collection applications

This is the peer reviewed version of the following article: Vazquez-Gallego F, Tuset-Peiró P, Alonso L, Alonso-Zarate J. Combining distributed queuing with energy harvesting to enable perpetual distributed data collection applications. Trans Emerging Tel Tech. 2017;e3195 , which has been published in final form at https://doi.org/10.1002/ett.3195. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.This paper presents, models, and evaluates energy harvesting–aware distributed queuing (EH-DQ), a novel medium access control protocol that combines distributed queuing with energy harvesting (EH) to address data collection applications in industrial scenarios using long-range and low-power wireless communication technologies. We model the medium access control protocol operation using a Markov chain and evaluate its ability to successfully transmit data without depleting the energy stored at the end devices. In particular, we compare the performance and energy consumption of EH-DQ with that of time-division multiple access (TDMA), which provides an upper limit in data delivery, and EH-aware reservation dynamic frame slotted ALOHA, which is an improved variation of frame slotted ALOHA. To evaluate the performance of these protocols, we use 2 performance metrics: delivery ratio and time efficiency. Delivery ratio measures the ability to successfully transmit data without depleting the energy reserves, whereas time efficiency measures the amount of data that can be transmitted in a certain amount of time. Results show that EH-DQ and TDMA perform close to the optimum in data delivery and outperform EH-aware reservation dynamic frame slotted ALOHA in data delivery and time efficiency. Compared to TDMA, the time efficiency of EH-DQ is insensitive to the amount of harvested energy, making it more suitable for energy-constrained applications. Moreover, compared to TDMA, EH-DQ does not require updated network information to maintain a collision-free schedule, making it suitable for very dynamic networks.Peer ReviewedPostprint (author's final draft

In-Network Distributed Solar Current Prediction

Long-term sensor network deployments demand careful power management. While managing power requires understanding the amount of energy harvestable from the local environment, current solar prediction methods rely only on recent local history, which makes them susceptible to high variability. In this paper, we present a model and algorithms for distributed solar current prediction, based on multiple linear regression to predict future solar current based on local, in-situ climatic and solar measurements. These algorithms leverage spatial information from neighbors and adapt to the changing local conditions not captured by global climatic information. We implement these algorithms on our Fleck platform and run a 7-week-long experiment validating our work. In analyzing our results from this experiment, we determined that computing our model requires an increased energy expenditure of 4.5mJ over simpler models (on the order of 10^{-7}% of the harvested energy) to gain a prediction improvement of 39.7%.Comment: 28 pages, accepted at TOSN and awaiting publicatio