Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Building a green connected future: smart (Internet of) Things for smart networks

The vision of Internet of Things (IoT) promises to reshape society by creating a future where we will be surrounded by a smart environment that is constantly aware of the users and has the ability to adapt to any changes. In the IoT, a huge variety of smart devices is interconnected to form a network of distributed agents that continuously share and process information. This communication paradigm has been recognized as one of the key enablers of the rapidly emerging applications that make up the fabric of the IoT. These networks, often called wireless sensor networks (WSNs), are characterized by the low cost of their components, their pervasive connectivity, and their self-organization features, which allow them to cooperate with other IoT elements to create large-scale heterogeneous information systems. However, a number of considerable challenges is arising when considering the design of large-scale WSNs. In particular, these networks are made up by embedded devices that suffer from severe power constraints and limited resources. The advent of low-power sensor nodes coupled with intelligent software and hardware technologies has led to the era of green wireless networks. From the hardware perspective, green sensor nodes are endowed with energy scavenging capabilities to overcome energy-related limitations. They are also endowed with low-power triggering techniques, i.e., wake-up radios, to eliminate idle listening-induced communication costs. Green wireless networks are considered a fundamental vehicle for enabling all those critical IoT applications where devices, for different reasons, do not carry batteries, and that therefore only harvest energy and store it for future use. These networks are considered to have the potential of infinite lifetime since they do not depend on batteries, or on any other limited power sources. Wake-up radios, coupled with energy provisioning techniques, further assist on overcoming the physical constraints of traditional WSNs. In addition, they are particularly important in green WSNs scenarios in which it is difficult to achieve energy neutrality due to limited harvesting rates. In this PhD thesis we set to investigate how different data forwarding mechanisms can make the most of these green wireless networks-enabling technologies, namely, energy harvesting and wake-up radios. Specifically, we present a number of cross-layer routing approaches with different forwarding design choices and study their consequences on network performance. Among the most promising protocol design techniques, the past decade has shown the increasingly intensive adoption of techniques based on various forms of machine learning to increase and optimize the performance of WSNs. However, learning techniques can suffer from high computational costs as nodes drain a considerable percentage of their energy budget to run sophisticated software procedures, predict accurate information and determine optimal decision. This thesis addresses also the problem of local computational requirements of learning-based data forwarding strategies by investigating their impact on the performance of the network. Results indicate that local computation can be a major source of energy consumption; it’s impact on network performance should not be neglected

Markov decision processes with applications in wireless sensor networks: A survey

Ministry of Education, Singapore under its Academic Research Funding Tier

Energy harvesting-aware design of wireless networks

Recent advances in low-power electronics and energy-harvesting (EH) technologies enable the design of self-sustained devices that collect part, or all, of the needed energy from the environment. Several systems can take advantage of EH, ranging from portable devices to wireless sensor networks (WSNs). While conventional design for battery-powered systems is mainly concerned with the battery lifetime, a key advantage of EH is that it enables potential perpetual operation of the devices, without requiring maintenance for battery substitutions. However, the inherent unpredictability regarding the amount of energy that can be collected from the environment might cause temporary energy shortages, which might prevent the devices to operate regularly. This uncertainty calls for the development of energy management techniques that are tailored to the EH dynamics. While most previous work on EH-capable systems has focused on energy management for single devices, the main contributions of this dissertation is the analysis and design of medium access control (MAC) protocols for WSNs operated by EH-capable devices. In particular, the dissertation first considers random access MAC protocols for single-hop EH networks, in which a fusion center collects data from a set of nodes distributed in its surrounding. MAC protocols commonly used in WSNs, such as time division multiple access (TDMA), framed-ALOHA (FA) and dynamic-FA (DFA) are investigated in the presence of EH-capable devices. A new ALOHA-based MAC protocol tailored to EH-networks, referred to as energy group-DFA (EG-DFA), is then proposed. In EG-DFA nodes with similar energy availability are grouped together and access the channel independently from other groups. It is shown that EG-DFA significantly outperforms the DFA protocol. Centralized scheduling-based MAC protocols for single-hop EH-networks with communication resource constraints are considered next. Two main scenarios are addressed, namely: i) nodes exclusively powered via EH; ii) nodes powered by a hybrid energy storage system, which is composed by a non-rechargeable battery and a capacitor charged via EH. For the former case the goal is the maximization of the network throughput, while in the latter the aim is maximizing the lifetime of the non-rechargeable batteries. For both scenarios optimal scheduling policies are derived by assuming different levels of information available at the fusion center about the energy availability at the nodes. When optimal policies are not derived explicitly, suboptimal policies are proposed and compared with performance upper bounds. Energy management policies for single devices have been investigated as well by focusing on radio frequency identification (RFID) systems, when the latter are operated by enhanced RFID tags with energy harvesting capabilities

Network Science for IoT

The research work presented in this thesis is based on the concept and defintion of network that can spread in several and different real world contexts. Indeed, we can refer to a network in a telecommunications sense considering a collection of transmitters, receivers, and communication channels that send or are used to send information to one another. However, as a matter of fact, in nature there are other several examples of networks: the human brain is one of them. The relationship between the actors in Hollywood can be studied in terms of network as well, a generic social community can be compared to a network, eco-systems are networks of species. The recent Network Science aims at studying all these systems using a set of common mathematical methods. In the following of the thesis, we will focus on some of well known telecommunications networks issues using standard telecommunications procedures to address them, with relevant reference to video flow transmissions and management of electric vehicles networks. At the same time, different models aiming at reach the same goals in contexts that may differ from a telecommunications setup can be used. In more details, we will evaluate queueing systems, jamming problems, groups recognition in networks, and mobile computing using game theoretic approaches. It is worth noting that this aspect can be also seen in a reverse order. Indeed, we will discuss how standard telecommunications analysis can be used to investigate on problems not directly related to a telecommunications background. In particular, one of our future purposes is to investigate on the brain connectivity that is raising significant interest in the recent scientific society

Applied Metaheuristic Computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC

Applied Methuerstic computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC