Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Optimal Compression and Transmission Rate Control for Node-Lifetime Maximization

We consider a system that is composed of an energy constrained sensor node and a sink node, and devise optimal data compression and transmission policies with an objective to prolong the lifetime of the sensor node. While applying compression before transmission reduces the energy consumption of transmitting the sensed data, blindly applying too much compression may even exceed the cost of transmitting raw data, thereby losing its purpose. Hence, it is important to investigate the trade-off between data compression and transmission energy costs. In this paper, we study the joint optimal compression-transmission design in three scenarios which differ in terms of the available channel information at the sensor node, and cover a wide range of practical situations. We formulate and solve joint optimization problems aiming to maximize the lifetime of the sensor node whilst satisfying specific delay and bit error rate (BER) constraints. Our results show that a jointly optimized compression-transmission policy achieves significantly longer lifetime (90% to 2000%) as compared to optimizing transmission only without compression. Importantly, this performance advantage is most profound when the delay constraint is stringent, which demonstrates its suitability for low latency communication in future wireless networks.Comment: accepted for publication in IEEE Transactions on Wireless Communicaiton

Adaptive tracking of people and vehicles using mobile platforms

Tracking algorithms have important applications in detection of humans and vehicles for border security and other areas. For large-scale deployment of such algorithms, it is critical to provide methods for their cost- and energy-efficient realization. To this end, commodity mobile devices have significant potential for use as prototyping and testing platforms due to their low cost, widespread availability, and integration of advanced communications, sensing, and processing features. Prototypes developed on mobile platforms can be tested, fine-tuned, and demonstrated in the field and then provide reference implementations for application-specific disposable sensor node implementations that are targeted for deployment. In this paper, we develop a novel, adaptive tracking system that is optimized for energy-efficient, real-time operation on off-the-shelf mobile platforms. Our tracking system applies principles of dynamic data-driven application systems (DDDAS) to periodically monitor system operating characteristics and apply these measurements to dynamically adapt the specific classifier configurations that the system employs. Our resulting adaptive approach enables powerful optimization of trade-offs among energy consumption, real-time performance, and tracking accuracy based on time-varying changes in operational characteristics. Through experiments employing an Android-based tablet platform, we demonstrate the efficiency of our proposed tracking system design for multimode detection of human and vehicle targets.publishedVersionPeer reviewe

Optimal finite horizon sensing for wirelessly powered devices

We are witnessing a significant advancements in the sensor technologies which has enabled a broad spectrum of applications. Often, the resolution of the produced data by the sensors significantly affects the output quality of an application. We study a sensing resolution optimization problem for a wireless powered device (WPD) that is powered by wireless power transfer (WPT) from an access point (AP). We study a class of harvest-first-transmit-later type of WPT policy, where an access point (AP) first employs RF power to recharge the WPD in the down-link, and then, collects the data from the WPD in the up-link. The WPD optimizes the sensing resolution, WPT duration and dynamic power control in the up-link to maximize an application dependant utility at the AP. The utility of a transmitted packet is only achieved if the data is delivered successfully within a finite time. Thus, we first study a finite horizon throughput maximization problem by jointly optimizing the WPT duration and power control. We prove that the optimal WPT duration obeys a time-dependent threshold form depending on the energy state of the WPD. In the subsequent data transmission stage, the optimal transmit power allocations for the WPD is shown to posses a channel-dependent fractional structure. Then, we optimize the sensing resolution of the WPD by using a Bayesian inference based multi armed bandit problem with fast convergence property to strike a balance between the quality of the sensed data and the probability of successfully delivering it

An Agent-Based Distributed Coordination Mechanism for Wireless Visual Sensor Nodes Using Dynamic Programming

The efficient management of the limited energy resources of a wireless visual sensor network is central to its successful operation. Within this context, this article focuses on the adaptive sampling, forwarding, and routing actions of each node in order to maximise the information value of the data collected. These actions are inter-related in a multi-hop routing scenario because each node’s energy consumption must be optimally allocated between sampling and transmitting its own data, receiving and forwarding the data of other nodes, and routing any data. Thus, we develop two optimal agent-based decentralised algorithms to solve this distributed constraint optimization problem. The first assumes that the route by which data is forwarded to the base station is fixed, and then calculates the optimal sampling, transmitting, and forwarding actions that each node should perform. The second assumes flexible routing, and makes optimal decisions regarding both the integration of actions that each node should choose, and also the route by which the data should be forwarded to the base station. The two algorithms represent a trade-off in optimality, communication cost, and processing time. In an empirical evaluation on sensor networks (whose underlying communication networks exhibit loops), we show that the algorithm with flexible routing is able to deliver approximately twice the quantity of information to the base station compared to the algorithm using fixed routing (where an arbitrary choice of route is made). However, this gain comes at a considerable communication and computational cost (increasing both by a factor of 100 times). Thus, while the algorithm with flexible routing is suitable for networks with a small numbers of nodes, it scales poorly, and as the size of the network increases, the algorithm with fixed routing is favoured

EC-CENTRIC: An Energy- and Context-Centric Perspective on IoT Systems and Protocol Design

The radio transceiver of an IoT device is often where most of the energy is consumed. For this reason, most research so far has focused on low power circuit and energy efficient physical layer designs, with the goal of reducing the average energy per information bit required for communication. While these efforts are valuable per se, their actual effectiveness can be partially neutralized by ill-designed network, processing and resource management solutions, which can become a primary factor of performance degradation, in terms of throughput, responsiveness and energy efficiency. The objective of this paper is to describe an energy-centric and context-aware optimization framework that accounts for the energy impact of the fundamental functionalities of an IoT system and that proceeds along three main technical thrusts: 1) balancing signal-dependent processing techniques (compression and feature extraction) and communication tasks; 2) jointly designing channel access and routing protocols to maximize the network lifetime; 3) providing self-adaptability to different operating conditions through the adoption of suitable learning architectures and of flexible/reconfigurable algorithms and protocols. After discussing this framework, we present some preliminary results that validate the effectiveness of our proposed line of action, and show how the use of adaptive signal processing and channel access techniques allows an IoT network to dynamically tune lifetime for signal distortion, according to the requirements dictated by the application

Task Allocation among Connected Devices: Requirements, Approaches and Challenges

Task allocation (TA) is essential when deploying application tasks to systems of connected devices with dissimilar and time-varying characteristics. The challenge of an efficient TA is to assign the tasks to the best devices, according to the context and task requirements. The main purpose of this paper is to study the different connotations of the concept of TA efficiency, and the key factors that most impact on it, so that relevant design guidelines can be defined. The paper first analyzes the domains of connected devices where TA has an important role, which brings to this classification: Internet of Things (IoT), Sensor and Actuator Networks (SAN), Multi-Robot Systems (MRS), Mobile Crowdsensing (MCS), and Unmanned Aerial Vehicles (UAV). The paper then demonstrates that the impact of the key factors on the domains actually affects the design choices of the state-of-the-art TA solutions. It results that resource management has most significantly driven the design of TA algorithms in all domains, especially IoT and SAN. The fulfillment of coverage requirements is important for the definition of TA solutions in MCS and UAV. Quality of Information requirements are mostly included in MCS TA strategies, similar to the design of appropriate incentives. The paper also discusses the issues that need to be addressed by future research activities, i.e.: allowing interoperability of platforms in the implementation of TA functionalities; introducing appropriate trust evaluation algorithms; extending the list of tasks performed by objects; designing TA strategies where network service providers have a role in TA functionalities’ provisioning