Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

A Survey on Delay-Aware Resource Control for Wireless Systems --- Large Deviation Theory, Stochastic Lyapunov Drift and Distributed Stochastic Learning

In this tutorial paper, a comprehensive survey is given on several major systematic approaches in dealing with delay-aware control problems, namely the equivalent rate constraint approach, the Lyapunov stability drift approach and the approximate Markov Decision Process (MDP) approach using stochastic learning. These approaches essentially embrace most of the existing literature regarding delay-aware resource control in wireless systems. They have their relative pros and cons in terms of performance, complexity and implementation issues. For each of the approaches, the problem setup, the general solution and the design methodology are discussed. Applications of these approaches to delay-aware resource allocation are illustrated with examples in single-hop wireless networks. Furthermore, recent results regarding delay-aware multi-hop routing designs in general multi-hop networks are elaborated. Finally, the delay performance of the various approaches are compared through simulations using an example of the uplink OFDMA systems.Comment: 58 pages, 8 figures; IEEE Transactions on Information Theory, 201

Cross-layer design of multi-hop wireless networks

MULTI -hop wireless networks are usually defined as a collection of nodes equipped with radio transmitters, which not only have the capability to communicate each other in a multi-hop fashion, but also to route each others’ data packets. The distributed nature of such networks makes them suitable for a variety of applications where there are no assumed reliable central entities, or controllers, and may significantly improve the scalability issues of conventional single-hop wireless networks. This Ph.D. dissertation mainly investigates two aspects of the research issues related to the efficient multi-hop wireless networks design, namely: (a) network protocols and (b) network management, both in cross-layer design paradigms to ensure the notion of service quality, such as quality of service (QoS) in wireless mesh networks (WMNs) for backhaul applications and quality of information (QoI) in wireless sensor networks (WSNs) for sensing tasks. Throughout the presentation of this Ph.D. dissertation, different network settings are used as illustrative examples, however the proposed algorithms, methodologies, protocols, and models are not restricted in the considered networks, but rather have wide applicability. First, this dissertation proposes a cross-layer design framework integrating a distributed proportional-fair scheduler and a QoS routing algorithm, while using WMNs as an illustrative example. The proposed approach has significant performance gain compared with other network protocols. Second, this dissertation proposes a generic admission control methodology for any packet network, wired and wireless, by modeling the network as a black box, and using a generic mathematical 0. Abstract 3 function and Taylor expansion to capture the admission impact. Third, this dissertation further enhances the previous designs by proposing a negotiation process, to bridge the applications’ service quality demands and the resource management, while using WSNs as an illustrative example. This approach allows the negotiation among different service classes and WSN resource allocations to reach the optimal operational status. Finally, the guarantees of the service quality are extended to the environment of multiple, disconnected, mobile subnetworks, where the question of how to maintain communications using dynamically controlled, unmanned data ferries is investigated

Achievable Secrecy Rates of an Energy Harvesting Device

The secrecy rate represents the amount of information per unit time that can be securely sent on a communication link. In this work, we investigate the achievable secrecy rates in an energy harvesting communication system composed of a transmitter, a receiver and a malicious eavesdropper. In particular, because of the energy constraints and the channel conditions, it is important to understand when a device should transmit and to optimize how much power should be used in order to improve security. Both full knowledge and partial knowledge of the channel are considered under a Nakagami fading scenario. We show that high secrecy rates can be obtained only with power and coding rate adaptation. Moreover, we highlight the importance of optimally dividing the transmission power in the frequency domain, and note that the optimal scheme provides high gains in secrecy rate over the uniform power splitting case. Analytically, we explain how to find the optimal policy and prove some of its properties. In our numerical evaluation, we discuss how the maximum achievable secrecy rate changes according to the various system parameters. Furthermore, we discuss the effects of a finite battery on the system performance and note that, in order to achieve high secrecy rates, it is not necessary to use very large batteries.Comment: Accepted for publication in IEEE Journal on Selected Areas in Communications (Mar. 2016

Optimal Compression and Transmission Rate Control for Node-Lifetime Maximization

We consider a system that is composed of an energy constrained sensor node and a sink node, and devise optimal data compression and transmission policies with an objective to prolong the lifetime of the sensor node. While applying compression before transmission reduces the energy consumption of transmitting the sensed data, blindly applying too much compression may even exceed the cost of transmitting raw data, thereby losing its purpose. Hence, it is important to investigate the trade-off between data compression and transmission energy costs. In this paper, we study the joint optimal compression-transmission design in three scenarios which differ in terms of the available channel information at the sensor node, and cover a wide range of practical situations. We formulate and solve joint optimization problems aiming to maximize the lifetime of the sensor node whilst satisfying specific delay and bit error rate (BER) constraints. Our results show that a jointly optimized compression-transmission policy achieves significantly longer lifetime (90% to 2000%) as compared to optimizing transmission only without compression. Importantly, this performance advantage is most profound when the delay constraint is stringent, which demonstrates its suitability for low latency communication in future wireless networks.Comment: accepted for publication in IEEE Transactions on Wireless Communicaiton