Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Distributed Optimization in Energy Harvesting Sensor Networks with Dynamic In-network Data Processing

Energy Harvesting Wireless Sensor Networks (EH- WSNs) have been attracting increasing interest in recent years. Most current EH-WSN approaches focus on sensing and net- working algorithm design, and therefore only consider the energy consumed by sensors and wireless transceivers for sensing and data transmissions respectively. In this paper, we incorporate CPU-intensive edge operations that constitute in-network data processing (e.g. data aggregation/fusion/compression) with sens- ing and networking; to jointly optimize their performance, while ensuring sustainable network operation (i.e. no sensor node runs out of energy). Based on realistic energy and network models, we formulate a stochastic optimization problem, and propose a lightweight on-line algorithm, namely Recycling Wasted Energy (RWE), to solve it. Through rigorous theoretical analysis, we prove that RWE achieves asymptotical optimality, bounded data queue size, and sustainable network operation. We implement RWE on a popular IoT operating system, Contiki OS, and eval- uate its performance using both real-world experiments based on the FIT IoT-LAB testbed, and extensive trace-driven simulations using Cooja. The evaluation results verify our theoretical analysis, and demonstrate that RWE can recycle more than 90% wasted energy caused by battery overflow, and achieve around 300% network utility gain in practical EH-WSNs

Feedback Allocation For OFDMA Systems With Slow Frequency-domain Scheduling

We study the problem of allocating limited feedback resources across multiple users in an orthogonal-frequency-division-multiple-access downlink system with slow frequency-domain scheduling. Many flavors of slow frequency-domain scheduling (e.g., persistent scheduling, semi-persistent scheduling), that adapt user-sub-band assignments on a slower time-scale, are being considered in standards such as 3GPP Long-Term Evolution. In this paper, we develop a feedback allocation algorithm that operates in conjunction with any arbitrary slow frequency-domain scheduler with the goal of improving the throughput of the system. Given a user-sub-band assignment chosen by the scheduler, the feedback allocation algorithm involves solving a weighted sum-rate maximization at each (slow) scheduling instant. We first develop an optimal dynamic-programming-based algorithm to solve the feedback allocation problem with pseudo-polynomial complexity in the number of users and in the total feedback bit budget. We then propose two approximation algorithms with complexity further reduced, for scenarios where the problem exhibits additional structure.Comment: Accepted to IEEE Transactions on Signal Processin

Cross-layer design of multi-hop wireless networks

MULTI -hop wireless networks are usually defined as a collection of nodes equipped with radio transmitters, which not only have the capability to communicate each other in a multi-hop fashion, but also to route each others’ data packets. The distributed nature of such networks makes them suitable for a variety of applications where there are no assumed reliable central entities, or controllers, and may significantly improve the scalability issues of conventional single-hop wireless networks. This Ph.D. dissertation mainly investigates two aspects of the research issues related to the efficient multi-hop wireless networks design, namely: (a) network protocols and (b) network management, both in cross-layer design paradigms to ensure the notion of service quality, such as quality of service (QoS) in wireless mesh networks (WMNs) for backhaul applications and quality of information (QoI) in wireless sensor networks (WSNs) for sensing tasks. Throughout the presentation of this Ph.D. dissertation, different network settings are used as illustrative examples, however the proposed algorithms, methodologies, protocols, and models are not restricted in the considered networks, but rather have wide applicability. First, this dissertation proposes a cross-layer design framework integrating a distributed proportional-fair scheduler and a QoS routing algorithm, while using WMNs as an illustrative example. The proposed approach has significant performance gain compared with other network protocols. Second, this dissertation proposes a generic admission control methodology for any packet network, wired and wireless, by modeling the network as a black box, and using a generic mathematical 0. Abstract 3 function and Taylor expansion to capture the admission impact. Third, this dissertation further enhances the previous designs by proposing a negotiation process, to bridge the applications’ service quality demands and the resource management, while using WSNs as an illustrative example. This approach allows the negotiation among different service classes and WSN resource allocations to reach the optimal operational status. Finally, the guarantees of the service quality are extended to the environment of multiple, disconnected, mobile subnetworks, where the question of how to maintain communications using dynamically controlled, unmanned data ferries is investigated

A Practical Cooperative Multicell MIMO-OFDMA Network Based on Rank Coordination

An important challenge of wireless networks is to boost the cell edge performance and enable multi-stream transmissions to cell edge users. Interference mitigation techniques relying on multiple antennas and coordination among cells are nowadays heavily studied in the literature. Typical strategies in OFDMA networks include coordinated scheduling, beamforming and power control. In this paper, we propose a novel and practical type of coordination for OFDMA downlink networks relying on multiple antennas at the transmitter and the receiver. The transmission ranks, i.e.\ the number of transmitted streams, and the user scheduling in all cells are jointly optimized in order to maximize a network utility function accounting for fairness among users. A distributed coordinated scheduler motivated by an interference pricing mechanism and relying on a master-slave architecture is introduced. The proposed scheme is operated based on the user report of a recommended rank for the interfering cells accounting for the receiver interference suppression capability. It incurs a very low feedback and backhaul overhead and enables efficient link adaptation. It is moreover robust to channel measurement errors and applicable to both open-loop and closed-loop MIMO operations. A 20% cell edge performance gain over uncoordinated LTE-A system is shown through system level simulations.Comment: IEEE Transactions or Wireless Communications, Accepted for Publicatio