Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Decentralized Delay Optimal Control for Interference Networks with Limited Renewable Energy Storage

In this paper, we consider delay minimization for interference networks with renewable energy source, where the transmission power of a node comes from both the conventional utility power (AC power) and the renewable energy source. We assume the transmission power of each node is a function of the local channel state, local data queue state and local energy queue state only. In turn, we consider two delay optimization formulations, namely the decentralized partially observable Markov decision process (DEC-POMDP) and Non-cooperative partially observable stochastic game (POSG). In DEC-POMDP formulation, we derive a decentralized online learning algorithm to determine the control actions and Lagrangian multipliers (LMs) simultaneously, based on the policy gradient approach. Under some mild technical conditions, the proposed decentralized policy gradient algorithm converges almost surely to a local optimal solution. On the other hand, in the non-cooperative POSG formulation, the transmitter nodes are non-cooperative. We extend the decentralized policy gradient solution and establish the technical proof for almost-sure convergence of the learning algorithms. In both cases, the solutions are very robust to model variations. Finally, the delay performance of the proposed solutions are compared with conventional baseline schemes for interference networks and it is illustrated that substantial delay performance gain and energy savings can be achieved

Hierarchical Parallelisation of Functional Renormalisation Group Calculations -- hp-fRG

The functional renormalisation group (fRG) has evolved into a versatile tool in condensed matter theory for studying important aspects of correlated electron systems. Practical applications of the method often involve a high numerical effort, motivating the question in how far High Performance Computing (HPC) can leverage the approach. In this work we report on a multi-level parallelisation of the underlying computational machinery and show that this can speed up the code by several orders of magnitude. This in turn can extend the applicability of the method to otherwise inaccessible cases. We exploit three levels of parallelisation: Distributed computing by means of Message Passing (MPI), shared-memory computing using OpenMP, and vectorisation by means of SIMD units (single-instruction-multiple-data). Results are provided for two distinct High Performance Computing (HPC) platforms, namely the IBM-based BlueGene/Q system JUQUEEN and an Intel Sandy-Bridge-based development cluster. We discuss how certain issues and obstacles were overcome in the course of adapting the code. Most importantly, we conclude that this vast improvement can actually be accomplished by introducing only moderate changes to the code, such that this strategy may serve as a guideline for other researcher to likewise improve the efficiency of their codes

A Middleware for the Internet of Things

The Internet of Things (IoT) connects everyday objects including a vast array of sensors, actuators, and smart devices, referred to as things to the Internet, in an intelligent and pervasive fashion. This connectivity gives rise to the possibility of using the tracking capabilities of things to impinge on the location privacy of users. Most of the existing management and location privacy protection solutions do not consider the low-cost and low-power requirements of things, or, they do not account for the heterogeneity, scalability, or autonomy of communications supported in the IoT. Moreover, these traditional solutions do not consider the case where a user wishes to control the granularity of the disclosed information based on the context of their use (e.g. based on the time or the current location of the user). To fill this gap, a middleware, referred to as the Internet of Things Management Platform (IoT-MP) is proposed in this paper.Comment: 20 pages, International Journal of Computer Networks & Communications (IJCNC) Vol.8, No.2, March 201

Energy and Spectral Efficient Wireless Communications

Energy and spectrum are two precious commodities for wireless communications. How to improve the energy and spectrum efficiency has become two critical issues for the designs of wireless communication systems. This dissertation is devoted to the development of energy and spectral efficient wireless communications. The developed techniques can be applied to a wide range of wireless communication systems, such as wireless sensor network (WSN) designed for structure health monitoring (SHM), medium access control (MAC) for multi-user systems, and cooperative spectrum sensing in cognitive radio systems. First, to improve the energy efficiency in SHM WSN, a new ultra low power (ULP) WSN is proposed to monitor the vibration properties of structures such as buildings, bridges, and the wings and bodies of aircrafts. The new scheme integrates energy harvesting, data sensing, and wireless communication into a unified process, and it achieves significant energy savings compared to existing WSNs. Second, a cross-layer collision tolerant (CT) MAC scheme is proposed to improve energy and spectral efficiency in a multi-user system with shared medium. When two users transmit simultaneously over a shared medium, a collision happens at the receiver. Conventional MAC schemes will discard the collided signals, which result in a waste of the precious energy and spectrum resources. In our proposed CT-MAC scheme, each user transmits multiple weighted replicas of a packet at randomly selected data slots in a frame, and the indices of the selected slots are transmitted in a special collision-free position slot at the beginning of each frame. Collisions of the data slots in the MAC layer are resolved by using multiuser detection (MUD) in the PHY layer. Compared to existing schemes, the proposed CT-MAC scheme can support more simultaneous users with a higher throughput. Third, a new cooperative spectrum sensing scheme is proposed to improve the energy and spectral efficiency of a cognitive radio network. A new Slepian-Wolf coded cooperation scheme is proposed for a cognitive radio network with two secondary users (SUs) performing cooperative spectrum sensing through a fusion center (FC). The proposed scheme can achieve significant performance gains compared to existing schemes