Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Spectrum Sharing between Cooperative Relay and Ad-hoc Networks: Dynamic Transmissions under Computation and Signaling Limitations

This paper studies a spectrum sharing scenario between a cooperative relay network (CRN) and a nearby ad-hoc network. In particular, we consider a dynamic spectrum access and resource allocation problem of the CRN. Based on sensing and predicting the ad-hoc transmission behaviors, the ergodic traffic collision time between the CRN and ad-hoc network is minimized subject to an ergodic uplink throughput requirement for the CRN. We focus on real-time implementation of spectrum sharing policy under practical computation and signaling limitations. In our spectrum sharing policy, most computation tasks are accomplished off-line. Hence, little real-time calculation is required which fits the requirement of practical applications. Moreover, the signaling procedure and computation process are designed carefully to reduce the time delay between spectrum sensing and data transmission, which is crucial for enhancing the accuracy of traffic prediction and improving the performance of interference mitigation. The benefits of spectrum sensing and cooperative relay techniques are demonstrated by our numerical experiments.Comment: 5 pages, 3 figures, to appear in IEEE International Conference on Communications (ICC 2011

A survey of self organisation in future cellular networks

This article surveys the literature over the period of the last decade on the emerging field of self organisation as applied to wireless cellular communication networks. Self organisation has been extensively studied and applied in adhoc networks, wireless sensor networks and autonomic computer networks; however in the context of wireless cellular networks, this is the first attempt to put in perspective the various efforts in form of a tutorial/survey. We provide a comprehensive survey of the existing literature, projects and standards in self organising cellular networks. Additionally, we also aim to present a clear understanding of this active research area, identifying a clear taxonomy and guidelines for design of self organising mechanisms. We compare strength and weakness of existing solutions and highlight the key research areas for further development. This paper serves as a guide and a starting point for anyone willing to delve into research on self organisation in wireless cellular communication networks

Hybrid performance modelling of opportunistic networks

We demonstrate the modelling of opportunistic networks using the process algebra stochastic HYPE. Network traffic is modelled as continuous flows, contact between nodes in the network is modelled stochastically, and instantaneous decisions are modelled as discrete events. Our model describes a network of stationary video sensors with a mobile ferry which collects data from the sensors and delivers it to the base station. We consider different mobility models and different buffer sizes for the ferries. This case study illustrates the flexibility and expressive power of stochastic HYPE. We also discuss the software that enables us to describe stochastic HYPE models and simulate them.Comment: In Proceedings QAPL 2012, arXiv:1207.055

An Optimal Medium Access Control with Partial Observations for Sensor Networks

We consider medium access control (MAC) in multihop sensor networks, where only partial information about the shared medium is available to the transmitter. We model our setting as a queuing problem in which the service rate of a queue is a function of a partially observed Markov chain representing the available bandwidth, and in which the arrivals are controlled based on the partial observations so as to keep the system in a desirable mildly unstable regime. The optimal controller for this problem satisfies a separation property: we first compute a probability measure on the state space of the chain, namely the information state, then use this measure as the new state on which the control decisions are based. We give a formal description of the system considered and of its dynamics, we formalize and solve an optimal control problem, and we show numerical simulations to illustrate with concrete examples properties of the optimal control law. We show how the ergodic behavior of our queuing model is characterized by an invariant measure over all possible information states, and we construct that measure. Our results can be specifically applied for designing efficient and stable algorithms for medium access control in multiple-accessed systems, in particular for sensor networks