Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Distributed state estimation for uncertain Markov-type sensor networks with mode-dependent distributed delays

This the post-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2012 John Wiley & Sons, Ltd.In this paper, the distributed state estimation problem is investigated for a class of sensor networks described by uncertain discrete-time dynamical systems with Markovian jumping parameters and distributed time-delays. The sensor network consists of sensor nodes characterized by a directed graph with a nonnegative adjacency matrix that specifies the interconnection topology (or the distribution in the space) of the network. Both the parameters of the target plant and the sensor measurements are subject to the switches from one mode to another at different times according to a Markov chain. The parameter uncertainties are norm-bounded that enter into both the plant system as well as the network outputs. Furthermore, the distributed time-delays are considered, which are also dependent on the Markovian jumping mode. Through the measurements from a small fraction of the sensors, this paper aims to design state estimators that allow the nodes of the sensor network to track the states of the plant in a distributed way. It is verified that such state estimators do exist if a set of matrix inequalities is solvable. A numerical example is provided to demonstrate the effectiveness of the designed distributed state estimators.This work was supported in part by the Royal Society of the U.K., the National Natural Science Foundation of China under Grants 60804028 and 61028008, the Specialized Research Fund for the Doctoral Program of Higher Education for New Teachers in China under Grant 200802861044, the Teaching and Research Fund for Excellent Young Teachers at Southeast University of China, the International Science and Technology Cooperation Project of China under Grant No. 2009DFA32050, and the Alexander von Humboldt Foundation of Germany

Practical applications of probabilistic model checking to communication protocols

Probabilistic model checking is a formal verification technique for the analysis of systems that exhibit stochastic behaviour. It has been successfully employed in an extremely wide array of application domains including, for example, communication and multimedia protocols, security and power management. In this chapter we focus on the applicability of these techniques to the analysis of communication protocols. An analysis of the performance of such systems must successfully incorporate several crucial aspects, including concurrency between multiple components, real-time constraints and randomisation. Probabilistic model checking, in particular using probabilistic timed automata, is well suited to such an analysis. We provide an overview of this area, with emphasis on an industrially relevant case study: the IEEE 802.3 (CSMA/CD) protocol. We also discuss two contrasting approaches to the implementation of probabilistic model checking, namely those based on numerical computation and those based on discrete-event simulation. Using results from the two tools PRISM and APMC, we summarise the advantages, disadvantages and trade-offs associated with these techniques

Design and analysis of a beacon-less routing protocol for large volume content dissemination in vehicular ad hoc networks

Largevolumecontentdisseminationispursuedbythegrowingnumberofhighquality applications for Vehicular Ad hoc NETworks(VANETs), e.g., the live road surveillance service and the video-based overtaking assistant service. For the highly dynamical vehicular network topology, beacon-less routing protocols have been proven to be efficient in achieving a balance between the system performance and the control overhead. However, to the authors’ best knowledge, the routing design for large volume content has not been well considered in the previous work, which will introduce new challenges, e.g., the enhanced connectivity requirement for a radio link. In this paper, a link Lifetime-aware Beacon-less Routing Protocol (LBRP) is designed for large volume content delivery in VANETs. Each vehicle makes the forwarding decision based on the message header information and its current state, including the speed and position information. A semi-Markov process analytical model is proposed to evaluate the expected delay in constructing one routing path for LBRP. Simulations show that the proposed LBRP scheme outperforms the traditional dissemination protocols in providing a low end-to-end delay. The analytical model is shown to exhibit a good match on the delay estimation with Monte Carlo simulations, as well