Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Lyapunov Approach to Consensus Problems

This paper investigates the weighted-averaging dynamic for unconstrained and constrained consensus problems. Through the use of a suitably defined adjoint dynamic, quadratic Lyapunov comparison functions are constructed to analyze the behavior of weighted-averaging dynamic. As a result, new convergence rate results are obtained that capture the graph structure in a novel way. In particular, the exponential convergence rate is established for unconstrained consensus with the exponent of the order of $1-O(1/(m\log_2m))$. Also, the exponential convergence rate is established for constrained consensus, which extends the existing results limited to the use of doubly stochastic weight matrices

Differential Inequalities in Multi-Agent Coordination and Opinion Dynamics Modeling

Distributed algorithms of multi-agent coordination have attracted substantial attention from the research community; the simplest and most thoroughly studied of them are consensus protocols in the form of differential or difference equations over general time-varying weighted graphs. These graphs are usually characterized algebraically by their associated Laplacian matrices. Network algorithms with similar algebraic graph theoretic structures, called being of Laplacian-type in this paper, also arise in other related multi-agent control problems, such as aggregation and containment control, target surrounding, distributed optimization and modeling of opinion evolution in social groups. In spite of their similarities, each of such algorithms has often been studied using separate mathematical techniques. In this paper, a novel approach is offered, allowing a unified and elegant way to examine many Laplacian-type algorithms for multi-agent coordination. This approach is based on the analysis of some differential or difference inequalities that have to be satisfied by the some "outputs" of the agents (e.g. the distances to the desired set in aggregation problems). Although such inequalities may have many unbounded solutions, under natural graphic connectivity conditions all their bounded solutions converge (and even reach consensus), entailing the convergence of the corresponding distributed algorithms. In the theory of differential equations the absence of bounded non-convergent solutions is referred to as the equation's dichotomy. In this paper, we establish the dichotomy criteria of Laplacian-type differential and difference inequalities and show that these criteria enable one to extend a number of recent results, concerned with Laplacian-type algorithms for multi-agent coordination and modeling opinion formation in social groups.Comment: accepted to Automatic