Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Minimizing Expected Cost Under Hard Boolean Constraints, with Applications to Quantitative Synthesis

In Boolean synthesis, we are given an LTL specification, and the goal is to construct a transducer that realizes it against an adversarial environment. Often, a specification contains both Boolean requirements that should be satisfied against an adversarial environment, and multi-valued components that refer to the quality of the satisfaction and whose expected cost we would like to minimize with respect to a probabilistic environment. In this work we study, for the first time, mean-payoff games in which the system aims at minimizing the expected cost against a probabilistic environment, while surely satisfying an $\omega$-regular condition against an adversarial environment. We consider the case the $\omega$-regular condition is given as a parity objective or by an LTL formula. We show that in general, optimal strategies need not exist, and moreover, the limit value cannot be approximated by finite-memory strategies. We thus focus on computing the limit-value, and give tight complexity bounds for synthesizing $\epsilon$-optimal strategies for both finite-memory and infinite-memory strategies. We show that our game naturally arises in various contexts of synthesis with Boolean and multi-valued objectives. Beyond direct applications, in synthesis with costs and rewards to certain behaviors, it allows us to compute the minimal sensing cost of $\omega$-regular specifications -- a measure of quality in which we look for a transducer that minimizes the expected number of signals that are read from the input

Plasticity facilitates sustainable growth in the commons

In the commons, communities whose growth depends on public goods, individuals often rely on surprisingly simple strategies, or heuristics, to decide whether to contribute to the common good (at risk of exploitation by free-riders). Although this appears a limitation, here we show how four heuristics lead to sustainable growth by exploiting specific environmental constraints. The two simplest ones --contribute permanently or switch stochastically between contributing or not-- are first shown to bring sustainability when the public good efficiently promotes growth. If efficiency declines and the commons is structured in small groups, the most effective strategy resides in contributing only when a majority of individuals are also contributors. In contrast, when group size becomes large, the most effective behavior follows a minimal-effort rule: contribute only when it is strictly necessary. Both plastic strategies are observed in natural systems what presents them as fundamental social motifs to successfully manage sustainability

Mean-Field-Type Games in Engineering

A mean-field-type game is a game in which the instantaneous payoffs and/or the state dynamics functions involve not only the state and the action profile but also the joint distributions of state-action pairs. This article presents some engineering applications of mean-field-type games including road traffic networks, multi-level building evacuation, millimeter wave wireless communications, distributed power networks, virus spread over networks, virtual machine resource management in cloud networks, synchronization of oscillators, energy-efficient buildings, online meeting and mobile crowdsensing.Comment: 84 pages, 24 figures, 183 references. to appear in AIMS 201