Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Predictive Duty Cycle Adaptation for Wireless Camera Networks

Wireless sensor networks (WSN) typically employ dynamic duty cycle schemes to efficiently handle different patterns of communication traffic in the network. However, existing duty cycling approaches are not suitable for event-driven WSN, in particular, camera-based networks designed to track humans and objects. A characteristic feature of such networks is the spatially-correlated bursty traffic that occurs in the vicinity of potentially highly mobile objects. In this paper, we propose a concept of indirect sensing in the MAC layer of a wireless camera network and an active duty cycle adaptation scheme based on Kalman filter that continuously predicts and updates the location of the object that triggers bursty communication traffic in the network. This prediction allows the camera nodes to alter their communication protocol parameters prior to the actual increase in the communication traffic. Our simulations demonstrate that our active adaptation strategy outperforms TMAC not only in terms of energy efficiency and communication latency, but also in terms of TIBPEA, a QoS metric for event-driven WSN

Let the Tree Bloom: Scalable Opportunistic Routing with ORPL

Routing in battery-operated wireless networks is challenging, posing a tradeoff between energy and latency. Previous work has shown that opportunistic routing can achieve low-latency data collection in duty-cycled networks. However, applications are now considered where nodes are not only periodic data sources, but rather addressable end points generating traffic with arbitrary patterns. We present ORPL, an opportunistic routing protocol that supports any-to-any, on-demand traffic. ORPL builds upon RPL, the standard protocol for low-power IPv6 networks. By combining RPL's tree-like topology with opportunistic routing, ORPL forwards data to any destination based on the mere knowledge of the nodes' sub-tree. We use bitmaps and Bloom filters to represent and propagate this information in a space-efficient way, making ORPL scale to large networks of addressable nodes. Our results in a 135-node testbed show that ORPL outperforms a number of state-of-the-art solutions including RPL and CTP, conciliating a sub-second latency and a sub-percent duty cycle. ORPL also increases robustness and scalability, addressing the whole network reliably through a 64-byte Bloom filter, where RPL needs kilobytes of routing tables for the same task

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial

Infective flooding in low-duty-cycle networks, properties and bounds

Flooding information is an important function in many networking applications. In some networks, as wireless sensor networks or some ad-hoc networks it is so essential as to dominate the performance of the entire system. Exploiting some recent results based on the distributed computation of the eigenvector centrality of nodes in the network graph and classical dynamic diffusion models on graphs, this paper derives a novel theoretical framework for efficient resource allocation to flood information in mesh networks with low duty-cycling without the need to build a distribution tree or any other distribution overlay. Furthermore, the method requires only local computations based on each node neighborhood. The model provides lower and upper stochastic bounds on the flooding delay averages on all possible sources with high probability. We show that the lower bound is very close to the theoretical optimum. A simulation-based implementation allows the study of specific topologies and graph models as well as scheduling heuristics and packet losses. Simulation experiments show that simple protocols based on our resource allocation strategy can easily achieve results that are very close to the theoretical minimum obtained building optimized overlays on the network

Brief announcement: Game theoretical approach for energy-delay balancing in distributed duty-cycled MAC protocols of wireless networks

Optimizing energy consumption and end-to-end (e2e) packet delay in energy constrained distributed wireless networks is a conflicting multi-objective optimization problem. This paper investigates this trade-off from a game-theoretic perspective, where the two optimization objectives are considered as virtual game players that attempt to optimize their utility values. The cost model of each player is mapped through a generalized optimization framework onto protocol specific MAC parameters. A cooperative game is then defined, in which the Nash Bargaining solution assures the balance between energy consumption and e2e packet delay. For illustration, this formulation is applied to three state-of-the-art wireless sensor network MAC protocols; X-MAC, DMAC, and LMAC as representatives of preamble sampling, slotted contention-based, and frame-based MAC categories, respectively. The paper shows the effectiveness of such framework in optimizing protocol parameters for achieving a fair energy-delay performance trade-off, under the application requirements in terms of initial energy budget and maximum e2e packet delay. The proposed framework is scalable with the increase in the number of nodes, as the players represent the optimization metrics instead of nodes.Postprint (author’s final draft

Game theory framework for MAC parameter optimization in energy-delay constrained sensor networks

Optimizing energy consumption and end-to-end (e2e) packet delay in energy-constrained, delay-sensitive wireless sensor networks is a conflicting multiobjective optimization problem. We investigate the problem from a game theory perspective, where the two optimization objectives are considered as game players. The cost model of each player is mapped through a generalized optimization framework onto protocol-specific MAC parameters. From the optimization framework, a game is first defined by the Nash bargaining solution (NBS) to assure energy consumption and e2e delay balancing. Secondy, the Kalai-Smorodinsky bargaining solution (KSBS) is used to find an equal proportion of gain between players. Both methods offer a bargaining solution to the duty-cycle MAC protocol under different axioms. As a result, given the two performance requirements (i.e., the maximum latency tolerated by the application and the initial energy budget of nodes), the proposed framework allows to set tunable system parameters to reach a fair equilibrium point that dually minimizes the system latency and energy consumption. For illustration, this formulation is applied to six state-of-the-art wireless sensor network (WSN) MAC protocols: B-MAC, X-MAC, RI-MAC, SMAC, DMAC, and LMAC. The article shows the effectiveness and scalability of such a framework in optimizing protocol parameters that achieve a fair energy-delay performance trade-off under the application requirements

MAC protocols for low-latency and energy-efficient WSN applications

Most of medium access control (MAC) protocols proposed for wireless sensor networks (WSN) are targeted only for single main objective, the energy efficiency. Other critical parameters such as low-latency, adaptivity to traffic conditions, scalability, system fairness, and bandwidth utilization are mostly overleaped or dealt as secondary objectives. The demand to address those issues increases with the growing interest in cheap, low-power, low- distance, and embedded WSNs. In this report, along with other vital parameters, we discuss suitability and limitations of different WSN MAC protocols for time critical and energy-efficient applications. As an example, we discuss the working of IEEE 802.15.4 in detail, explore its limitations, and derive efficient application-specific network parameter settings for time, energy, and bandwidth critical applications. Eventually, a new WSN MAC protocol Asynchronous Real-time Energy-efficient and Adaptive MAC (AREA-MAC) is proposed, which is intended to deal efficiently with time critical applications, and at the same time, to provide a better trade-off between other vital parameters, such as energy-efficiency, system fairness, throughput, scalability, and adaptivity to traffic conditions. On the other hand, two different optimization problems have been formulated using application-based traffic generating scenario to minimize network latency and maximize its lifetime