Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Activity-Aware Sensor Networks for Smart Environments

The efficient designs of Wireless Sensor Network protocols and intelligent Machine Learning algorithms, together have led to the advancements of various systems and applications for Smart Environments. By definition, Smart Environments are the typical physical worlds used in human daily life, those are seamlessly embedded with smart tiny devices equipped with sensors, actuators and computational elements. Since human user is a key component in Smart Environments, human motion activity patterns have key importance in building sensor network systems and applications for Smart Environments. Motivated by this, in this thesis my work is focused on human motion activity-aware sensor networks for Smart Environments. The main contributions of this thesis are in two important aspects: (i) Designing event activity context-aware sensor networks for efficient performance optimization as well as resource usage; and (ii) Using binary motion sensing sensor networks\u27 collective data for device-free real-time tracking of multiple users. Firstly, I describe the design of our proposed event activity context-aware sensor network protocols and system design for Smart Environments. The main motivation behind this work is as follows. A sensor network, unlike a traditional communication network, provides high degree of visibility into the environmental physical processes. Therefore its operation is driven by the activities in the environment. In long-term operations, these activities usually show certain patterns which can be learned and effectively utilized to optimize network design. In this thesis I have designed several novel protocols: (i) ActSee for activity-aware radio duty-cycling, (ii) EAR for activity-aware and energy balanced routing, and (iii) ActiSen complete working system with protocol suites for activity-aware sensing/ duty-cycling/ routing. Secondly, I have proposed and designed FindingHuMo (Finding Human Motion), a Machine Learning based real-time user tracking algorithm for Smart Environments using Sensor Networks. This work has been motivated by increasing adoption of sensor network enabled Ubiquitous Computing in key Smart Environment applications, like Smart Healthcare. Our proposed FindingHuMo protocol and system can perform device-free tracking of multiple (unknown and variable number of) users in the hallway environments, just from non-invasive and anonymous binary motion sensor data

Cross-layer energy optimisation of routing protocols in wireless sensor networks

Recent technological developments in embedded systems have led to the emergence of a new class of networks, known asWireless Sensor Networks (WSNs), where individual nodes cooperate wirelessly with each other with the goal of sensing and interacting with the environment.Many routing protocols have been developed tomeet the unique and challenging characteristics of WSNs (notably very limited power resources to sustain an expected lifetime of perhaps years, and the restricted computation, storage and communication capabilities of nodes that are nonetheless required to support large networks and diverse applications). No standards for routing have been developed yet for WSNs, nor has any protocol gained a dominant position among the research community. Routing has a significant influence on the overall WSN lifetime, and providing an energy efficient routing protocol remains an open problem. This thesis addresses the issue of designing WSN routing methods that feature energy efficiency. A common time reference across nodes is required in mostWSN applications. It is needed, for example, to time-stamp sensor samples and for duty cycling of nodes. Alsomany routing protocols require that nodes communicate according to some predefined schedule. However, independent distribution of the time information, without considering the routing algorithm schedule or network topology may lead to a failure of the synchronisation protocol. This was confirmed empirically, and was shown to result in loss of connectivity. This can be avoided by integrating the synchronisation service into the network layer with a so-called cross-layer approach. This approach introduces interactions between the layers of a conventional layered network stack, so that the routing layer may share information with other layers. I explore whether energy efficiency can be enhanced through the use of cross-layer optimisations and present three novel cross-layer routing algorithms. The first protocol, designed for hierarchical, cluster based networks and called CLEAR (Cross Layer Efficient Architecture for Routing), uses the routing algorithm to distribute time information which can be used for efficient duty cycling of nodes. The second method - called RISS (Routing Integrated Synchronization Service) - integrates time synchronization into the network layer and is designed to work well in flat, non-hierarchical network topologies. The third method - called SCALE (Smart Clustering Adapted LEACH) - addresses the influence of the intra-cluster topology on the energy dissipation of nodes. I also investigate the impact of the hop distance on network lifetime and propose a method of determining the optimal location of the relay node (the node through which data is routed in a two-hop network). I also address the problem of predicting the transition region (the zone separating the region where all packets can be received and that where no data can be received) and I describe a way of preventing the forwarding of packets through relays belonging in this transition region. I implemented and tested the performance of these solutions in simulations and also deployed these routing techniques on sensor nodes using TinyOS. I compared the average power consumption of the nodes and the precision of time synchronization with the corresponding parameters of a number of existing algorithms. All proposed schemes extend the network lifetime and due to their lightweight architecture they are very efficient on WSN nodes with constrained resources. Hence it is recommended that a cross-layer approach should be a feature of any routing algorithm for WSNs