Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Robotic Wireless Sensor Networks

In this chapter, we present a literature survey of an emerging, cutting-edge, and multi-disciplinary field of research at the intersection of Robotics and Wireless Sensor Networks (WSN) which we refer to as Robotic Wireless Sensor Networks (RWSN). We define a RWSN as an autonomous networked multi-robot system that aims to achieve certain sensing goals while meeting and maintaining certain communication performance requirements, through cooperative control, learning and adaptation. While both of the component areas, i.e., Robotics and WSN, are very well-known and well-explored, there exist a whole set of new opportunities and research directions at the intersection of these two fields which are relatively or even completely unexplored. One such example would be the use of a set of robotic routers to set up a temporary communication path between a sender and a receiver that uses the controlled mobility to the advantage of packet routing. We find that there exist only a limited number of articles to be directly categorized as RWSN related works whereas there exist a range of articles in the robotics and the WSN literature that are also relevant to this new field of research. To connect the dots, we first identify the core problems and research trends related to RWSN such as connectivity, localization, routing, and robust flow of information. Next, we classify the existing research on RWSN as well as the relevant state-of-the-arts from robotics and WSN community according to the problems and trends identified in the first step. Lastly, we analyze what is missing in the existing literature, and identify topics that require more research attention in the future

Energy Efficient and Reliable Wireless Sensor Networks - An Extension to IEEE 802.15.4e

Collecting sensor data in industrial environments from up to some tenth of battery powered sensor nodes with sampling rates up to 100Hz requires energy aware protocols, which avoid collisions and long listening phases. The IEEE 802.15.4 standard focuses on energy aware wireless sensor networks (WSNs) and the Task Group 4e has published an amendment to fulfill up to 100 sensor value transmissions per second per sensor node (Low Latency Deterministic Network (LLDN) mode) to satisfy demands of factory automation. To improve the reliability of the data collection in the star topology of the LLDN mode, we propose a relay strategy, which can be performed within the LLDN schedule. Furthermore we propose an extension of the star topology to collect data from two-hop sensor nodes. The proposed Retransmission Mode enables power savings in the sensor node of more than 33%, while reducing the packet loss by up to 50%. To reach this performance, an optimum spatial distribution is necessary, which is discussed in detail

Wireless Backhaul Node Placement for Small Cell Networks

Small cells have been proposed as a vehicle for wireless networks to keep up with surging demand. Small cells come with a significant challenge of providing backhaul to transport data to(from) a gateway node in the core network. Fiber based backhaul offers the high rates needed to meet this requirement, but is costly and time-consuming to deploy, when not readily available. Wireless backhaul is an attractive option for small cells as it provides a less expensive and easy-to-deploy alternative to fiber. However, there are multitude of bands and features (e.g. LOS/NLOS, spatial multiplexing etc.) associated with wireless backhaul that need to be used intelligently for small cells. Candidate bands include: sub-6 GHz band that is useful in non-line-of-sight (NLOS) scenarios, microwave band (6-42 GHz) that is useful in point-to-point line-of-sight (LOS) scenarios, and millimeter wave bands (e.g. 60, 70 and 80 GHz) that are recently being commercially used in LOS scenarios. In many deployment topologies, it is advantageous to use aggregator nodes, located at the roof tops of tall buildings near small cells. These nodes can provide high data rate to multiple small cells in NLOS paths, sustain the same data rate to gateway nodes using LOS paths and take advantage of all available bands. This work performs the joint cost optimal aggregator node placement, power allocation, channel scheduling and routing to optimize the wireless backhaul network. We formulate mixed integer nonlinear programs (MINLP) to capture the different interference and multiplexing patterns at sub-6 GHz and microwave band. We solve the MINLP through linear relaxation and branch-and-bound algorithm and apply our algorithm in an example wireless backhaul network of downtown Manhattan.Comment: Invited paper at Conference on Information Science & Systems (CISS) 201

Adaptive Controller Placement for Wireless Sensor-Actuator Networks with Erasure Channels

Wireless sensor-actuator networks offer flexibility for control design. One novel element which may arise in networks with multiple nodes is that the role of some nodes does not need to be fixed. In particular, there is no need to pre-allocate which nodes assume controller functions and which ones merely relay data. We present a flexible architecture for networked control using multiple nodes connected in series over analog erasure channels without acknowledgments. The control architecture proposed adapts to changes in network conditions, by allowing the role played by individual nodes to depend upon transmission outcomes. We adopt stochastic models for transmission outcomes and characterize the distribution of controller location and the covariance of system states. Simulation results illustrate that the proposed architecture has the potential to give better performance than limiting control calculations to be carried out at a fixed node.Comment: 10 pages, 8 figures, to be published in Automatic

Deploy-As-You-Go Wireless Relay Placement: An Optimal Sequential Decision Approach using the Multi-Relay Channel Model

We use information theoretic achievable rate formulas for the multi-relay channel to study the problem of as-you-go deployment of relay nodes. The achievable rate formulas are for full-duplex radios at the relays and for decode-and-forward relaying. Deployment is done along the straight line joining a source node and a sink node at an unknown distance from the source. The problem is for a deployment agent to walk from the source to the sink, deploying relays as he walks, given that the distance to the sink is exponentially distributed with known mean. As a precursor, we apply the multi-relay channel achievable rate formula to obtain the optimal power allocation to relays placed along a line, at fixed locations. This permits us to obtain the optimal placement of a given number of nodes when the distance between the source and sink is given. Numerical work suggests that, at low attenuation, the relays are mostly clustered near the source in order to be able to cooperate, whereas at high attenuation they are uniformly placed and work as repeaters. We also prove that the effect of path-loss can be entirely mitigated if a large enough number of relays are placed uniformly between the source and the sink. The structure of the optimal power allocation for a given placement of the nodes, then motivates us to formulate the problem of as-you-go placement of relays along a line of exponentially distributed length, and with the exponential path-loss model, so as to minimize a cost function that is additive over hops. The hop cost trades off a capacity limiting term, motivated from the optimal power allocation solution, against the cost of adding a relay node. We formulate the problem as a total cost Markov decision process, establish results for the value function, and provide insights into the placement policy and the performance of the deployed network via numerical exploration.Comment: 21 pages. arXiv admin note: substantial text overlap with arXiv:1204.432