Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

A Low Power Architectural Framework for Automated Surveillance System with Low Bit Rate Transmission

Abstract The changed security scenario of the modern time has necessitated increased and sophisticated vigilance of the countries' borders. The technological challenges involved in accomplishing such feat of automated security system are many and require research at the components-and-algorithms as well as the architectural levels.  This paper proposes an architectural framework for automated video surveillance comprising a network of sensors and closed circuit television cameras as well as proposing algorithmic/component research of software and hardware for the core functioning of the framework, such as: communication protocols, object detection, data-integration, object identification, object tracking, video compression, threat identification, and alarm generation. In this paper, we are addressing some general topological and routing features that would be adopted in our system. There are two types of data with regard to data communication – video stream and object detection. The network is broken down into several disjoint, almost equal zones. A zone have one or more one cluster. A zone manager is chosen among the cluster heads depending on their relative residual energies. There are several levels of control that could be implemented with this arrangement with localized decision made, to get distributed effect at all levels. A cell tracks each target in its zone. If the target moves out of the range of a cell, the cell manager will send the target description to estimated next cell. The next cell starts tracking the target. If the estimated cell is wrongly chosen, corrections will be made by the cluster heads to get the new target-tracking. We also propose bitrate reduction algorithms to accommodate the limited bandwidth. One of the main feature of this paper is introducing a Low-Power Low-Bit rate video compression algorithm to accommodate the low power requirements at sensor nodes, and the low bit rate requirement for the communication protocol. We proposed two algorithms the ALBR and LPHSME. ALBR is addressing low bit rate required for sensors network with limited bandwidth which achieves a reduction in Average number of bits per Iframe by approximately 60% in case of low motion video sequences and 53% in case of fast motion video sequences . LPHSME addresses low power requirements of multi sensor network that has limited power batteries. The performance of the proposed LPHSME algorithm versus full search and three step search indicates  a reduction in motion estimation time by approximately 89% in case of low motion video sequences (e.g., Claire ) and 84% in case of fast motion video sequences. The reduced complexity of  LPHSME results in low power requirements

Survey of Inter-satellite Communication for Small Satellite Systems: Physical Layer to Network Layer View

Small satellite systems enable whole new class of missions for navigation, communications, remote sensing and scientific research for both civilian and military purposes. As individual spacecraft are limited by the size, mass and power constraints, mass-produced small satellites in large constellations or clusters could be useful in many science missions such as gravity mapping, tracking of forest fires, finding water resources, etc. Constellation of satellites provide improved spatial and temporal resolution of the target. Small satellite constellations contribute innovative applications by replacing a single asset with several very capable spacecraft which opens the door to new applications. With increasing levels of autonomy, there will be a need for remote communication networks to enable communication between spacecraft. These space based networks will need to configure and maintain dynamic routes, manage intermediate nodes, and reconfigure themselves to achieve mission objectives. Hence, inter-satellite communication is a key aspect when satellites fly in formation. In this paper, we present the various researches being conducted in the small satellite community for implementing inter-satellite communications based on the Open System Interconnection (OSI) model. This paper also reviews the various design parameters applicable to the first three layers of the OSI model, i.e., physical, data link and network layer. Based on the survey, we also present a comprehensive list of design parameters useful for achieving inter-satellite communications for multiple small satellite missions. Specific topics include proposed solutions for some of the challenges faced by small satellite systems, enabling operations using a network of small satellites, and some examples of small satellite missions involving formation flying aspects.Comment: 51 pages, 21 Figures, 11 Tables, accepted in IEEE Communications Surveys and Tutorial

Using Hybrid Angle/Distance Information for Distributed Topology Control in Vehicular Sensor Networks

In a vehicular sensor network (VSN), the key design issue is how to organize vehicles effectively, such that the local network topology can be stabilized quickly. In this work, each vehicle with on-board sensors can be considered as a local controller associated with a group of communication members. In order to balance the load among the nodes and govern the local topology change, a group formation scheme using localized criteria is implemented. The proposed distributed topology control method focuses on reducing the rate of group member change and avoiding the unnecessary information exchange. Two major phases are sequentially applied to choose the group members of each vehicle using hybrid angle/distance information. The operation of Phase I is based on the concept of the cone-based method, which can select the desired vehicles quickly. Afterwards, the proposed time-slot method is further applied to stabilize the network topology. Given the network structure in Phase I, a routing scheme is presented in Phase II. The network behaviors are explored through simulation and analysis in a variety of scenarios. The results show that the proposed mechanism is a scalable and effective control framework for VSNs

Estimation and stability of nonlinear control systems under intermittent information with applications to multi-agent robotics

This dissertation investigates the role of intermittent information in estimation and control problems and applies the obtained results to multi-agent tasks in robotics. First, we develop a stochastic hybrid model of mobile networks able to capture a large variety of heterogeneous multi-agent problems and phenomena. This model is applied to a case study where a heterogeneous mobile sensor network cooperatively detects and tracks mobile targets based on intermittent observations. When these observations form a satisfactory target trajectory, a mobile sensor is switched to the pursuit mode and deployed to capture the target. The cost of operating the sensors is determined from the geometric properties of the network, environment and probability of target detection. The above case study is motivated by the Marco Polo game played by children in swimming pools. Second, we develop adaptive sampling of targets positions in order to minimize energy consumption, while satisfying performance guarantees such as increased probability of detection over time, and no-escape conditions. A parsimonious predictor-corrector tracking filter, that uses geometrical properties of targets\u27 tracks to estimate their positions using imperfect and intermittent measurements, is presented. It is shown that this filter requires substantially less information and processing power than the Unscented Kalman Filter and Sampling Importance Resampling Particle Filter, while providing comparable estimation performance in the presence of intermittent information. Third, we investigate stability of nonlinear control systems under intermittent information. We replace the traditional periodic paradigm, where the up-to-date information is transmitted and control laws are executed in a periodic fashion, with the event-triggered paradigm. Building on the small gain theorem, we develop input-output triggered control algorithms yielding stable closed-loop systems. In other words, based on the currently available (but outdated) measurements of the outputs and external inputs of a plant, a mechanism triggering when to obtain new measurements and update the control inputs is provided. Depending on the noise environment, the developed algorithm yields stable, asymptotically stable, and Lp-stable (with bias) closed-loop systems. Control loops are modeled as interconnections of hybrid systems for which novel results on Lp-stability are presented. Prediction of a triggering event is achieved by employing Lp-gains over a finite horizon in the small gain theorem. By resorting to convex programming, a method to compute Lp-gains over a finite horizon is devised. Next, we investigate optimal intermittent feedback for nonlinear control systems. Using the currently available measurements from a plant, we develop a methodology that outputs when to update the control law with new measurements such that a given cost function is minimized. Our cost function captures trade-offs between the performance and energy consumption of the control system. The optimization problem is formulated as a Dynamic Programming problem, and Approximate Dynamic Programming is employed to solve it. Instead of advocating a particular approximation architecture for Approximate Dynamic Programming, we formulate properties that successful approximation architectures satisfy. In addition, we consider problems with partially observable states, and propose Particle Filtering to deal with partially observable states and intermittent feedback. Finally, we investigate a decentralized output synchronization problem of heterogeneous linear systems. We develop a self-triggered output broadcasting policy for the interconnected systems. Broadcasting time instants adapt to the current communication topology. For a fixed topology, our broadcasting policy yields global exponential output synchronization, and Lp-stable output synchronization in the presence of disturbances. Employing a converse Lyapunov theorem for impulsive systems, we provide an average dwell time condition that yields disturbance-to-state stable output synchronization in case of switching topology. Our approach is applicable to directed and unbalanced communication topologies.\u2

Energy-Efficient Self-Organization of Wireless Acoustic Sensor Networks for Ground Target Tracking

With the developments in computing and communication technologies, wireless sensor networks have become popular in wide range of application areas such as health, military, environment and habitant monitoring. Moreover, wireless acoustic sensor networks have been widely used for target tracking applications due to their passive nature, reliability and low cost. Traditionally, acoustic sensor arrays built in linear, circular or other regular shapes are used for tracking acoustic sources. The maintaining of relative geometry of the acoustic sensors in the array is vital for accurate target tracking, which greatly reduces the flexibility of the sensor network. To overcome this limitation, we propose using only a single acoustic sensor at each sensor node. This design greatly improves the flexibility of the sensor network and makes it possible to deploy the sensor network in remote or hostile regions through air-drop or other stealth approaches. Acoustic arrays are capable of performing the target localization or generating the bearing estimations on their own. However, with only a single acoustic sensor, the sensor nodes will not be able to generate such measurements. Thus, self-organization of sensor nodes into virtual arrays to perform the target localization is essential. We developed an energy-efficient and distributed self-organization algorithm for target tracking using wireless acoustic sensor networks. The major error sources of the localization process were studied, and an energy-aware node selection criterion was developed to minimize the target localization errors. Using this node selection criterion, the self-organization algorithm selects a near-optimal localization sensor group to minimize the target tracking errors. In addition, a message passing protocol was developed to implement the self-organization algorithm in a distributed manner. In order to achieve extended sensor network lifetime, energy conservation was incorporated into the self-organization algorithm by incorporating a sleep-wakeup management mechanism with a novel cross layer adaptive wakeup probability adjustment scheme. The simulation results confirm that the developed self-organization algorithm provides satisfactory target tracking performance. Moreover, the energy saving analysis confirms the effectiveness of the cross layer power management scheme in achieving extended sensor network lifetime without degrading the target tracking performance

Exploiting Heterogeneity in Networks of Aerial and Ground Robotic Agents

By taking advantage of complementary communication technologies, distinct sensing functionalities and varied motion dynamics present in a heterogeneous multi-robotic network, it is possible to accomplish a main mission objective by assigning specialized sub-tasks to specific members of a robotic team. An adequate selection of the team members and an effective coordination are some of the challenges to fully exploit the unique capabilities that these types of systems can offer. Motivated by real world applications, we focus on a multi-robotic network consisting off aerial and ground agents which has the potential to provide critical support to humans in complex settings. For instance, aerial robotic relays are capable of transporting small ground mobile sensors to expand the communication range and the situational awareness of first responders in hazardous environments. In the first part of this dissertation, we extend work on manipulation of cable-suspended loads using aerial robots by solving the problem of lifting the cable-suspended load from the ground before proceeding to transport it. Since the suspended load-quadrotor system experiences switching conditions during this critical maneuver, we define a hybrid system and show that it is differentially-flat. This property facilitates the design of a nonlinear controller which tracks a waypoint-based trajectory associated with the discrete states of the hybrid system. In addition, we address the case of unknown payload mass by combining a least-squares estimation method with the designed controller. Second, we focus on the coordination of a heterogeneous team formed by a group of ground mobile sensors and a flying communication router which is deployed to sense areas of interest in a cluttered environment. Using potential field methods, we propose a controller for the coordinated mobility of the team to guarantee inter-robot and obstacle collision avoidance as well as connectivity maintenance among the ground agents while the main goal of sensing is carried out. For the case of the aerial communications relays, we combine antenna diversity with reinforcement learning to dynamically re-locate these relays so that the received signal strength is maintained above a desired threshold. Motivated by the recent interest of combining radio frequency and optical wireless communications, we envision the implementation of an optical link between micro-scale aerial and ground robots. This type of link requires maintaining a sufficient relative transmitter-receiver position for reliable communications. In the third part of this thesis, we tackle this problem. Based on the link model, we define a connectivity cone where a minimum transmission rate is guaranteed. For example, the aerial robot has to track the ground vehicle to stay inside this cone. The control must be robust to noisy measurements. Thus, we use particle filters to obtain a better estimation of the receiver position and we design a control algorithm for the flying robot to enhance the transmission rate. Also, we consider the problem of pairing a ground sensor with an aerial vehicle, both equipped with a hybrid radio-frequency/optical wireless communication system. A challenge is positioning the flying robot within optical range when the sensor location is unknown. Thus, we take advantage of the hybrid communication scheme by developing a control strategy that uses the radio signal to guide the aerial platform to the ground sensor. Once the optical-based signal strength has achieved a certain threshold, the robot hovers within optical range. Finally, we investigate the problem of building an alliance of agents with different skills in order to satisfy the requirements imposed by a given task. We find this alliance, known also as a coalition, by using a bipartite graph in which edges represent the relation between agent capabilities and required resources for task execution. Using this graph, we build a coalition whose total capability resources can satisfy the task resource requirements. Also, we study the heterogeneity of the formed coalition to analyze how it is affected for instance by the amount of capability resources present in the agents