Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks (WSNs). We study the problem of scheduling the sleep times of the individual sensors to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous state-action spaces, in a manner similar to (Fuemmeler and Veeravalli [2008]). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation (SPSA) estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation for the Q-values) is updated in an on-policy temporal difference (TD) algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model. Our simulation results on a 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work

CAREER: Data Management for Ad-Hoc Geosensor Networks

This project explores data management methods for geosensor networks, i.e. large collections of very small, battery-driven sensor nodes deployed in the geographic environment that measure the temporal and spatial variations of physical quantities such as temperature or ozone levels. An important task of such geosensor networks is to collect, analyze and estimate information about continuous phenomena under observation such as a toxic cloud close to a chemical plant in real-time and in an energy-efficient way. The main thrust of this project is the integration of spatial data analysis techniques with in-network data query execution in sensor networks. The project investigates novel algorithms such as incremental, in-network kriging that redefines a traditional, highly computationally intensive spatial data estimation method for a distributed, collaborative and incremental processing between tiny, energy and bandwidth constrained sensor nodes. This work includes the modeling of location and sensing characteristics of sensor devices with regard to observed phenomena, the support of temporal-spatial estimation queries, and a focus on in-network data aggregation algorithms for complex spatial estimation queries. Combining high-level data query interfaces with advanced spatial analysis methods will allow domain scientists to use sensor networks effectively in environmental observation. The project has a broad impact on the community involving undergraduate and graduate students in spatial database research at the University of Maine as well as being a key component of a current IGERT program in the areas of sensor materials, sensor devices and sensor. More information about this project, publications, simulation software, and empirical studies are available on the project\u27s web site (http://www.spatial.maine.edu/~nittel/career/)

Outlier detection techniques for wireless sensor networks: A survey

In the field of wireless sensor networks, those measurements that significantly deviate from the normal pattern of sensed data are considered as outliers. The potential sources of outliers include noise and errors, events, and malicious attacks on the network. Traditional outlier detection techniques are not directly applicable to wireless sensor networks due to the nature of sensor data and specific requirements and limitations of the wireless sensor networks. This survey provides a comprehensive overview of existing outlier detection techniques specifically developed for the wireless sensor networks. Additionally, it presents a technique-based taxonomy and a comparative table to be used as a guideline to select a technique suitable for the application at hand based on characteristics such as data type, outlier type, outlier identity, and outlier degree

Outlier Detection Techniques For Wireless Sensor Networks: A Survey

In the field of wireless sensor networks, measurements that significantly deviate from the normal pattern of sensed data are considered as outliers. The potential sources of outliers include noise and errors, events, and malicious attacks on the network. Traditional outlier detection techniques are not directly applicable to wireless sensor networks due to the multivariate nature of sensor data and specific requirements and limitations of the wireless sensor networks. This survey provides a comprehensive overview of existing outlier detection techniques specifically developed for the wireless sensor networks. Additionally, it presents a technique-based taxonomy and a decision tree to be used as a guideline to select a technique suitable for the application at hand based on characteristics such as data type, outlier type, outlier degree

Design of Integrated Artificial Intelligence Techniques for Video Surveillance on IoT Enabled Wireless Multimedia Sensor Networks

The recent advancements in the Internet of Things (IoT) and Wireless Multimedia Sensor Networks (WMSN) made high-speed multimedia streaming, data processing, and essential analytics processes with minimal delay. Multimedia sensors used in WMSN-based surveillance applications are beneficial helpful in attaining accurate and elaborate details. However, it has become essential to design an effective and lightweight solution for data traffic management in WMSN owing to the massive quantities of data, generated by multimedia sensors. The development of Artificial Intelligence (AI) and Machine Learning (ML) techniques can be leveraged to investigate, collect, store, and process multimedia streaming data for decision-making in real-time scenarios. In this aspect, the current study develops an Integrated AI technique for Video Surveillance in IoT-enabled WMSN, called IAIVS-WMSN. The proposed IAIVS-WMSN technique aims to design a practical scheme for object detection and data transmission in WMSN. The proposed IAIVS-WMSN approach encompasses three stages: object detection, image compression, and clustering. The Mask Regional Convolutional Neural Network (Mask RCNN) technique is primarily utilized for object detection in the target region. Besides, Neighbourhood Correlation Sequence-based Image Compression (NCSIC) technique is applied to reduce data transmission. Finally, Artificial Flora Algorithm (AFA)-based clustering technique is designed for the election of Cluster Heads (CHs) and construction clusters. The design of object detection with compression and clustering techniques for WMSN shows the novelty of the work. These three processes’ designs enable one to accomplish effective data transmission in IoT-enabled WMSN. The researchers conducted multiple simulations to highlight the supreme performance of the IAIVS-WMSN approach. The simulation outcomes inferred the enhanced performance of the IAIVS-WMSN algorithm to the existing approaches

Multi-dimensional data indexing and range query processing via Voronoi diagram for internet of things

In a typical Internet of Things (IoT) deployment such as smart cities and Industry 4.0, the amount of sensory data collected from physical world is significant and wide-ranging. Processing large amount of real-time data from the diverse IoT devices is challenging. For example, in IoT environment, wireless sensor networks (WSN) are typically used for the monitoring and collecting of data in some geographic area. Spatial range queries with location constraints to facilitate data indexing are traditionally employed in such applications, which allows the querying and managing the data based on SQL structure. One particular challenge is to minimize communication cost and storage requirements in multi-dimensional data indexing approaches. In this paper, we present an energy- and time-efficient multidimensional data indexing scheme, which is designed to answer range query. Specifically, we propose data indexing methods which utilize hierarchical indexing structures, using binary space partitioning (BSP), such as kd-tree, quad-tree, k-means clustering, and Voronoi-based methods to provide more efficient routing with less latency. Simulation results demonstrate that the Voronoi Diagram-based algorithm minimizes the average energy consumption and query response time