Design of Integrated Artificial Intelligence Techniques for Video Surveillance on IoT Enabled Wireless Multimedia Sensor Networks

The recent advancements in the Internet of Things (IoT) and Wireless Multimedia Sensor Networks (WMSN) made high-speed multimedia streaming, data processing, and essential analytics processes with minimal delay. Multimedia sensors used in WMSN-based surveillance applications are beneficial helpful in attaining accurate and elaborate details. However, it has become essential to design an effective and lightweight solution for data traffic management in WMSN owing to the massive quantities of data, generated by multimedia sensors. The development of Artificial Intelligence (AI) and Machine Learning (ML) techniques can be leveraged to investigate, collect, store, and process multimedia streaming data for decision-making in real-time scenarios. In this aspect, the current study develops an Integrated AI technique for Video Surveillance in IoT-enabled WMSN, called IAIVS-WMSN. The proposed IAIVS-WMSN technique aims to design a practical scheme for object detection and data transmission in WMSN. The proposed IAIVS-WMSN approach encompasses three stages: object detection, image compression, and clustering. The Mask Regional Convolutional Neural Network (Mask RCNN) technique is primarily utilized for object detection in the target region. Besides, Neighbourhood Correlation Sequence-based Image Compression (NCSIC) technique is applied to reduce data transmission. Finally, Artificial Flora Algorithm (AFA)-based clustering technique is designed for the election of Cluster Heads (CHs) and construction clusters. The design of object detection with compression and clustering techniques for WMSN shows the novelty of the work. These three processes’ designs enable one to accomplish effective data transmission in IoT-enabled WMSN. The researchers conducted multiple simulations to highlight the supreme performance of the IAIVS-WMSN approach. The simulation outcomes inferred the enhanced performance of the IAIVS-WMSN algorithm to the existing approaches

Computation-Communication Trade-offs and Sensor Selection in Real-time Estimation for Processing Networks

Recent advances in electronics are enabling substantial processing to be performed at each node (robots, sensors) of a networked system. Local processing enables data compression and may mitigate measurement noise, but it is still slower compared to a central computer (it entails a larger computational delay). However, while nodes can process the data in parallel, the centralized computational is sequential in nature. On the other hand, if a node sends raw data to a central computer for processing, it incurs communication delay. This leads to a fundamental communication-computation trade-off, where each node has to decide on the optimal amount of preprocessing in order to maximize the network performance. We consider a network in charge of estimating the state of a dynamical system and provide three contributions. First, we provide a rigorous problem formulation for optimal real-time estimation in processing networks in the presence of delays. Second, we show that, in the case of a homogeneous network (where all sensors have the same computation) that monitors a continuous-time scalar linear system, the optimal amount of local preprocessing maximizing the network estimation performance can be computed analytically. Third, we consider the realistic case of a heterogeneous network monitoring a discrete-time multi-variate linear system and provide algorithms to decide on suitable preprocessing at each node, and to select a sensor subset when computational constraints make using all sensors suboptimal. Numerical simulations show that selecting the sensors is crucial. Moreover, we show that if the nodes apply the preprocessing policy suggested by our algorithms, they can largely improve the network estimation performance.Comment: 15 pages, 16 figures. Accepted journal versio

A Survey on Multimedia-Based Cross-Layer Optimization in Visual Sensor Networks

Visual sensor networks (VSNs) comprised of battery-operated electronic devices endowed with low-resolution cameras have expanded the applicability of a series of monitoring applications. Those types of sensors are interconnected by ad hoc error-prone wireless links, imposing stringent restrictions on available bandwidth, end-to-end delay and packet error rates. In such context, multimedia coding is required for data compression and error-resilience, also ensuring energy preservation over the path(s) toward the sink and improving the end-to-end perceptual quality of the received media. Cross-layer optimization may enhance the expected efficiency of VSNs applications, disrupting the conventional information flow of the protocol layers. When the inner characteristics of the multimedia coding techniques are exploited by cross-layer protocols and architectures, higher efficiency may be obtained in visual sensor networks. This paper surveys recent research on multimedia-based cross-layer optimization, presenting the proposed strategies and mechanisms for transmission rate adjustment, congestion control, multipath selection, energy preservation and error recovery. We note that many multimedia-based cross-layer optimization solutions have been proposed in recent years, each one bringing a wealth of contributions to visual sensor networks

Compressive Sensing for Target Detection and Tracking within Wireless Visual Sensor Networks-based Surveillance applications

Wireless Visual Sensor Networks (WVSNs) have gained significant importance in the last few years and have emerged in several distinctive applications. The main aim of this research is to investigate the use of adaptive Compressive Sensing (CS) for efficient target detection and tracking in WVSN-based surveillance applications. CS is expected to overcome the WVSN resource constraints such as memory limitation, communication bandwidth and battery constraints. In addition, adaptive CS dynamically chooses variable compression rates according to different data sets to represent captured images in an efficient way hence saving energy and memory space. In this work, a literature review on compressive sensing, target detection and tracking for WVSN is carried out to investigate existing techniques. Only single view target tracking is considered to keep minimum number of visual sensor nodes in a wake-up state to optimize the use of nodes and save battery life which is limited in WVSNs. To reduce the size of captured images an adaptive block CS technique is proposed and implemented to compress the high volume data images before being transmitted through the wireless channel. The proposed technique divides the image to blocks and adaptively chooses the compression rate for relative blocks containing the target according to the sparsity nature of images. At the receiver side, the compressed image is then reconstructed and target detection and tracking are performed to investigate the effect of CS on the tracking performance. Least mean square adaptive filter is used to predicts target’s next location, an iterative quantized clipped LMS technique is proposed and compared with other variants of LMS and results have shown that it achieved lower error rates than other variants of lMS. The tracking is performed in both indoor and outdoor environments for single/multi targets. Results have shown that with adaptive block compressive sensing (CS) up to 31% measurements of data are required to be transmitted for less sparse images and 15% for more sparse, while preserving the 33dB image quality and the required detection and tracking performance. Adaptive CS resulted in 82% energy saving as compared to transmitting the required image with no C

Correlation-based communication in wireless multimedia sensor networks

Wireless multimedia sensor networks (WMSNs) are networks of interconnected devices that allow retrieving video and audio streams, still images, and scalar data from the environment. In a densely deployed WMSN, there exists correlation among the observations of camera sensors with overlapped coverage areas, which introduces substantial data redundancy in the network. In this dissertation, efficient communication schemes are designed for WMSNs by leveraging the correlation of visual information observed by camera sensors. First, a spatial correlation model is developed to estimate the correlation of visual information and the joint entropy of multiple correlated camera sensors. The compression performance of correlated visual information is then studied. An entropy-based divergence measure is proposed to predict the compression efficiency of performing joint coding on the images from correlated cameras. Based on the predicted compression efficiency, a clustered coding technique is proposed that maximizes the overall compression gain of the visual information gathered in WMSNs. The correlation of visual information is then utilized to design a network scheduling scheme to maximize the lifetime of WMSNs. Furthermore, as many WMSN applications require QoS support, a correlation-aware QoS routing algorithm is introduced that can efficiently deliver visual information under QoS constraints. Evaluation results show that, by utilizing the correlation of visual information in the communication process, the energy efficiency and networking performance of WMSNs could be improved significantly.PhDCommittee Chair: Akyildiz, Ian; Committee Member: Ammar, Mostafa; Committee Member: Ji, Chuanyi; Committee Member: Li, Ye; Committee Member: Romberg, Justi

An Immersive Telepresence System using RGB-D Sensors and Head Mounted Display

We present a tele-immersive system that enables people to interact with each other in a virtual world using body gestures in addition to verbal communication. Beyond the obvious applications, including general online conversations and gaming, we hypothesize that our proposed system would be particularly beneficial to education by offering rich visual contents and interactivity. One distinct feature is the integration of egocentric pose recognition that allows participants to use their gestures to demonstrate and manipulate virtual objects simultaneously. This functionality enables the instructor to ef- fectively and efficiently explain and illustrate complex concepts or sophisticated problems in an intuitive manner. The highly interactive and flexible environment can capture and sustain more student attention than the traditional classroom setting and, thus, delivers a compelling experience to the students. Our main focus here is to investigate possible solutions for the system design and implementation and devise strategies for fast, efficient computation suitable for visual data processing and network transmission. We describe the technique and experiments in details and provide quantitative performance results, demonstrating our system can be run comfortably and reliably for different application scenarios. Our preliminary results are promising and demonstrate the potential for more compelling directions in cyberlearning.Comment: IEEE International Symposium on Multimedia 201