Metadata extraction and organization for intelligent video surveillance system

Proceedings of the IEEE International Conference on Mechatronics and Automation (ICMA), 2010, p. 489-494The research for metadata extraction originates from the intelligent video surveillance system, which is widely used in outdoor and indoor environment for the aims of traffic monitor, security guard, and intelligent robot. Various features are extracted from the surveillance image sequences such as target detection, target tracking, object's shape and activities. However, the trend of more and more features being used and shared in video surveillance system calls for more attention to bridge the gap between specific analysis algorithms and enduser's expectation. This paper proposes a three-layer object oriented model to extract the surveillance metadata including shape, motion speed, and trajectory of the object emerging in image sequence. Meanwhile, the high-level semantic metadata including entry/exit point, object duration time is organized and stored which are provided for the further end-user queries. The paper also presents the experiment results in different indoor and outdoor surveillance scenarios. At last, a comparative analysis with another traditional method is presented. © 2010 IEEE.published_or_final_versio

Evaluation of video based pedestrian and vehicle detection algorithms

Video based detection systems rely on the ability to detect moving objects in video streams. Video based detection systems have applications in many fields like, intelligent transportation, automated surveillance etc. There are many approaches adopted for video based detection. Evaluation and selecting a suitable approach for pedestrian and vehicle detection is a challenging task. While evaluating the object detection algorithms, many factors should be considered in order to cope with unconstrained environments, non stationary background, different object motion patterns and the variation in types of object being detected. In this thesis, we implement and evaluate different video based detection algorithms used for pedestrian and vehicle detection. Video based pedestrian and vehicle detection involves object detection through background foreground segmentation and object tracking. For background foreground segmentation, frame differencing, background averaging, mixture of Gaussians and codebook methods were implemented. For object tracking, Mean-Shift tracking and Lucas Kanade optical flow tracking algorithms were implemented. The performance of each of these algorithms is evaluated by a comparative study; based on their performance such as ability to get good detection and tracking, CodeBook algorithm is selected as a candidate algorithm for background foreground segmentation and Mean-Shift tracking is used to track the detected objects for pedestrian and vehicle detection

A survey on object detection and tracking algorithms

Object detection and tracking are important and challenging task in many computer vision applications such as surveillance, vehicle navigation and autonomous robot navigation. Video surveillance in dynamic environment, especially for humans and vehicles, is one of the current challenging research topics in computer vision. It is a key technology to fight against terrorism, crime, public safety and for efficient management of traffic. The work involves designing of efficient video surveillance system in complex environments. In video surveillance, detection of moving objects from a video is important for object detection, target tracking, and behaviour understanding. Detection of moving objects in video streams is the first relevant step of information and background subtraction is a very popular approach for foreground segmentation. In this thesis, we have simulated different background subtraction methods to overcome the problem of illumination variation, background clutter and shadows. Detecting and tracking of human body parts is important in understanding human activities. Intelligent and automated security surveillance systems have become an active research area in recent time due to an increasing demand for such systems in public areas such as airports, underground stations and mass events. In this context, tracking of stationary foreground regions is one of the most critical requirements for surveillance systems based on the tracking of abandoned or stolen objects or parked vehicles

Application of improved you only look once model in road traffic monitoring system

The present research focuses on developing an intelligent traffic management solution for tracking the vehicles on roads. Our proposed work focuses on a much better you only look once (YOLOv4) traffic monitoring system that uses the CSPDarknet53 architecture as its foundation. Deep-sort learning methodology for vehicle multi-target detection from traffic video is also part of our research study. We have included features like the Kalman filter, which estimates unknown objects and can track moving targets. Hungarian techniques identify the correct frame for the object. We are using enhanced object detection network design and new data augmentation techniques with YOLOv4, which ultimately aids in traffic monitoring. Until recently, object identification models could either perform quickly or draw conclusions quickly. This was a big improvement, as YOLOv4 has an astoundingly good performance for a very high frames per second (FPS). The current study is focused on developing an intelligent video surveillance-based vehicle tracking system that tracks the vehicles using a neural network, image-based tracking, and YOLOv4. Real video sequences of road traffic are used to test the effectiveness of the method that has been suggested in the research. Through simulations, it is demonstrated that the suggested technique significantly increases graphics processing unit (GPU) speed and FSP as compared to baseline algorithms

INTELLIGENT VIDEO SURVEILLANCE OF HUMAN MOTION: ANOMALY DETECTION

Intelligent video surveillance is a system that can highlight extraction and video summarization that require recognition of the activities occurring in the video without any human supervision. Surveillance systems are extremely helpful to guard or protect you from any dangerous condition. In this project, we propose a system that can track and detect abnormal behavior in indoor environment. By concentrating on inside house enviromnent, we want to detect any abnormal behavior between adult and toddler to avoid abusing to happen. In general, the frameworks of a video surveillance system include the following stages: background estimator, segmentation, detection, tracking, behavior understanding and description. We use training behavior profile to collect the description and generate statistically behavior to perform anomaly detection later. We begin with modeling the simplest actions like: stomping, slapping, kicking, pointed sharp or blunt object that do not require sophisticated modeling. A method to model actions with more complex dynamic are then discussed. The results of the system manage to track adult figure, toddler figure and harm object as third subject. With this system, it can bring attention of human personnel security. For future work, we recommend to continue design methods for higher level representation of complex activities to do the matching anomaly detection with real-time video surveillance. We also propose the system to embed with hardware solution for triggered the matching detection as output

Threshold adaptation and XOR accumulation algorithm for objects detection

Object detection, tracking and video analysis are vital and energetic tasks for intelligent video surveillance systems and computer vision applications. Object detection based on background modelling is a major technique used in dynamically objects extraction over video streams. This paper presents the threshold adaptation and XOR accumulation (TAXA) algorithm in three systematic stages throughout video sequences. First, the continuous calculation, updating and elimination of noisy background details with hybrid statistical techniques. Second, thresholds are calculated with an effective mean and gaussian for the detection of the pixels of the objects. The third is a novel step in making decisions by using XOR-accumulation to extract pixels of the objects from the thresholds accurately. Each stage was presented with practical representations and theoretical explanations. On high resolution video which has difficult scenes and lighting conditions, the proposed algorithm was used and tested. As a result, with a precision average of 0.90% memory uses of 6.56% and the use of CPU 20% as well as time performance, the result excellent overall superior to all the major used foreground object extraction algorithms. As a conclusion, in comparison to other popular OpenCV methods the proposed TAXA algorithm has excellent detection ability

Modelling of Intelligent Object Detection and Classification using Aquila Optimizer with Deep Learning on Surveillance Videos

Object Detection (OD) in surveillance video is the way of automatically detecting and tracking object classes of interest within the video recording. It includes the application of a Computer Vision (CV) technique to analyze the video frame and identify the classes of objects or the presence of specific objects. Various OD techniques are used to find objects within the footage video. This algorithm analyzes the visual feature of the frames and employs Machine Learning (ML) approaches namely Deep Neural Network (DNN), to detect and track objects. It is worth mentioning that the accuracy and performance of OD in surveillance video depends on factors including the choice of algorithms and models, the availability of labelled training data, and the quality of the video frame for the specific object of interest. This study introduces a new modeling of Intelligent Object Recognition and Classification by employing Aquila Optimizer with Deep Learning (IODC-AODL) approach in Surveillance Video. The goal of the IODC-AODL technique is to integrate the DL model with the hyperparameter tuning process for object detection and classification. In the proposed IODC-AODL approach, a Faster RCNN method is enforced for the process of OD. Next, Long Short-Term Memory (LSTM) networking approach is implemented for the object classification process. At last, the AO approach is enforced for the optimum hyperparameter tuning of the LSTM network and it assists in improving the classifier rate. A widespread simulation sets are performed to exhibit the superior performance of the IODC-AODL approach. The experimental result analysis portrayed the supremacy of the IODC-AODL algorithm over other models

Spatial Pyramid Context-Aware Moving Object Detection and Tracking for Full Motion Video and Wide Aerial Motion Imagery

A robust and fast automatic moving object detection and tracking system is essential to characterize target object and extract spatial and temporal information for different functionalities including video surveillance systems, urban traffic monitoring and navigation, robotic. In this dissertation, I present a collaborative Spatial Pyramid Context-aware moving object detection and Tracking system. The proposed visual tracker is composed of one master tracker that usually relies on visual object features and two auxiliary trackers based on object temporal motion information that will be called dynamically to assist master tracker. SPCT utilizes image spatial context at different level to make the video tracking system resistant to occlusion, background noise and improve target localization accuracy and robustness. We chose a pre-selected seven-channel complementary features including RGB color, intensity and spatial pyramid of HoG to encode object color, shape and spatial layout information. We exploit integral histogram as building block to meet the demands of real-time performance. A novel fast algorithm is presented to accurately evaluate spatially weighted local histograms in constant time complexity using an extension of the integral histogram method. Different techniques are explored to efficiently compute integral histogram on GPU architecture and applied for fast spatio-temporal median computations and 3D face reconstruction texturing. We proposed a multi-component framework based on semantic fusion of motion information with projected building footprint map to significantly reduce the false alarm rate in urban scenes with many tall structures. The experiments on extensive VOTC2016 benchmark dataset and aerial video confirm that combining complementary tracking cues in an intelligent fusion framework enables persistent tracking for Full Motion Video and Wide Aerial Motion Imagery.Comment: PhD Dissertation (162 pages

A Low Cost and Computationally Efficient Approach for Occlusion Handling in Video Surveillance Systems

In the development of intelligent video surveillance systems for tracking a vehicle, occlusions are one of the major challenges. It becomes difficult to retain features during occlusion especially in case of complete occlusion. In this paper, a target vehicle tracking algorithm for Smart Video Surveillance (SVS) is proposed to track an unidentified target vehicle even in case of occlusions. This paper proposes a computationally efficient approach for handling occlusions named as Kalman Filter Assisted Occlusion Handling (KFAOH) technique. The algorithm works through two periods namely tracking period when no occlusion is seen and detection period when occlusion occurs, thus depicting its hybrid nature. Kanade-Lucas-Tomasi (KLT) feature tracker governs the operation of algorithm during the tracking period, whereas, a Cascaded Object Detector (COD) of weak classifiers, specially trained on a large database of cars governs the operation during detection period or occlusion with the assistance of Kalman Filter (KF). The algorithm’s tracking efficiency has been tested on six different tracking scenarios with increasing complexity in real-time. Performance evaluation under different noise variances and illumination levels shows that the tracking algorithm has good robustness against high noise and low illumination. All tests have been conducted on the MATLAB platform. The validity and practicality of the algorithm are also verified by success plots and precision plots for the test cases