Advanced vision based vehicle classification for traffic surveillance system using neural networks

Master's thesis in Cybernetics and signal processingThis master thesis focus on traffic monitoring, which are of importance to fulfill planning and traffic management of road networks. An important requirement is data interpretation accuracy to provide adequate characteristic data from the acquired vision-data. A vision-based system has been developed, using new methods and technologies to achieve an automated traffic monitoring system, without the use of additional sensors. The thesis is based upon Erik Sudland’s master thesis from 2016, which investigated available litterateur containing adequate algorithms for traffic monitoring. However in the current master thesis, methods have been further analyzed and experimentally optimized on vision-data from real traffic situations. In addition, a new classification method based upon neural networks has been implemented and verified with successful result

Segmentation of Football Video Broadcast

In this paper a novel segmentation system for football player detection in broadcasted video is presented. Proposed detection system is a complex solution incorporating a dominant color based segmentation technique of a football playfield, a 3D playfield modeling algorithm based on Hough transform and a dedicated algorithm for player tracking, player detection system based on the combination of Histogram of Oriented Gradients (HOG) descriptors with Principal Component Analysis (PCA) and linear Support Vector Machine (SVM) classification. For the shot classification the several classification technique SVM, artificial neural network and Linear Discriminant Analysis (LDA) are used. Evaluation of the system is carried out using HD (1280×720) resolution test material. Additionally, performance of the proposed system is tested with different lighting conditions (including non-uniform pith lightning and multiple player shadows) and various camera positions. Experimental results presented in this paper show that combination of these techniques seems to be a promising solution for locating and segmenting objects in a broadcasted video

Design Of Computer Vision Systems For Optimizing The Threat Detection Accuracy

This dissertation considers computer vision (CV) systems in which a central monitoring station receives and analyzes the video streams captured and delivered wirelessly by multiple cameras. It addresses how the bandwidth can be allocated to various cameras by presenting a cross-layer solution that optimizes the overall detection or recognition accuracy. The dissertation presents and develops a real CV system and subsequently provides a detailed experimental analysis of cross-layer optimization. Other unique features of the developed solution include employing the popular HTTP streaming approach, utilizing homogeneous cameras as well as heterogeneous ones with varying capabilities and limitations, and including a new algorithm for estimating the effective medium airtime. The results show that the proposed solution significantly improves the CV accuracy. Additionally, the dissertation features an improved neural network system for object detection. The proposed system considers inherent video characteristics and employs different motion detection and clustering algorithms to focus on the areas of importance in consecutive frames, allowing the system to dynamically and efficiently distribute the detection task among multiple deployments of object detection neural networks. Our experimental results indicate that our proposed method can enhance the mAP (mean average precision), execution time, and required data transmissions to object detection networks. Finally, as recognizing an activity provides significant automation prospects in CV systems, the dissertation presents an efficient activity-detection recurrent neural network that utilizes fast pose/limbs estimation approaches. By combining object detection with pose estimation, the domain of activity detection is shifted from a volume of RGB (Red, Green, and Blue) pixel values to a time-series of relatively small one-dimensional arrays, thereby allowing the activity detection system to take advantage of highly capable neural networks that have been trained on large GPU clusters for thousands of hours. Consequently, capable activity detection systems with considerably fewer training sets and processing hours can be built

Tracking interacting targets in multi-modal sensors

PhDObject tracking is one of the fundamental tasks in various applications such as surveillance, sports, video conferencing and activity recognition. Factors such as occlusions, illumination changes and limited field of observance of the sensor make tracking a challenging task. To overcome these challenges the focus of this thesis is on using multiple modalities such as audio and video for multi-target, multi-modal tracking. Particularly, this thesis presents contributions to four related research topics, namely, pre-processing of input signals to reduce noise, multi-modal tracking, simultaneous detection and tracking, and interaction recognition. To improve the performance of detection algorithms, especially in the presence of noise, this thesis investigate filtering of the input data through spatio-temporal feature analysis as well as through frequency band analysis. The pre-processed data from multiple modalities is then fused within Particle filtering (PF). To further minimise the discrepancy between the real and the estimated positions, we propose a strategy that associates the hypotheses and the measurements with a real target, using a Weighted Probabilistic Data Association (WPDA). Since the filtering involved in the detection process reduces the available information and is inapplicable on low signal-to-noise ratio data, we investigate simultaneous detection and tracking approaches and propose a multi-target track-beforedetect Particle filtering (MT-TBD-PF). The proposed MT-TBD-PF algorithm bypasses the detection step and performs tracking in the raw signal. Finally, we apply the proposed multi-modal tracking to recognise interactions between targets in regions within, as well as outside the cameras’ fields of view. The efficiency of the proposed approaches are demonstrated on large uni-modal, multi-modal and multi-sensor scenarios from real world detections, tracking and event recognition datasets and through participation in evaluation campaigns

Change detection in combination with spatial models and its effectiveness on underwater scenarios

This thesis proposes a novel change detection approach for underwater scenarios and combines it with different especially developed spatial models, this allows accurate and spatially coherent detection of any moving objects with a static camera in arbitrary environments. To deal with the special problems of underwater imaging pre-segmentations based on the optical flow and other special adaptions were added to the change detection algorithm so that it can better handle typical underwater scenarios like a scene crowded by a whole fish swarm

A REAL-TIME TRAFFIC CONDITION ASSESSMENT AND PREDICTION FRAMEWORK USING VEHICLE-INFRASTRUCTURE INTEGRATION (VII) WITH COMPUTATIONAL INTELLIGENCE

This research developed a real-time traffic condition assessment and prediction framework using Vehicle-Infrastructure Integration (VII) with computational intelligence to improve the existing traffic surveillance system. Due to the prohibited expenses and complexity involved for the field experiment of such a system, this study adopted state-of-the-art simulation tools as an efficient alternative. This work developed an integrated traffic and communication simulation platform to facilitate the design and evaluation of a wide range of online traffic surveillance and management system in both traffic and communication domain. Using the integrated simulator, the author evaluated the performance of different combination of communication medium and architecture. This evaluation led to the development of a hybrid VII framework exemplified by hierarchical architecture, which is expected to eliminate single point failures, enhance scalability and easy integration of control functions for traffic condition assessment and prediction. In the proposed VII framework, the vehicle on-board equipments and roadside units (RSUs) work collaboratively, based on an intelligent paradigm known as \u27Support Vector Machine (SVM),\u27 to determine the occurrence and characteristics of an incident with the kinetics data generated by vehicles. In addition to incident detection, this research also integrated the computational intelligence paradigm called \u27Support Vector Regression (SVR)\u27 within the hybrid VII framework for improving the travel time prediction capabilities, and supporting on-line leaning functions to improve its performance over time. Two simulation models that fully implemented the functionalities of real-time traffic surveillance were developed on calibrated and validated simulation network for study sites in Greenville and Spartanburg, South Carolina. The simulation models\u27 encouraging performance on traffic condition assessment and prediction justifies further research on field experiment of such a system to address various research issues in the areas covered by this work, such as availability and accuracy of vehicle kinetic and maneuver data, reliability of wireless communication, maintenance of RSUs and wireless repeaters. The impact of this research will provide a reliable alternative to traditional traffic sensors to assess and predict the condition of the transportation system. The integrated simulation methodology and open source software will provide a tool for design and evaluation of any real-time traffic surveillance and management systems. Additionally, the developed VII simulation models will be made available for use by future researchers and designers of other similar VII systems. Future implementation of the research in the private and public sector will result in new VII related equipment in vehicles, greater control of traffic loading, faster incident detection, improved safety, mitigated congestion, and reduced emissions and fuel consumption

Navigation and Control of Mobile Robots

The rapid development of robotics has benefited by more and more people putting their attention to it. In the 1920s, ‘Robota’, a similar concept, was first known to the world. It is proposed in Karel Capek’ s drama, Rossum’ s Universal Robots (RUR). From then on, numbers of automatic machines were created all over the world, which are known as the robots of the early periods. Gradually, the demand for robots is growing for the purpose of fulfilling tasks instead of humans. From industrial uses, to the military, to education and entertainment, di↵erent kinds of robots began to serve humans in various scenarios. Based on this, how to control the robot better is becoming a hot topic. For the topic of navigating and controlling mobile robots, number of related problems have been carried out. Obstacle avoidance, path planning, cooperative work of multi-robots. In this thesis, we focus on the first two problems, and mention the last one as a future direction in the last part. For obstacle avoidance, we proposed algorithms for both 2D planar environ- ments and 3D space environments. The example cases we raise are those that need to be addressed but have always been ignored. To be specific, the motion of the obstacles are not fixed, the shape of the obstacles are changeable, and the sensors that could be deployed for underwater environments are limited. We even put those problems together to solve them. The methods we proposed are based on the biologically inspired algorithm and Back Propagation Neural network (BPNN). In addition, we put e↵orts into trajectory planning for robots. The two scenarios we set are self-driving cars on the road and reconnaissance and surveillance of drones. The methods we deployed are the Convolutional Neural Network (CNN) method and the two-phase strategy, respectively. When we proposed the strategies, we gave a detailed description of the robot systems, the proposed algorithms. We showed the performance with simulation results to demonstrate the solutions proposed are feasible. For future expectations, there are some possible directions. When applying traditional navigation algorithms, for example, biologically inspired algorithms, we have to pay attention to the limitations of the environment. However, high-tech algorithms sometimes are not computationally friendly. How to combine them together so as to fulfill the tasks perfectly while the computational e ciency is not too high is a worthy topic. In addition, extending the obstacle avoidance al- gorithms to more competitive situations, such as applying to autonomous UAVs, is also being considered. Moreover, for cooperation among multi robots, which could be regarded as Network Control System (NCS), the issues, such as how to complete their respective tasks, how to choose the optimal routes for them are worth attention by researchers. All in all, there is still a long way to go for the development of navigation and control of mobile robots. Despite this, we believe we do not need to wait for too long time to see the revolution of robots

Development of a Healthcare Software System for the Elderly

This research focused on the implementation of a reliable intelligent fall detection system so as to reduce accidental falls among the elderly people. A video-based detection system was used because it preserved privacy while monitoring the activities of the senior citizens. Another advantage of the video-based system is that the senior citizens are able to move freely without experiencing any hassles in wearing them as opposed to portable fall detection sensors so that they can have a more independent and happy life. A scientific research method was employed to improve the existing fall detection systems in terms of reliability and accuracy. This thesis consists of four stages where the first stage reviews the literature on the current fall detection systems, the second stage investigates the various algorithms of these existing fall detection systems, the third stage describes the proposed fall detection algorithm in detecting falls using two distinct approaches. The first approach deals with the use of specific features of the silhouette, an extracted binary map obtained from the subtraction of the foreground from the background, to determine the fall angle (FA), the bounding box (BB) ratio, the Hidden Markov Models (HMM) and the combination of FA, BB, and HMM. The second approach used is the neural network approach which is incorporated in the algorithm to identify a predetermined set of situations such as praying, sitting, standing, bending, kneeling, and lying down. The fourth stage involves the evalua- tion of the developed video-based fall detection system using different metrics which measure sensitivity (i.e. the capacity of the fall detection system to detect as well as declare a fall) and specificity (i.e. the capacity of the algorithm to detect only falls) of this algorithm. The video camera was properly positioned to avoid any occluding objects and also to cover a certain range of motion of the stunt participants performing the falls. The silhouette is extracted using an approximate median filtering approach and the threshold criteria value of 30 pixels was used. Morphological filtering methods that were dilation and erosion were used to remove any spurious noises from the extracted image prior to subsequent feature analysis. Then, this extracted silhouette was scaled and quantised using 8 bits/pixel and compared to the set of predetermined scenarios using a neural network of perceptrons. This neural network was trained based on various situations and the falls of the participants which represent inputs to the neural network algorithm during the neural learning process. In this research study, the built neural network consisted of 600 inputs, as well as 10 neurons in the hidden layer together with 7 distinct outputs which represent the set of predefined situations. Furthermore, an alarm generation algorithm was included in the fall detection algorithm such that there were three states that were STATE NULL (set at 0), STATE LYING (set at 1) and STATE ALL OTHERS (set at 2) and the initial alarm count was set to 90 frames (meaning 3 seconds of recorded consecutive images at 30 frames per second). Therefore, an alarm was generated only when the in-built counter surpassed this threshold of 90 frames to signal that a fall occurred. Following the evaluation stage, it was found that the combination of the first approach fall detection algorithm method (fall angle, bounding box, and hidden Markov) was 89% with specificity and 84.2% with sensitivity which is better than individual performance. Moreover, it was found that the second approach fall detection algorithm method (neural network performance) 94.3% of the scenarios were successfully classified whereby the specificity of the developed algorithm was determined to be 94.8% and the sensitivity was 93.8% which altogether show a promising overall performance of the fall detection video-based intelligent system. Moreover, the developed fall detection system were tested using two types of handicaps such as limping and stumbling stunt participants to observe how well this detection algorithm can detect falls as in the practical situations encountered or present in elderly people. In these cases it was found that about 90.2% of the falls were detected which showed still that the developed algorithm was quite robust and reliable subjected to these two physical handicaps motion behaviours

Visual / acoustic detection and localisation in embedded systems

©Cranfield UniversityThe continuous miniaturisation of sensing and processing technologies is increasingly offering a variety of embedded platforms, enabling the accomplishment of a broad range of tasks using such systems. Motivated by these advances, this thesis investigates embedded detection and localisation solutions using vision and acoustic sensors. Focus is particularly placed on surveillance applications using sensor networks. Existing vision-based detection solutions for embedded systems suffer from the sensitivity to environmental conditions. In the literature, there seems to be no algorithm able to simultaneously tackle all the challenges inherent to real-world videos. Regarding the acoustic modality, many research works have investigated acoustic source localisation solutions in distributed sensor networks. Nevertheless, it is still a challenging task to develop an ecient algorithm that deals with the experimental issues, to approach the performance required by these systems and to perform the data processing in a distributed and robust manner. The movement of scene objects is generally accompanied with sound emissions with features that vary from an environment to another. Therefore, considering the combination of the visual and acoustic modalities would offer a significant opportunity for improving the detection and/or localisation using the described platforms. In the light of the described framework, we investigate in the first part of the thesis the use of a cost-effective visual based method that can deal robustly with the issue of motion detection in static, dynamic and moving background conditions. For motion detection in static and dynamic backgrounds, we present the development and the performance analysis of a spatio- temporal form of the Gaussian mixture model. On the other hand, the problem of motion detection in moving backgrounds is addressed by accounting for registration errors in the captured images. By adopting a robust optimisation technique that takes into account the uncertainty about the visual measurements, we show that high detection accuracy can be achieved. In the second part of this thesis, we investigate solutions to the problem of acoustic source localisation using a trust region based optimisation technique. The proposed method shows an overall higher accuracy and convergence improvement compared to a linear-search based method. More importantly, we show that through characterising the errors in measurements, which is a common problem for such platforms, higher accuracy in the localisation can be attained. The last part of this work studies the different possibilities of combining visual and acoustic information in a distributed sensors network. In this context, we first propose to include the acoustic information in the visual model. The obtained new augmented model provides promising improvements in the detection and localisation processes. The second investigated solution consists in the fusion of the measurements coming from the different sensors. An evaluation of the accuracy of localisation and tracking using a centralised/decentralised architecture is conducted in various scenarios and experimental conditions. Results have shown the capability of this fusion approach to yield higher accuracy in the localisation and tracking of an active acoustic source than by using a single type of data