Advanced framework for microscopic and lane‐level macroscopic traffic parameters estimation from UAV video

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/166282/1/itr2bf00873.pd

AURORA:autonomous real-time on-board video analytics

In this paper, we describe the design and implementation of a small light weight, low-cost and power-efficient payload system for the use in unmanned aerial vehicles (UAVs). The primary application of the payload system is that of performing real-time autonomous objects detection and tracking in the videos taken from a UAV camera. The implemented objects detection and tracking algorithms utilise Recursive Density Estimation (RDE) and Evolving Local Means (ELM) clustering to perform detection and tracking moving objects. Furthermore, experiments are presented which demonstrate that the introduced system is able to detect by on-board processing any moving objects from a UAV and start tracking them in real-time while at the same time sending important data only to a control station located on the ground

Implementation of Unmanned aerial vehicles (UAVs) for assessment of transportation infrastructure - Phase II

Technological advances in unmanned aerial vehicle (UAV) technologies continue to enable these tools to become easier to use, more economical, and applicable for transportation-related operations, maintenance, and asset management while also increasing safety and decreasing cost. This Phase 2 project continued to test and evaluate five main UAV platforms with a combination of optical, thermal, and lidar sensors to determine how to implement them into MDOT workflows. Field demonstrations were completed at bridges, a construction site, road corridors, and along highways with data being processed and analyzed using customized algorithms and tools. Additionally, a cost-benefit analysis was conducted, comparing manual and UAV-based inspection methods. The project team also gave a series of technical demonstrations and conference presentations, enabling outreach to interested audiences who gained understanding of the potential implementation of this technology and the advanced research that MDOT is moving to implementation. The outreach efforts and research activities performed under this contract demonstrated how implementing UAV technologies into MDOT workflows can provide many benefits to MDOT and the motoring public; such as advantages in improved cost-effectiveness, operational management, and timely maintenance of Michigan’s transportation infrastructure

Automatic vehicle detection and tracking in aerial video

This thesis is concerned with the challenging tasks of automatic and real-time vehicle detection and tracking from aerial video. The aim of this thesis is to build an automatic system that can accurately localise any vehicles that appear in aerial video frames and track the target vehicles with trackers. Vehicle detection and tracking have many applications and this has been an active area of research during recent years; however, it is still a challenge to deal with certain realistic environments. This thesis develops vehicle detection and tracking algorithms which enhance the robustness of detection and tracking beyond the existing approaches. The basis of the vehicle detection system proposed in this thesis has different object categorisation approaches, with colour and texture features in both point and area template forms. The thesis also proposes a novel Self-Learning Tracking and Detection approach, which is an extension to the existing Tracking Learning Detection (TLD) algorithm. There are a number of challenges in vehicle detection and tracking. The most difficult challenge of detection is distinguishing and clustering the target vehicle from the background objects and noises. Under certain conditions, the images captured from Unmanned Aerial Vehicles (UAVs) are also blurred; for example, turbulence may make the vehicle shake during flight. This thesis tackles these challenges by applying integrated multiple feature descriptors for real-time processing. In this thesis, three vehicle detection approaches are proposed: the HSV-GLCM feature approach, the ISM-SIFT feature approach and the FAST-HoG approach. The general vehicle detection approaches used have highly flexible implicit shape representations. They are based on training samples in both positive and negative sets and use updated classifiers to distinguish the targets. It has been found that the detection results attained by using HSV-GLCM texture features can be affected by blurring problems; the proposed detection algorithms can further segment the edges of the vehicles from the background. Using the point descriptor feature can solve the blurring problem, however, the large amount of information contained in point descriptors can lead to processing times that are too long for real-time applications. So the FAST-HoG approach combining the point feature and the shape feature is proposed. This new approach is able to speed up the process that attains the real-time performance. Finally, a detection approach using HoG with the FAST feature is also proposed. The HoG approach is widely used in object recognition, as it has a strong ability to represent the shape vector of the object. However, the original HoG feature is sensitive to the orientation of the target; this method improves the algorithm by inserting the direction vectors of the targets. For the tracking process, a novel tracking approach was proposed, an extension of the TLD algorithm, in order to track multiple targets. The extended approach upgrades the original system, which can only track a single target, which must be selected before the detection and tracking process. The greatest challenge to vehicle tracking is long-term tracking. The target object can change its appearance during the process and illumination and scale changes can also occur. The original TLD feature assumed that tracking can make errors during the tracking process, and the accumulation of these errors could cause tracking failure, so the original TLD proposed using a learning approach in between the tracking and the detection by adding a pair of inspectors (positive and negative) to constantly estimate errors. This thesis extends the TLD approach with a new detection method in order to achieve multiple-target tracking. A Forward and Backward Tracking approach has been proposed to eliminate tracking errors and other problems such as occlusion. The main purpose of the proposed tracking system is to learn the features of the targets during tracking and re-train the detection classifier for further processes. This thesis puts particular emphasis on vehicle detection and tracking in different extreme scenarios such as crowed highway vehicle detection, blurred images and changes in the appearance of the targets. Compared with currently existing detection and tracking approaches, the proposed approaches demonstrate a robust increase in accuracy in each scenario

Gaussian mixture model classifiers for detection and tracking in UAV video streams.

Masters Degree. University of KwaZulu-Natal, Durban.Manual visual surveillance systems are subject to a high degree of human-error and operator fatigue. The automation of such systems often employs detectors, trackers and classifiers as fundamental building blocks. Detection, tracking and classification are especially useful and challenging in Unmanned Aerial Vehicle (UAV) based surveillance systems. Previous solutions have addressed challenges via complex classification methods. This dissertation proposes less complex Gaussian Mixture Model (GMM) based classifiers that can simplify the process; where data is represented as a reduced set of model parameters, and classification is performed in the low dimensionality parameter-space. The specification and adoption of GMM based classifiers on the UAV visual tracking feature space formed the principal contribution of the work. This methodology can be generalised to other feature spaces. This dissertation presents two main contributions in the form of submissions to ISI accredited journals. In the first paper, objectives are demonstrated with a vehicle detector incorporating a two stage GMM classifier, applied to a single feature space, namely Histogram of Oriented Gradients (HoG). While the second paper demonstrates objectives with a vehicle tracker using colour histograms (in RGB and HSV), with Gaussian Mixture Model (GMM) classifiers and a Kalman filter. The proposed works are comparable to related works with testing performed on benchmark datasets. In the tracking domain for such platforms, tracking alone is insufficient. Adaptive detection and classification can assist in search space reduction, building of knowledge priors and improved target representations. Results show that the proposed approach improves performance and robustness. Findings also indicate potential further enhancements such as a multi-mode tracker with global and local tracking based on a combination of both papers

Exploring the Use of Drones for Conducting Traffic Mobility and Safety Studies

ABSTRACT Advanced traffic data collection methods, including the application of aerial sensors (drones) as traffic data collectors, can provide real-time traffic information more efficiently, effectively, and safely than traditional methods. Traffic trajectory data like vehicles’ coordinates and point timestamps are challenging to obtain at intersections using traditional field survey methods. The coordinates and timestamps crucial in calculating trajectories can be obtained using drones and their particular integrated software. Thus, this study explores the use of unmanned aerial systems (UAS), particularly tethered drones, to obtain traffic parameters for traffic mobility and safety studies at an unsignalized intersection in Tallahassee, Florida. Tethered drones provided more flexibility in heights and angles and collected data over a relatively larger space needed for the proposed approach. Turning movement counts, gap study, speed study, and Level of Service (LOS) analysis for the stated intersection were the traffic studies conducted in this research. The turning movements were counted through ArcGIS Pro. From the drone footages, the gap study followed by the LOS analysis was carried out. A speed algorithm was developed to calculate speed during a speed study. Based on the results, the intersection operates under capacity with LOS B during the time. Also, the results indicated that the through movement traffic tends to slow down as they approach the intersection while south-bound right and east-bound left-turning traffic increase their speeds as they make a turn. Accuracy assessment was done by comparing the drone footages with the results displayed in ArcGIS software. The drone’s data collection was 100% accurate in traffic movement counting and 96% accurate in traffic movement classification. The level of accuracy is sufficient compared to other advanced traffic data collection methods. In this study, safety was assessed by the surrogate safety measures (SSMs). SSMs can be the viable alternatives for locations with insufficient historical data and indicate potential future conflicts between roadway users. The surrogate measures used in this study include the Time to Collision (TTC), Deceleration-based Surrogate Safety Measure (DSSM), and Post-encroachment Time (PET). TTC and DSSM were used for rear-end conflicts, while PET was used to evaluate cross conflicts and other conflicts such as sideswipes. The number of potential conflicts obtained in a one-hour study period was around 20 per 1000 vehicles traversing the intersection. The number of potential conflicts in one non-peak hour may indicate a safety problem associated with the intersection. This study’s findings can help develop appropriate guidelines and recommendations to transportation agencies in evaluating and justifying the feasibility of using tethered drones as safer and cheaper data collection alternatives while significantly improving intersection safety and operations

Advances and Applications of Computer Vision Techniques in Vehicle Trajectory Generation and Surrogate Traffic Safety Indicators

The application of Computer Vision (CV) techniques massively stimulates microscopic traffic safety analysis from the perspective of traffic conflicts and near misses, which is usually measured using Surrogate Safety Measures (SSM). However, as video processing and traffic safety modeling are two separate research domains and few research have focused on systematically bridging the gap between them, it is necessary to provide transportation researchers and practitioners with corresponding guidance. With this aim in mind, this paper focuses on reviewing the applications of CV techniques in traffic safety modeling using SSM and suggesting the best way forward. The CV algorithm that are used for vehicle detection and tracking from early approaches to the state-of-the-art models are summarized at a high level. Then, the video pre-processing and post-processing techniques for vehicle trajectory extraction are introduced. A detailed review of SSMs for vehicle trajectory data along with their application on traffic safety analysis is presented. Finally, practical issues in traffic video processing and SSM-based safety analysis are discussed, and the available or potential solutions are provided. This review is expected to assist transportation researchers and engineers with the selection of suitable CV techniques for video processing, and the usage of SSMs for various traffic safety research objectives

Towards Collection of Smart City Data for Cloud Storage Using UAVs

The article describes the methodology and process of collecting smart city data using drones for cities that do not have a sufficiently developed infrastructure. For storage and subsequent analysis of data, a cloud server is required; TUC DriveCloud is presented as an example of such a server in the article. Traffic analysis and building inspection are described as examples of drone data collection tasks. The advantages and disadvantages of collecting data using a thermal imaging camera are also discussed using the example of the problem of detecting and tracking the movement of people