Automatic vehicle counting area creation based on vehicle deep learning detection and DBSCAN

Deep learning and high-performance computing have augmented and speed-up the scope of video-based vehicles' massive counting. The automatic vehicle counts result from the detection and tracking of the vehicles in certain areas or Regions of Interest (ROI). In this paper, we propose a technique to create a counting area with different traffic-flow directions based on YOLO and DBSCAN You Only Look Once version five (YOLOv5) and Density-Based Spatial Clustering of Applications with Noise (DBSCAN). We compare the performance of the method against the manually counted ground truth. The proposed method showed that it is possible to generate the ROIs (counting areas) according to the traffic flow using deep learning techniques with relatively good accuracy (less than 5 % error). These results are promising but we need to explore the limits of this method with more street-view configurations, time and other detection and tracking algorithms, and in an HPC environment.Peer ReviewedPostprint (author's final draft

Vision-Based Incoming Traffic Estimator Using Deep Neural Network on General Purpose Embedded Hardware

Traffic management is a serious problem in many cities around the world. Even the suburban areas are now experiencing regular traffic congestion. Inappropriate traffic control wastes fuel, time, and the productivity of nations. Though traffic signals are used to improve traffic flow, they often cause problems due to inappropriate or obsolete timing that does not tally with the actual traffic intensity at the intersection. Traffic intensity determination based on statistical methods only gives the average intensity expected at any given time. However, to control traffic accurately, it is required to know the real-time traffic intensity. In this research, image processing and machine learning have been used to estimate actual traffic intensity in real time. General-purpose electronic hardware has been used for in-situ image processing based on the edge-detection method. A deep neural network (DNN) was trained to infer traffic intensity in each image in real time. The trained DNN estimated traffic intensity accurately in 90% of the real-time images during road tests. The electronic system was implemented on a Raspberry Pi single-board computer; hence, it is cost-effective for large-scale deployment.Comment: 6 pages, 11 figures, journa

ProGroTrack: Deep Learning-Assisted Tracking of Intracellular Protein Growth Dynamics

Accurate tracking of cellular and subcellular structures, along with their dynamics, plays a pivotal role in understanding the underlying mechanisms of biological systems. This paper presents a novel approach, ProGroTrack, that combines the You Only Look Once (YOLO) and ByteTrack algorithms within the detection-based tracking (DBT) framework to track intracellular protein nanostructures. Focusing on iPAK4 protein fibers as a representative case study, we conducted a comprehensive evaluation of YOLOv5 and YOLOv8 models, revealing the superior performance of YOLOv5 on our dataset. Notably, YOLOv5x achieved an impressive mAP50 of 0.839 and F-score of 0.819. To further optimize detection capabilities, we incorporated semi-supervised learning for model improvement, resulting in enhanced performances in all metrics. Subsequently, we successfully applied our approach to track the growth behavior of iPAK4 protein fibers, revealing their two distinct growth phases consistent with a previously reported kinetic model. This research showcases the promising potential of our approach, extending beyond iPAK4 fibers. It also offers a significant advancement in precise tracking of dynamic processes in live cells, and fostering new avenues for biomedical research

Visual servoing on wheels: robust robot orientation estimation in remote viewpoint control

This work proposes a fast deployment pipeline for visually-servoed robots which does not assume anything about either the robot - e.g. sizes, colour or the presence of markers - or the deployment environment. Specifically, we apply a learning based approach to reliably estimate the pose of a robot in the image frame of a 2D camera upon which a visual servoing control system can be deployed. To alleviate the time-consuming process of labelling image data, we propose a weakly supervised pipeline that can produce a vast amount of data in a small amount of time. We evaluate our approach on a dataset of remote camera images captured in various indoor environments demonstrating high tracking performances when integrated into a fully-autonomous pipeline with a simple controller. With this, we then analyse the data requirement of our approach, showing how it is possible to deploy a new robot in a new environment in fewer than 30.00 min

Vision-based vehicle detection and tracking in intelligent transportation system

This thesis aims to realize vision-based vehicle detection and tracking in the Intelligent Transportation System. First, it introduces the methods for vehicle detection and tracking. Next, it establishes the sensor fusion framework of the system, including dynamic model and sensor model. Then, it simulates the traffic scene at a crossroad by a driving simulator, where the research target is one single car, and the traffic scene is ideal. YOLO Neural Network is applied to the image sequence for vehicle detection. Kalman filter method, extended Kalman filter method, and particle filter method are utilized and compared for vehicle tracking. The Following part is the practical experiment where there are multiple vehicles at the same time, and the traffic scene is in real life with various interference factors. YOLO Neural Network combined with OpenCV is adopted to realize real-time vehicle detection. Kalman filter and extended Kalman filter are applied for vehicle tracking; an identification algorithm is proposed to solve the occlusion of the vehicles. The effects of process noise as well as measurement noise are analysed using variable-controlling approach. Additionally, perspective transformation is illustrated and implemented to transfer the coordinates from the image plane to the ground plane. If the vision-based vehicle detection and tracking can be realized and popularized in daily lives, vehicle information can be shared among infrastructures, vehicles, and users, so as to build interactions inside the Intelligent Transportation System

Prediction of Pedestrians\u27 Red Light Violations Using Deep Learning

Pedestrians are regarded as Vulnerable Road Users (VRUs). Each year, thousands of pedestrians\u27 deaths are caused by traffic crashes, which take up 16% of the total road fatalities and injuries in the U.S. (FHWA, 2018). Crashes can happen if there are interactions between VRUs and motorized transportation. And pedestrians\u27 unexpected crossings, such as red-light violations at the signalized intersections, would expose them to motorized transportation and cause potential collisions. This thesis is intended to predict the pedestrians\u27 red-light violation behaviors at the signalized crosswalks based on an LSTM (Long Short-term Memory) neural network. With video data collected from real traffic scenes, it is found that pedestrians that crossed during the red-light periods are more in danger of being struck by vehicles, from the perspective of Surrogate Safety Measures (SSMs). Pedestrians\u27 features are generated using computer vision techniques. An LSTM model is used to predict pedestrians\u27 red-light violations using these features. The experiment results at one signalized intersection show that the LSTM model achieves an accuracy of 91.6%. Drivers can be more prepared for these unexpected crossing pedestrians if the model is to be implemented in the vehicle-to-infrastructure (V2I) communication system

Sistema de monitoramento, contagem e classificação de fluxo de veículos usando Redes Neurais Convolucionais

O uso de Redes Neurais Convolucionais na detecção, classificação e contagem de veículos é uma opção de baixo custo e alta eficiência na modernização de processos corriqueiros em serviços relacionados ao fluxo de veículos em estradas de rodagem. Este projeto apresenta os detalhes do projeto, análise e implementação de um sistema para identificação, classificação e contagem de veículos em estradas de rodagem com uso de um detector de objetos baseado no YOLOv4, que é baseado em uma rede neural convolucional (CNN) e de um rastreador de objetos, também baseado em uma CNN, que utiliza uma variação de um algoritmo denominado Simple Online and Realtime Tracking algorithm - DeepSORT. Resultados preliminares mostraram que o sistema desenvolvido obteve um RMSE normalizado de 2.67% em um contexto de aplicação simples, sendo capaz de detectar e rastrear os veículos em intersecções e rodovias em imagens com cenas claras e sem obstáculos, possibilitando a contabilização e a registro das rotas. Os próximos passos do trabalho incluem aperfei- çoamentos para incrementar a viabilidade do sistema frente à obstáculos e oclusões ou certos eventos quando a câmera que obtém as imagens não possui uma vista clara e uma boa resolução, visto que os resultados obtidos em exemplos desse tipo apresentaram uma redução de 50% na sobre contagem de carros com os modelos re-treinados desenvolvidos no projeto.The use of Convolutional Neural Networks on detection, classification and counting of vehicles is a low-cost and high-efficiency option for modernizing daily activities in public agencies and entities that are responsible for the vehicle flow in roadways. This project presents the details on the planning, analysis and implementation of a system to identify, classify and count vehicles in roadways, with the aid of an object detector based on YOLOv4, which is based on a convolutional neural network (CNN) and an object tracker, which is also based on a CNN, which uses a variation of an algorithm denominated Simple Online and Realtime Tracking algorithm - DeepSORT. Preliminary results show that the developed system achieves a normalized RMSE of 2.67% in a simple appliction scenario, being able to detect and track vehicles in intersections and roadways in images of clear scenes without any obstacles, making it possible to account and register the routes. Next steps of such project may include further improvements to increment the system robustness on the presence of obstacles and occlusions, or events where the camera that collects the images does not provide a clear vision and has low resolution, since the achieved results on such scenarios showed a reduction of 50% in the overcounting of cars with the re-trained models developed during the project

Scene and crowd analysis using synthetic data generation with 3D quality improvements and deep network architectures

In this thesis, a scene analysis mainly focusing on vision-based techniques have been explored. The vision-based scene analysis techniques have a wide range of applications from surveillance, security to agriculture. A vision sensor can provide rich information about the environment such as colour, depth, shape, size and much more. This information can be further processed to have an in-depth knowledge of the scene such as type of environment, objects and distances. Hence, this thesis covers initially the background on human detection in particular pedestrian and crowd detection methods and introduces various vision-based techniques used in human detection. Followed by a detailed analysis of the use of synthetic data to improve the performance of state-of-the-art Deep Learning techniques and a multi-purpose synthetic data generation tool is proposed. The tool is a real-time graphics simulator which generates multiple types of synthetic data applicable for pedestrian detection, crowd density estimation, image segmentation, depth estimation, and 3D pose estimation. In the second part of the thesis, a novel technique has been proposed to improve the quality of the synthetic data. The inter-reflection also known as global illumination is a naturally occurring phenomena and is a major problem for 3D scene generation from an image. Thus, the proposed methods utilised a reverted ray-tracing technique to reduce the effect of inter-reflection problem and increased the quality of generated data. In addition, a method to improve the quality of the density map is discussed in the following chapter. The density map is the most commonly used technique to estimate crowds. However, the current procedure used to generate the map is not content-aware i.e., density map does not highlight the humans’ heads according to their size in the image. Thus, a novel method to generate a content-aware density map was proposed and demonstrated that the use of such maps can elevate the performance of an existing Deep Learning architecture. In the final part, a Deep Learning architecture has been proposed to estimate the crowd in the wild. The architecture tackled the challenging aspect such as perspective distortion by implementing several techniques like pyramid style inputs, scale aggregation method and self-attention mechanism to estimate a crowd density map and achieved state-of-the-art results at the time