Vehicle-Rear: A New Dataset to Explore Feature Fusion for Vehicle Identification Using Convolutional Neural Networks

This work addresses the problem of vehicle identification through non-overlapping cameras. As our main contribution, we introduce a novel dataset for vehicle identification, called Vehicle-Rear, that contains more than three hours of high-resolution videos, with accurate information about the make, model, color and year of nearly 3,000 vehicles, in addition to the position and identification of their license plates. To explore our dataset we design a two-stream CNN that simultaneously uses two of the most distinctive and persistent features available: the vehicle's appearance and its license plate. This is an attempt to tackle a major problem: false alarms caused by vehicles with similar designs or by very close license plate identifiers. In the first network stream, shape similarities are identified by a Siamese CNN that uses a pair of low-resolution vehicle patches recorded by two different cameras. In the second stream, we use a CNN for OCR to extract textual information, confidence scores, and string similarities from a pair of high-resolution license plate patches. Then, features from both streams are merged by a sequence of fully connected layers for decision. In our experiments, we compared the two-stream network against several well-known CNN architectures using single or multiple vehicle features. The architectures, trained models, and dataset are publicly available at https://github.com/icarofua/vehicle-rear

Vision-based traffic surveys in urban environments

This paper presents a state-of-the-art, vision-based vehicle detection and type classification to perform traffic surveys from a roadside closed-circuit television camera. Vehicles are detected using background subtraction based on a Gaussian mixture model that can cope with vehicles that become stationary over a significant period of time. Vehicle silhouettes are described using a combination of shape and appearance features using an intensity-based pyramid histogram of orientation gradients (HOG). Classification is performed using a support vector machine, which is trained on a small set of hand-labeled silhouette exemplars. These exemplars are identified using a model-based preclassifier that utilizes calibrated images mapped by Google Earth to provide accurately surveyed scene geometry matched to visible image landmarks. Kalman filters track the vehicles to enable classification by majority voting over several consecutive frames. The system counts vehicles and separates them into four categories: car, van, bus, and motorcycle (including bicycles). Experiments with real-world data have been undertaken to evaluate system performance and vehicle detection rates of 96.45% and classification accuracy of 95.70% have been achieved on this data.The authors gratefully acknowledge the Royal Borough of Kingston for providing the video data. S.A. Velastin is grateful to funding received from the Universidad Carlos III de Madrid, the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement nº 600371, el Ministerio de Economía y Competitividad (COFUND2013-51509) and Banco Santander

Sistema de monitoramento, contagem e classificação de fluxo de veículos usando Redes Neurais Convolucionais

O uso de Redes Neurais Convolucionais na detecção, classificação e contagem de veículos é uma opção de baixo custo e alta eficiência na modernização de processos corriqueiros em serviços relacionados ao fluxo de veículos em estradas de rodagem. Este projeto apresenta os detalhes do projeto, análise e implementação de um sistema para identificação, classificação e contagem de veículos em estradas de rodagem com uso de um detector de objetos baseado no YOLOv4, que é baseado em uma rede neural convolucional (CNN) e de um rastreador de objetos, também baseado em uma CNN, que utiliza uma variação de um algoritmo denominado Simple Online and Realtime Tracking algorithm - DeepSORT. Resultados preliminares mostraram que o sistema desenvolvido obteve um RMSE normalizado de 2.67% em um contexto de aplicação simples, sendo capaz de detectar e rastrear os veículos em intersecções e rodovias em imagens com cenas claras e sem obstáculos, possibilitando a contabilização e a registro das rotas. Os próximos passos do trabalho incluem aperfei- çoamentos para incrementar a viabilidade do sistema frente à obstáculos e oclusões ou certos eventos quando a câmera que obtém as imagens não possui uma vista clara e uma boa resolução, visto que os resultados obtidos em exemplos desse tipo apresentaram uma redução de 50% na sobre contagem de carros com os modelos re-treinados desenvolvidos no projeto.The use of Convolutional Neural Networks on detection, classification and counting of vehicles is a low-cost and high-efficiency option for modernizing daily activities in public agencies and entities that are responsible for the vehicle flow in roadways. This project presents the details on the planning, analysis and implementation of a system to identify, classify and count vehicles in roadways, with the aid of an object detector based on YOLOv4, which is based on a convolutional neural network (CNN) and an object tracker, which is also based on a CNN, which uses a variation of an algorithm denominated Simple Online and Realtime Tracking algorithm - DeepSORT. Preliminary results show that the developed system achieves a normalized RMSE of 2.67% in a simple appliction scenario, being able to detect and track vehicles in intersections and roadways in images of clear scenes without any obstacles, making it possible to account and register the routes. Next steps of such project may include further improvements to increment the system robustness on the presence of obstacles and occlusions, or events where the camera that collects the images does not provide a clear vision and has low resolution, since the achieved results on such scenarios showed a reduction of 50% in the overcounting of cars with the re-trained models developed during the project

Efficient multi-camera vehicle detection, tracking, and identification in a tunnel surveillance application

This paper presents an integrated solution for the problem of detecting, tracking and identifying vehicles in a tunnel surveillance application, taking into account practical constraints including real-time operation, poor imaging conditions, and a decentralized architecture. Vehicles are followed through the tunnel by a network of non-overlapping cameras. They are detected and tracked in each camera and then identified, i.e. matched to any of the vehicles detected in the previous camera (s). To limit the computational load, we propose to reuse the same set of Haar-features for each of these steps. For the detection, we use an AdaBoost cascade. Here we introduce a composite confidence score, integrating information from all stages of the cascade. A subset of the features used for detection is then selected, optimizing for the identification problem. This results in a compact binary 'vehicle fingerprint', requiring minimal bandwidth. Finally, we show that the same subset of features can also be used effectively for tracking. This Haar-features based 'tracking-by-identification' yields surprisingly good results on standard datasets, without the need to update the model online. The general multi-camera framework is validated using three tunnel surveillance videos. © 2012 Elsevier Inc. All rights reserved.Reyes R.-C., Tuytelaars T., Van Gool L., ''Efficient multi-camera vehicle detection, tracking, and identification in a tunnel surveillance application'', Computer vision and image understanding, vol. 116, no. 6, pp. 742-753, June 2012.status: publishe

Computer Vision Based Structural Identification Framework for Bridge Health Mornitoring

The objective of this dissertation is to develop a comprehensive Structural Identification (St-Id) framework with damage for bridge type structures by using cameras and computer vision technologies. The traditional St-Id frameworks rely on using conventional sensors. In this study, the collected input and output data employed in the St-Id system are acquired by series of vision-based measurements. The following novelties are proposed, developed and demonstrated in this project: a) vehicle load (input) modeling using computer vision, b) bridge response (output) using full non-contact approach using video/image processing, c) image-based structural identification using input-output measurements and new damage indicators. The input (loading) data due vehicles such as vehicle weights and vehicle locations on the bridges, are estimated by employing computer vision algorithms (detection, classification, and localization of objects) based on the video images of vehicles. Meanwhile, the output data as structural displacements are also obtained by defining and tracking image key-points of measurement locations. Subsequently, the input and output data sets are analyzed to construct novel types of damage indicators, named Unit Influence Surface (UIS). Finally, the new damage detection and localization framework is introduced that does not require a network of sensors, but much less number of sensors. The main research significance is the first time development of algorithms that transform the measured video images into a form that is highly damage-sensitive/change-sensitive for bridge assessment within the context of Structural Identification with input and output characterization. The study exploits the unique attributes of computer vision systems, where the signal is continuous in space. This requires new adaptations and transformations that can handle computer vision data/signals for structural engineering applications. This research will significantly advance current sensor-based structural health monitoring with computer-vision techniques, leading to practical applications for damage detection of complex structures with a novel approach. By using computer vision algorithms and cameras as special sensors for structural health monitoring, this study proposes an advance approach in bridge monitoring through which certain type of data that could not be collected by conventional sensors such as vehicle loads and location, can be obtained practically and accurately