Car make and model recognition system using rear-lamp features and convolutional neural networks

Recognizing cars based on their features is a difficult task. We propose a solution that uses a convolutional neural network (CNN) and image binarization method for car make and model classification. Unlike many previous works in this area, we use a feature extraction method combined with a binarization method. In the first stage of the pre-processing part we normalize and change the size of an image. The image is then used to recognize where the rear-lamps are placed on the image. We extract the region and use the image binarization method. The binarized image is used as input to the CNN network that finds the features of a specific car model. We have tested the combinations of three different neural network architectures and eight binarization methods. The convolutional neural network with parameters of the highest quality metrics value is used to find the characteristics of the rear lamps on the binary image. The convolutional network is tested with four different gradient algorithms. We have tested the method on two data sets which differ in the way the images were taken. Each data set consists of three subsets of the same car, but is scaled to different image dimensions. Compared to related works that are based on CNN, we use rear view images in different position and light exposure. The proposed method gives better results compared to most available methods. It is also less complex, and faster to train compared to other methods. The proposed approach achieves an average accuracy of 93,9% on the first data set and 84,5% on the second set

Real-time classification of vehicle types within infra-red imagery.

Real-time classification of vehicles into sub-category types poses a significant challenge within infra-red imagery due to the high levels of intra-class variation in thermal vehicle signatures caused by aspects of design, current operating duration and ambient thermal conditions. Despite these challenges, infra-red sensing offers significant generalized target object detection advantages in terms of all-weather operation and invariance to visual camouflage techniques. This work investigates the accuracy of a number of real-time object classification approaches for this task within the wider context of an existing initial object detection and tracking framework. Specifically we evaluate the use of traditional feature-driven bag of visual words and histogram of oriented gradient classification approaches against modern convolutional neural network architectures. Furthermore, we use classical photogrammetry, within the context of current target detection and classification techniques, as a means of approximating 3D target position within the scene based on this vehicle type classification. Based on photogrammetric estimation of target position, we then illustrate the use of regular Kalman filter based tracking operating on actual 3D vehicle trajectories. Results are presented using a conventional thermal-band infra-red (IR) sensor arrangement where targets are tracked over a range of evaluation scenarios

Integrating water-energy-nexus in carbon footprint analysis in water utility company

The purpose of this paper is to highlight the water-energy-nexus within the context of carbon footprint methodology and water utility industry. In particular, the carbon management for water utility industry is crucial in reducing carbon emission within the upstream water distribution system. The concept of water-energy nexus alone however can be misleading due to exclusion of indirect and embodied energy involved in the water production. The study highlights the total energy use within water supply system as well as embedded carbon emission through carbon footprint methodology. The case study approach is used as a research method. The carbon footprint analysis includes data collection from water utility company; and data identification of direct and indirect carbon emission from corporation operation. The result indicates that the indirect and embodied energy may not be significant in certain operation area but the energy use may be ambiguous when these elements are excluded. Integrating carbon footprint methodology within the water supply system can improve the understanding on water-energy-nexus when direct and indirect energy use is included in the analysis. This paper aims to benefit academics, government agencies and particularly water utility companies in integrating carbon footprint analysis in water production

A study of the general characteristics of Serial fusion of classifiers

Con el continuo avance tecnológico de las últimas décadas, el volumen y la precisión de los datos a manejar se ha visto incrementado de forma directa, haciéndose necesario a su vez el estudio y el diseño de métodos que permitan analizarlos y clasificarlos de forma automática. Con este cometido se han desarrollado multitud de clasificadores siguiendo, desde modelos matemáticos a modelos bioinspirados, en muchos casos con un desempeño mucho mayor que aceptable en escenarios concretos. Con el fin de mejorar el rendimiento de estos clasificadores, algunas investigaciones, sobre todo en las últimas dos décadas, se han centrado en el estudio de modelos formados por la unión de varios clasificadores individuales, creando una estructura común que proporciona una solución única al problema, hallada a partir de los resultados obtenidos por los distintos clasificadores que la forman. Las estructuras más comúnmente estudiadas conectan cada uno de los módulos individuales en paralelo entre sí, calculando la salida común una vez que todos los clasificadores individuales han tomado un valor en su salida. Este tipo de combinación de clasificadores se ha probado en diversos estudios como una alternativa con una mejor precisión frente a los clasificadores individuales, siendo además más robustos frente a errores puntuales e intentos de falsificación o suplantación, como pueden darse en sistemas de identificación mediante reconocimiento de patrones biométricos. Aun así, al combinar varios clasificadores en paralelo se generan algunas desventajas, como la necesidad de desarrollar cada clasificador por separado para cumplir con unas necesidades específicas dentro del conjunto, el aumento de recursos necesarios para la ejecución de cada uno de los módulos que componen la estructura al mismo tiempo, o el aumento del tiempo medio necesario para ofrecer un resultado, limitado en este caso por el clasificador más lento. Por otro lado, las estructuras de clasificadores en serie o cascada buscan un equilibrio entre el tiempo de respuesta y la precisión, manteniéndose a la vez estables ante suplantaciones y optimizando el uso de recursos en la medida de lo posible. En la última década han proliferado las publicaciones sobre este modelo, investigando la aplicación de varios clasificadores en serie a diversos escenarios, como la identificación mediante patrones biométricos, diagnóstico del nivel de stress en sujetos individuales o reconocimiento de vehículos a motor mediante imágenes, entre otros, ya sea utilizando el resultado de un solo clasificador para decidir en cualquier estado o tomando la fusión de los clasificadores en serie utilizados para calcular la puntuación definitiva.Along the continuous technological advance of the last decades, the amount of data to be handled has been incremented in a proportional way. Therefore, a new way to analyze and classify huge amounts of data automatically is needed to be studied and designed. Following this goal, a lot of classifiers have been investigated using mathematical models or even bio-inspired models, performing better than expected in specific scenarios. Aiming to improve those classifiers performance, some researches made in the last twenty years pointed to more complex structures that solve the problem by connecting individual classifiers and fusing each individual score into a global one. The ensembles most commonly studied connect each single classifier in parallel, evaluation system’s output once each individual classifier has given its own score. This kind of classifier ensembles has been tested as an alternative to individual classifiers with better precision, more reliable to certain errors and spoofing attacks, most common threat of identification systems based on biometric pattern recognition. Nevertheless, some problems are shown on combining different classifiers in parallel. Each classifier needs to be designed and trained individually, the amount of resources needed for each module execution at the same time or the increment of the time needed to calculate the final score, limited in this case by the slowest classifier of the ensemble. On the other hand, serial and cascade ensembles are designed to find a tradeoff between latency and precision, optimize resource usage while being at least as reliable against spoofing attacks as parallel system are. In the last decade, a lot of work related to this model has been published, doing some research on the implementation of serial fusion ensembles in real-life scenarios, like biometric pattern recognition, stress level diagnosis of individual subjects or vehicle recognition based on images. These examples were designed using different approaches, like accepting the score of a single classifier of the ensemble on a certain stage as the final score or performing a fusion of the scores calculated at a certain stage

Car make and model recognition under limited lighting conditions at night

A thesis submitted to the University of Bedfordshire in partial fulfilment of the requirements for the degree of Doctor of PhilosophyCar make and model recognition (CMMR) has become an important part of intelligent transport systems. Information provided by CMMR can be utilized when licence plate numbers cannot be identified or fake number plates are used. CMMR can also be used when automatic identification of a certain model of a vehicle by camera is required. The majority of existing CMMR methods are designed to be used only in daytime when most car features can be easily seen. Few methods have been developed to cope with limited lighting conditions at night where many vehicle features cannot be detected. This work identifies car make and model at night by using available rear view features. A binary classifier ensemble is presented, designed to identify a particular car model of interest from other models. The combination of salient geographical and shape features of taillights and licence plates from the rear view are extracted and used in the recognition process. The majority vote of individual classifiers, support vector machine, decision tree, and k-nearest neighbours is applied to verify a target model in the classification process. The experiments on 100 car makes and models captured under limited lighting conditions at night against about 400 other car models show average high classification accuracy about 93%. The classification accuracy of the presented technique, 93%, is a bit lower than the daytime technique, as reported at 98 % tested on 21 CMMs (Zhang, 2013). However, with the limitation of car appearances at night, the classification accuracy of the car appearances gained from the technique used in this study is satisfied

Vehicle make and model recognition for intelligent transportation monitoring and surveillance.

Vehicle Make and Model Recognition (VMMR) has evolved into a significant subject of study due to its importance in numerous Intelligent Transportation Systems (ITS), such as autonomous navigation, traffic analysis, traffic surveillance and security systems. A highly accurate and real-time VMMR system significantly reduces the overhead cost of resources otherwise required. The VMMR problem is a multi-class classification task with a peculiar set of issues and challenges like multiplicity, inter- and intra-make ambiguity among various vehicles makes and models, which need to be solved in an efficient and reliable manner to achieve a highly robust VMMR system. In this dissertation, facing the growing importance of make and model recognition of vehicles, we present a VMMR system that provides very high accuracy rates and is robust to several challenges. We demonstrate that the VMMR problem can be addressed by locating discriminative parts where the most significant appearance variations occur in each category, and learning expressive appearance descriptors. Given these insights, we consider two data driven frameworks: a Multiple-Instance Learning-based (MIL) system using hand-crafted features and an extended application of deep neural networks using MIL. Our approach requires only image level class labels, and the discriminative parts of each target class are selected in a fully unsupervised manner without any use of part annotations or segmentation masks, which may be costly to obtain. This advantage makes our system more intelligent, scalable, and applicable to other fine-grained recognition tasks. We constructed a dataset with 291,752 images representing 9,170 different vehicles to validate and evaluate our approach. Experimental results demonstrate that the localization of parts and distinguishing their discriminative powers for categorization improve the performance of fine-grained categorization. Extensive experiments conducted using our approaches yield superior results for images that were occluded, under low illumination, partial camera views, or even non-frontal views, available in our real-world VMMR dataset. The approaches presented herewith provide a highly accurate VMMR system for rea-ltime applications in realistic environments.\\ We also validate our system with a significant application of VMMR to ITS that involves automated vehicular surveillance. We show that our application can provide law inforcement agencies with efficient tools to search for a specific vehicle type, make, or model, and to track the path of a given vehicle using the position of multiple cameras

Siamese networks for surveillance and security

This thesis investigates the usage of Siamese networks across three surveillance and security tasks for land border security. Siamese networks (also known as a twin-pair network) are a layout of neural networks that contain a segment that contains duplicated architecture and configuration parameters for feature extraction of two inputs, combining the outputs into one vector for comparison in a final set of layers to produce a similarity score. The effectiveness of multiple architectures of Siamese networks crafted from multiple generations of Convolutional Neural Networks and Residual Neural Networks are examined for side-profile vehicle classification and Differential Morphing Attack Detection (D-MAD), and with a novel architecture for trajectory similarity analysis. The challenging domain of automated vehicle classification from pole-mounted roadway cameras from side-profile views is evaluated. Three Siamese networks based on existing non-Siamese architectures are proposed and compared against five existing methods on a novel and published dataset. The evaluation undertaken shows that the residual based Siamese network is able to outperform other state of the art methods on datasets with a small number of classes. An end-to-end Siamese trajectory network framework is proposed for the purpose of trajectory similarity analysis in surveillance tasks. A deep feature auto-encoding network is used as part of a discriminative Siamese architecture to perform trajectory similarity analysis. The effectiveness of this method is evaluated on four challenging public real-world datasets containing both vehicle and pedestrian targets, and compared with five existing methods. The proposed method outperforms the existing methods on three of the four datasets. Face morphing attacks pose an increasingly severe threat to automatic face recognition systems in border control environments. Three Siamese architectures built up from multiple generations of non-Siamese Convolutional and Residual Neural Networks for D-MAD are proposed, showing the effectiveness of these networks against a pre-established Convolutional architecture for Single-image Morphing Attack Detection (S-MAD). The residual network based architecture outperforms representative convolutional architectures from the literature, with the Siamese D-MAD architecture able to outperform its S-MAD variant

Computer Vision Based Structural Identification Framework for Bridge Health Mornitoring

The objective of this dissertation is to develop a comprehensive Structural Identification (St-Id) framework with damage for bridge type structures by using cameras and computer vision technologies. The traditional St-Id frameworks rely on using conventional sensors. In this study, the collected input and output data employed in the St-Id system are acquired by series of vision-based measurements. The following novelties are proposed, developed and demonstrated in this project: a) vehicle load (input) modeling using computer vision, b) bridge response (output) using full non-contact approach using video/image processing, c) image-based structural identification using input-output measurements and new damage indicators. The input (loading) data due vehicles such as vehicle weights and vehicle locations on the bridges, are estimated by employing computer vision algorithms (detection, classification, and localization of objects) based on the video images of vehicles. Meanwhile, the output data as structural displacements are also obtained by defining and tracking image key-points of measurement locations. Subsequently, the input and output data sets are analyzed to construct novel types of damage indicators, named Unit Influence Surface (UIS). Finally, the new damage detection and localization framework is introduced that does not require a network of sensors, but much less number of sensors. The main research significance is the first time development of algorithms that transform the measured video images into a form that is highly damage-sensitive/change-sensitive for bridge assessment within the context of Structural Identification with input and output characterization. The study exploits the unique attributes of computer vision systems, where the signal is continuous in space. This requires new adaptations and transformations that can handle computer vision data/signals for structural engineering applications. This research will significantly advance current sensor-based structural health monitoring with computer-vision techniques, leading to practical applications for damage detection of complex structures with a novel approach. By using computer vision algorithms and cameras as special sensors for structural health monitoring, this study proposes an advance approach in bridge monitoring through which certain type of data that could not be collected by conventional sensors such as vehicle loads and location, can be obtained practically and accurately