A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector

Automatic License Plate Recognition (ALPR) has been a frequent topic of research due to many practical applications. However, many of the current solutions are still not robust in real-world situations, commonly depending on many constraints. This paper presents a robust and efficient ALPR system based on the state-of-the-art YOLO object detector. The Convolutional Neural Networks (CNNs) are trained and fine-tuned for each ALPR stage so that they are robust under different conditions (e.g., variations in camera, lighting, and background). Specially for character segmentation and recognition, we design a two-stage approach employing simple data augmentation tricks such as inverted License Plates (LPs) and flipped characters. The resulting ALPR approach achieved impressive results in two datasets. First, in the SSIG dataset, composed of 2,000 frames from 101 vehicle videos, our system achieved a recognition rate of 93.53% and 47 Frames Per Second (FPS), performing better than both Sighthound and OpenALPR commercial systems (89.80% and 93.03%, respectively) and considerably outperforming previous results (81.80%). Second, targeting a more realistic scenario, we introduce a larger public dataset, called UFPR-ALPR dataset, designed to ALPR. This dataset contains 150 videos and 4,500 frames captured when both camera and vehicles are moving and also contains different types of vehicles (cars, motorcycles, buses and trucks). In our proposed dataset, the trial versions of commercial systems achieved recognition rates below 70%. On the other hand, our system performed better, with recognition rate of 78.33% and 35 FPS.Comment: Accepted for presentation at the International Joint Conference on Neural Networks (IJCNN) 201

Vehicle-Rear: A New Dataset to Explore Feature Fusion for Vehicle Identification Using Convolutional Neural Networks

This work addresses the problem of vehicle identification through non-overlapping cameras. As our main contribution, we introduce a novel dataset for vehicle identification, called Vehicle-Rear, that contains more than three hours of high-resolution videos, with accurate information about the make, model, color and year of nearly 3,000 vehicles, in addition to the position and identification of their license plates. To explore our dataset we design a two-stream CNN that simultaneously uses two of the most distinctive and persistent features available: the vehicle's appearance and its license plate. This is an attempt to tackle a major problem: false alarms caused by vehicles with similar designs or by very close license plate identifiers. In the first network stream, shape similarities are identified by a Siamese CNN that uses a pair of low-resolution vehicle patches recorded by two different cameras. In the second stream, we use a CNN for OCR to extract textual information, confidence scores, and string similarities from a pair of high-resolution license plate patches. Then, features from both streams are merged by a sequence of fully connected layers for decision. In our experiments, we compared the two-stream network against several well-known CNN architectures using single or multiple vehicle features. The architectures, trained models, and dataset are publicly available at https://github.com/icarofua/vehicle-rear

Detection and Recognition of License Plates by Convolutional Neural Networks

The current advancements in machine intelligence have expedited the process of recognizing vehicles and other objects on the roads. The License Plate Recognition system (LPR) is an open challenge for many researchers to develop a reliable and accurate system for automatic license plate recognition. Several methods including Deep Learning techniques have been proposed recently for LPR, yet those methods are limited to specific regions or privately collected datasets. In this thesis, we propose an end-to-end Deep Convolutional Neural Network system for license plate recognition that is not limited to a specific region or country. We apply a modified version of YOLO v2 to first recognize the vehicle and then localize the license plate. Moreover, through the convolutional procedures, we improve an Optical Character Recognition network (OCR-Net) to recognize the license plate numbers and letters. Our method performs well for different vehicle types such as sedans, SUVs, buses, motorbikes, and trucks. The system works reliably on images of the front and rear views of the vehicle, and it also overcomes tilted or distorted license plate images and performs adequately under various illumination conditions, and noisy backgrounds. Several experiments have been carried out on various types of images from privately collected and publicly available datasets including OPEN-ALPR (BR, EU, US) which consists of 115 Brazilian, 108 European, and 222 North American images, CENPARMI includes 440 from Chinese, US, and different provinces of Canada and UFPR-ALPR includes 4500 Brazilian license plate images; images of those datasets have several challenges: i.e. single to multiple vehicles in an image, license plates of different countries, vehicles at different distances, and images taken by several types of cameras including cellphone cameras. Our experimental results show that the proposed system achieves 98.04% accuracy on average for OPEN-ALPR dataset, 88.5% for the more challenging CENPARMI dataset and 97.42% for UFPR-ALPR dataset respectively, outperforming the state-of-the-art commercial and academics

CNN-RNN based method for license plate recognition

Achieving good recognition results for License plates is challenging due to multiple adverse factors. For instance, in Malaysia, where private vehicle (e.g., cars) have numbers with dark background, while public vehicle (taxis/cabs) have numbers with white background. To reduce the complexity of the problem, we propose to classify the above two types of images such that one can choose an appropriate method to achieve better results. Therefore, in this work, we explore the combination of Convolutional Neural Networks (CNN) and Recurrent Neural Networks namely, BLSTM (Bi-Directional Long Short Term Memory), for recognition. The CNN has been used for feature extraction as it has high discriminative ability, at the same time, BLSTM has the ability to extract context information based on the past information. For classification, we propose Dense Cluster based Voting (DCV), which separates foreground and background for successful classification of private and public. Experimental results on live data given by MIMOS, which is funded by Malaysian Government and the standard dataset UCSD show that the proposed classification outperforms the existing methods. In addition, the recognition results show that the recognition performance improves significantly after classification compared to before classification

An Efficient and Layout-Independent Automatic License Plate Recognition System Based on the YOLO detector

This paper presents an efficient and layout-independent Automatic License Plate Recognition (ALPR) system based on the state-of-the-art YOLO object detector that contains a unified approach for license plate (LP) detection and layout classification to improve the recognition results using post-processing rules. The system is conceived by evaluating and optimizing different models, aiming at achieving the best speed/accuracy trade-off at each stage. The networks are trained using images from several datasets, with the addition of various data augmentation techniques, so that they are robust under different conditions. The proposed system achieved an average end-to-end recognition rate of 96.9% across eight public datasets (from five different regions) used in the experiments, outperforming both previous works and commercial systems in the ChineseLP, OpenALPR-EU, SSIG-SegPlate and UFPR-ALPR datasets. In the other datasets, the proposed approach achieved competitive results to those attained by the baselines. Our system also achieved impressive frames per second (FPS) rates on a high-end GPU, being able to perform in real time even when there are four vehicles in the scene. An additional contribution is that we manually labeled 38,351 bounding boxes on 6,239 images from public datasets and made the annotations publicly available to the research community

License Plate Detection using Deep Learning and Font Evaluation

License plate detection (LPD) in context is a challenging problem due to its sensitivity to environmental factors. Moreover, the chosen font type in the license plate (LP) plays a vital role in the recognition phase in computer-based studies. This work is two folded. On one hand, we propose to employ Deep Learning technique (namely, You Only Look Once (YOLO)) in the LPD. On the other hand, we propose to evaluate font characteristics in the LP context. This work uses 2 different datasets: UFPR-ALPR, and the newly created CENPARMI datasets. We propose a YOLO-based adaptive algorithm with tuned parameters to enhance its performance. In addition to report the recall ratio results, this work will conduct a detailed error analysis to provide some insights into the type of false positives. The proposed model achieved competitive recall ratio of 98.38% with a single YOLO network. Some fonts are challenging for humans to read; however, other fonts are challenging for computer systems to recognize. Here, we present 2 sets of results for font evaluation: font anatomy results, and commercial products recognition results. For anatomy results, 2 fonts are considered: Mandatory, and Driver Gothic. Moreover, we evaluate the effect of the used fonts in context for the two datasets using 2 commercial products: OpenALPR and Plate Recognizer. The font anatomy results revealed some important confusion cases and some quality features of both fonts. The obtained results show that the Driver font has no severe confusion cases in contrast to the Mandatory font

Leveraging Model Fusion for Improved License Plate Recognition

License Plate Recognition (LPR) plays a critical role in various applications, such as toll collection, parking management, and traffic law enforcement. Although LPR has witnessed significant advancements through the development of deep learning, there has been a noticeable lack of studies exploring the potential improvements in results by fusing the outputs from multiple recognition models. This research aims to fill this gap by investigating the combination of up to 12 different models using straightforward approaches, such as selecting the most confident prediction or employing majority vote-based strategies. Our experiments encompass a wide range of datasets, revealing substantial benefits of fusion approaches in both intra- and cross-dataset setups. Essentially, fusing multiple models reduces considerably the likelihood of obtaining subpar performance on a particular dataset/scenario. We also found that combining models based on their speed is an appealing approach. Specifically, for applications where the recognition task can tolerate some additional time, though not excessively, an effective strategy is to combine 4-6 models. These models may not be the most accurate individually, but their fusion strikes an optimal balance between accuracy and speed.Comment: Accepted for presentation at the Iberoamerican Congress on Pattern Recognition (CIARP) 202