A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector

Automatic License Plate Recognition (ALPR) has been a frequent topic of research due to many practical applications. However, many of the current solutions are still not robust in real-world situations, commonly depending on many constraints. This paper presents a robust and efficient ALPR system based on the state-of-the-art YOLO object detector. The Convolutional Neural Networks (CNNs) are trained and fine-tuned for each ALPR stage so that they are robust under different conditions (e.g., variations in camera, lighting, and background). Specially for character segmentation and recognition, we design a two-stage approach employing simple data augmentation tricks such as inverted License Plates (LPs) and flipped characters. The resulting ALPR approach achieved impressive results in two datasets. First, in the SSIG dataset, composed of 2,000 frames from 101 vehicle videos, our system achieved a recognition rate of 93.53% and 47 Frames Per Second (FPS), performing better than both Sighthound and OpenALPR commercial systems (89.80% and 93.03%, respectively) and considerably outperforming previous results (81.80%). Second, targeting a more realistic scenario, we introduce a larger public dataset, called UFPR-ALPR dataset, designed to ALPR. This dataset contains 150 videos and 4,500 frames captured when both camera and vehicles are moving and also contains different types of vehicles (cars, motorcycles, buses and trucks). In our proposed dataset, the trial versions of commercial systems achieved recognition rates below 70%. On the other hand, our system performed better, with recognition rate of 78.33% and 35 FPS.Comment: Accepted for presentation at the International Joint Conference on Neural Networks (IJCNN) 201

An End-to-End Automated License Plate Recognition System Using YOLO Based Vehicle and License Plate Detection with Vehicle Classification

An accurate and robust Automatic License Plate Recognition (ALPR) method proves surprising versatility in an Intelligent Transportation and Surveillance (ITS) system. However, most of the existing approaches often use prior knowledge or fixed pre-and-post processing rules and are thus limited by poor generalization in complex real-life conditions. In this paper, we leverage a YOLO-based end-to-end generic ALPR pipeline for vehicle detection (VD), license plate (LP) detection and recognition without exploiting prior knowledge or additional steps in inference. We assess the whole ALPR pipeline, starting from vehicle detection to the LP recognition stage, including a vehicle classifier for emergency vehicles and heavy trucks. We used YOLO v2 in the initial stage of the pipeline and remaining stages are based on the state-of-the-art YOLO v4 detector with various data augmentation and generation techniques to obtain LP recognition accuracy on par with current proposed methods. To evaluate our approach, we used five public datasets from different regions, and we achieved an average recognition accuracy of 90.3% while maintaining an acceptable frames per second (FPS) on a low-end GPU

Detection and Recognition of License Plates by Convolutional Neural Networks

The current advancements in machine intelligence have expedited the process of recognizing vehicles and other objects on the roads. The License Plate Recognition system (LPR) is an open challenge for many researchers to develop a reliable and accurate system for automatic license plate recognition. Several methods including Deep Learning techniques have been proposed recently for LPR, yet those methods are limited to specific regions or privately collected datasets. In this thesis, we propose an end-to-end Deep Convolutional Neural Network system for license plate recognition that is not limited to a specific region or country. We apply a modified version of YOLO v2 to first recognize the vehicle and then localize the license plate. Moreover, through the convolutional procedures, we improve an Optical Character Recognition network (OCR-Net) to recognize the license plate numbers and letters. Our method performs well for different vehicle types such as sedans, SUVs, buses, motorbikes, and trucks. The system works reliably on images of the front and rear views of the vehicle, and it also overcomes tilted or distorted license plate images and performs adequately under various illumination conditions, and noisy backgrounds. Several experiments have been carried out on various types of images from privately collected and publicly available datasets including OPEN-ALPR (BR, EU, US) which consists of 115 Brazilian, 108 European, and 222 North American images, CENPARMI includes 440 from Chinese, US, and different provinces of Canada and UFPR-ALPR includes 4500 Brazilian license plate images; images of those datasets have several challenges: i.e. single to multiple vehicles in an image, license plates of different countries, vehicles at different distances, and images taken by several types of cameras including cellphone cameras. Our experimental results show that the proposed system achieves 98.04% accuracy on average for OPEN-ALPR dataset, 88.5% for the more challenging CENPARMI dataset and 97.42% for UFPR-ALPR dataset respectively, outperforming the state-of-the-art commercial and academics

An Efficient and Layout-Independent Automatic License Plate Recognition System Based on the YOLO detector

This paper presents an efficient and layout-independent Automatic License Plate Recognition (ALPR) system based on the state-of-the-art YOLO object detector that contains a unified approach for license plate (LP) detection and layout classification to improve the recognition results using post-processing rules. The system is conceived by evaluating and optimizing different models, aiming at achieving the best speed/accuracy trade-off at each stage. The networks are trained using images from several datasets, with the addition of various data augmentation techniques, so that they are robust under different conditions. The proposed system achieved an average end-to-end recognition rate of 96.9% across eight public datasets (from five different regions) used in the experiments, outperforming both previous works and commercial systems in the ChineseLP, OpenALPR-EU, SSIG-SegPlate and UFPR-ALPR datasets. In the other datasets, the proposed approach achieved competitive results to those attained by the baselines. Our system also achieved impressive frames per second (FPS) rates on a high-end GPU, being able to perform in real time even when there are four vehicles in the scene. An additional contribution is that we manually labeled 38,351 bounding boxes on 6,239 images from public datasets and made the annotations publicly available to the research community

Automated license plate recognition for resource-constrained environments

The incorporation of deep-learning techniques in embedded systems has enhanced the capabilities of edge computing to a great extent. However, most of these solutions rely on high-end hardware and often require a high processing capacity, which cannot be achieved with resource-constrained edge computing. This study presents a novel approach and a proof of concept for a hardware-efficient automated license plate recognition system for a constrained environment with limited resources. The proposed solution is purely implemented for low-resource edge devices and performed well for extreme illumination changes such as day and nighttime. The generalisability of the proposed models has been achieved using a novel set of neural networks for different hardware configurations based on the computational capabilities and low cost. The accuracy, energy efficiency, communication, and computational latency of the proposed models are validated using different license plate datasets in the daytime and nighttime and in real time. Meanwhile, the results obtained from the proposed study have shown competitive performance to the state-of-the-art server-grade hardware solutions as well

License Plate Detection using Deep Learning and Font Evaluation

License plate detection (LPD) in context is a challenging problem due to its sensitivity to environmental factors. Moreover, the chosen font type in the license plate (LP) plays a vital role in the recognition phase in computer-based studies. This work is two folded. On one hand, we propose to employ Deep Learning technique (namely, You Only Look Once (YOLO)) in the LPD. On the other hand, we propose to evaluate font characteristics in the LP context. This work uses 2 different datasets: UFPR-ALPR, and the newly created CENPARMI datasets. We propose a YOLO-based adaptive algorithm with tuned parameters to enhance its performance. In addition to report the recall ratio results, this work will conduct a detailed error analysis to provide some insights into the type of false positives. The proposed model achieved competitive recall ratio of 98.38% with a single YOLO network. Some fonts are challenging for humans to read; however, other fonts are challenging for computer systems to recognize. Here, we present 2 sets of results for font evaluation: font anatomy results, and commercial products recognition results. For anatomy results, 2 fonts are considered: Mandatory, and Driver Gothic. Moreover, we evaluate the effect of the used fonts in context for the two datasets using 2 commercial products: OpenALPR and Plate Recognizer. The font anatomy results revealed some important confusion cases and some quality features of both fonts. The obtained results show that the Driver font has no severe confusion cases in contrast to the Mandatory font