Recognizing License Plates in Real-Time

License plate detection and recognition (LPDR) is of growing importance for enabling intelligent transportation and ensuring the security and safety of the cities. However, LPDR faces a big challenge in a practical environment. The license plates can have extremely diverse sizes, fonts and colors, and the plate images are usually of poor quality caused by skewed capturing angles, uneven lighting, occlusion, and blurring. In applications such as surveillance, it often requires fast processing. To enable real-time and accurate license plate recognition, in this work, we propose a set of techniques: 1) a contour reconstruction method along with edge-detection to quickly detect the candidate plates; 2) a simple zero-one-alternation scheme to effectively remove the fake top and bottom borders around plates to facilitate more accurate segmentation of characters on plates; 3) a set of techniques to augment the training data, incorporate SIFT features into the CNN network, and exploit transfer learning to obtain the initial parameters for more effective training; and 4) a two-phase verification procedure to determine the correct plate at low cost, a statistical filtering in the plate detection stage to quickly remove unwanted candidates, and the accurate CR results after the CR process to perform further plate verification without additional processing. We implement a complete LPDR system based on our algorithms. The experimental results demonstrate that our system can accurately recognize license plate in real-time. Additionally, it works robustly under various levels of illumination and noise, and in the presence of car movement. Compared to peer schemes, our system is not only among the most accurate ones but is also the fastest, and can be easily applied to other scenarios.Comment: License Plate Detection and Recognition, Computer Vision, Supervised Learnin

ЗАСТОСУВАННЯ ЗГОРТКОВИХ НЕЙРОННИХ МЕРЕЖ ДЛЯ БЕЗПЕКИ РОЗПІЗНАВАННЯ ОБ'ЄКТІВ У ВІДЕОПОТОЦІ

The article is devoted to analyzing methods for recognizing images and finding them in the video stream. The evolution of the structure of convolutional neural networks used in the field of computer video flow diagnostics is analyzed. The performance of video flow diagnostics algorithms and car license plate recognition has been evaluated. The technique of recognizing the license plates of cars in the video stream of transport neural networks is described. The study focuses on the creation of a combined system that combines artificial intelligence and computer vision based on fuzzy logic. To solve the problem of license plate image recognition in the video stream of the transport system, a method of image recognition in a continuous video stream with its implementation based on the composition of traditional image processing methods and neural networks with convolutional and periodic layers is proposed. The structure and peculiarities of functioning of the intelligent distributed system of urban transport safety, which feature is the use of mobile devices connected to a single network, are described. A practical implementation of a software application for recognizing car license plates by mobile devices on the Android operating system platform has been proposed and implemented. Various real-time vehicle license plate recognition scenarios have been developed and stored in a database for further analysis and use. The proposed application uses two different specialized neural networks: one for detecting objects in the video stream, the other for recognizing text from the selected image. Testing and analysis of software applications on the Android operating system platform for license plate recognition in real time confirmed the functionality of the proposed mathematical software and can be used to securely analyze the license plates of cars in the scanned video stream by comparing with license plates in the existing database. The authors have implemented the operation of the method of convolutional neural networks detection and recognition of license plates, personnel and critical situations in the video stream from cameras of mobile devices in real time. The possibility of its application in the field of safe identification of car license plates has been demonstrated.Стаття присвячена аналізу методів розпізнавання зображень та пошуку їх у відеопотоці. Проаналізовано еволюцію структури згорткових нейронних мереж, що використовуються в області діагностики комп'ютерних відеопотоків. Здійснено оцінку ефективності алгоритмів діагностики відеопотоків та розпізнавання номерних знаків автомобілів. Описана методика розпізнавання номерних знаків автомобілів, що знаходяться у відеопотоці транспортних нейронних мереж. В дослідженні приділено увагу створенню комбінованої системи, яка поєднує в собі технологію штучного інтелекту та комп'ютерного зору на основі нечіткої логіки. Для вирішення проблеми розпізнавання зображень номерних знаків у відеопотоці транспортної системи запропоновано метод розпізнавання зображень у безперервному відеопотоці з його реалізацією на основі складу традиційних методів обробки зображень та нейронних мереж із згортковими та періодичними шарами. Описано структуру та особливості функціонування інтелектуальної розподіленої системи безпеки міського транспорту, особливістю якої є використання мобільних пристроїв, підключених до єдиної мережі. Запропоновано та здійснено практичну реалізацію програмного застосування для розпізнавання автомобільних номерних знаків мобільними пристроями на платформі операційної системи Android. Розроблено різні сценарії розпізнавання номерних знаків автомобілів у реальному часі та збереження їх у базі даних для подальшого аналізу та використання. В запропонованому застосуванні використовуються дві різні спеціалізовані нейромережі: одна - для детектування об’єктів у відеопотоці, інша – для розпізнавання тексту з виділеного зображення. Проведене випробовування та аналіз програмного застосування на платформі операційної системи Android для розпізнавання автомобільних номерних знаків у реальному часі підтвердив працездатність запропонованого математичного забезпечення і може використовуватися для безпечного аналізу номерних знаків автомобілів у сканованому відео потоці шляхом порівняння з номерними знаками в існуючій базі даних. Авторами реалізовано функціонування метод згорткових нейронних мереж виявлення та розпізнавання номерних знаків, персоналу та критичних ситуацій у відеопотоці з камер мобільних пристроїв у режимі реального часу. Продемонстрована можливість його застосування у сфері безпечної ідентифікації номерних знаків автомобілів

An improved Malaysian automatic license plate recognition (M-ALPR) system using hybrid fuzzy in C++ environment

In this paper, an improved hybrid fuzzy technique (Fuzzy Logic and Template matching) for Malaysian Automatic License Plate Recognition (M-ALPR) system is proposed. The system is proposed to reduce the program complexity of the existing M-ALPR system and to decrease the processing time of recognizing Malaysian license plates. First, the algorithm to recognize the license plates is presented, by taking advantage of Matlab and C++ programming language benefits in order to increase system efficiency. Feature extraction using vertical line counter is introduced in this system. Later, with the help of OpenCV, the hybrid fuzzy technique is developed using the C++ language. Then, the comparison between these two implementations on M-ALPR system was reported. The improved system was tested on 740 samples images from real scene and the results show that the proposed improvement supports the accurateness and high speed processing of M-ALPR system

Analysis of the vehicle country of origin distribution on the Zagreb bypass using video detection methods

Radi povećanja iskoristivosti cestovne infrastrukture sve se više koriste napredne metode upravljanja iz domene inteligentnih transportnih sustava (ITS). Takve metode podrazumijevaju korištenje povijesnih i stvarno-vremenskih prometnih parametara. Jedan od senzora, koji se danas sve više koristi u ITSu, je video kamera koja omogućuje mjerenje većeg broja prometnih parametara kao i praćenje vozila prepoznavanjem registarskih oznaka. Na osnovu prepoznatih registarskih oznaka moguće je odrediti zemlju porijekla cestovnog vozila te njihovu distribuciju. U radu je opisana kamera kao senzor za mjerenje prometnih parametara, prikazana arhitektura sustava za prepoznavanje registarskih oznaka cestovnih vozila, napravljen pregled najčešće korištenih metoda za prepoznavanje registarskih oznaka cestovnih vozila, prilagođena i testirana postojeća aplikacija za prepoznavanje registarskih oznaka zasnovana na biblioteci CARMEN za prepoznavanje registarskih oznaka vozila korištenjem stvarnih video snimki prometa zagrebačke obilaznice.Advanced management methods from the domain of intelligent transport systems (ITS) are being used today to increase the efficiency of road infrastructure. Those methods involve usage of historical and real-time traffic parameters. Video camera is one of the sensors that are used in ITS which allows measurement of several traffic parameters simultaneously, as well as vehicle tracking by recognizing the license plate. On the basis of the identified license plates it is possible to determine the country of origin of the vehicles and their distribution. In this thesis video camera is described as a sensor to measure traffic parameters, system architecture for license plate recognition is shown and described, overview of the most commonly used methods for recognizing license plates is made, the existing applications for license plate recognition based on the CARMEN license plate recognition library using actual video footage of traffic Zagreb bypass is adapted and tested

Analysis of the vehicle country of origin distribution on the Zagreb bypass using video detection methods

Radi povećanja iskoristivosti cestovne infrastrukture sve se više koriste napredne metode upravljanja iz domene inteligentnih transportnih sustava (ITS). Takve metode podrazumijevaju korištenje povijesnih i stvarno-vremenskih prometnih parametara. Jedan od senzora, koji se danas sve više koristi u ITSu, je video kamera koja omogućuje mjerenje većeg broja prometnih parametara kao i praćenje vozila prepoznavanjem registarskih oznaka. Na osnovu prepoznatih registarskih oznaka moguće je odrediti zemlju porijekla cestovnog vozila te njihovu distribuciju. U radu je opisana kamera kao senzor za mjerenje prometnih parametara, prikazana arhitektura sustava za prepoznavanje registarskih oznaka cestovnih vozila, napravljen pregled najčešće korištenih metoda za prepoznavanje registarskih oznaka cestovnih vozila, prilagođena i testirana postojeća aplikacija za prepoznavanje registarskih oznaka zasnovana na biblioteci CARMEN za prepoznavanje registarskih oznaka vozila korištenjem stvarnih video snimki prometa zagrebačke obilaznice.Advanced management methods from the domain of intelligent transport systems (ITS) are being used today to increase the efficiency of road infrastructure. Those methods involve usage of historical and real-time traffic parameters. Video camera is one of the sensors that are used in ITS which allows measurement of several traffic parameters simultaneously, as well as vehicle tracking by recognizing the license plate. On the basis of the identified license plates it is possible to determine the country of origin of the vehicles and their distribution. In this thesis video camera is described as a sensor to measure traffic parameters, system architecture for license plate recognition is shown and described, overview of the most commonly used methods for recognizing license plates is made, the existing applications for license plate recognition based on the CARMEN license plate recognition library using actual video footage of traffic Zagreb bypass is adapted and tested

Multi-Object Tracking based Roadside Parking Behavior Recognition

Roadside parking spaces can alleviate the shortage of parking spaces, but there are some shortcomings to the charges for roadside parking. The popular charging methods at present mainly include manual charging, geomagnetic detection charging, meter charging, etc. These methods have certain limitations, such as high cost, difficult deployment, and low acceptance of people. To solve the shortcomings of roadside parking charges, this thesis proposes a scheme based on deep learning and image recognition. More specifically, the thesis proposes a scheme for detecting and tracking vehicles, recognizing license plates, recognizing vehicle parking behavior, and recording vehicle parking periods through the monocular camera to solve the problem of roadside parking charges. The scheme has the advantages of convenient deployment, low labor cost, high efficiency, and high accuracy. The main work of this thesis is as follows: 1. Based on the You Only Look Once (YOLO) algorithm, this thesis proposes a trapezoidal convolution algorithm to detect objects and improve the detection efficiency for the problem that the vehicle is far and small in the image. 2. Proposes a one-stage license plate recognition scheme based on YOLO, aiming to simplify the license plate recognition process. 3. Depending on the characteristics of the vehicle, this thesis proposes a feature extraction model of the vehicle, called the horizontal and vertical separation model, which use to combine with the deep Simple Online and Real-time Tracking (SORT) object tracking framework to track the vehicle and improve the tracking efficiency. 4. Uses a Long Short-Term Memory (LSTM) model to classify the behavior of the vehicle into three types: Park, leave, and no behavior. 5. Groups these modules together, and the engineering code is debugged a lot to realize a complete Roadside Parking Behavior Recognition (RPBR) system

Template Neural Particle Optimization For Vehicle License Plate Recognition

The need for vehicle recognition has emerged from cases such as security, smart toll collections and traffic monitoring systems. This type of applications produces high demands especially on the accuracy of license plate recognition (LPR). The challenge of LPR is to select the best method for recognizing characters. Since the importance of LPR arises over times, there is a need to find the best alternative to overcome the problem. The detection and extraction of license plate is conventionally based on image processing methods. The image processing method in license plate recognition generally comprises of five stages including pre-processing, morphological operation, feature extraction, segmentation and character recognition. Pre-processing is an initial step in image processing to improve image quality for more suitability in visualizing perception or computational processing while filtering is required to solve contrast enhancement, noise suppression, blurry issue and data reduction. Feature extraction is applied to locate accurately the license plate position and segmentation is used to find and segment the isolated characters on the plates, without losing features of the characters. Finally, character recognition determines each character, identity and displays it into machine readable form. This study introduces five methods of character recognition namely template matching (TM), back-propagation neural network (BPNN), Particle Swarm Optimization neural network (PSONN), hybrid of TM with BPNN (TM-BPNN) and hybrid of TM with PSONN (TM-PSONN). PSONN is proposed as an alternative to train feed-forward neural network, while TM-BPNN and TM-PSONN are proposed to produce a better recognition result. The performance evaluation is carried out based on mean squared error, processing time, number of training iteration, correlation value and percentage of accuracy. The performance of the selected methods was analyzed by making use real images of 300 vehicles. The hybrid of TM-BPNN gives the highest recognition result with 94% accuracy, followed by the hybrid of TM-PSONN with 91.3%, TM with 77.3%, BPNN with 61.7% and lastly PSONN with 37.7%

A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector

Automatic License Plate Recognition (ALPR) has been a frequent topic of research due to many practical applications. However, many of the current solutions are still not robust in real-world situations, commonly depending on many constraints. This paper presents a robust and efficient ALPR system based on the state-of-the-art YOLO object detector. The Convolutional Neural Networks (CNNs) are trained and fine-tuned for each ALPR stage so that they are robust under different conditions (e.g., variations in camera, lighting, and background). Specially for character segmentation and recognition, we design a two-stage approach employing simple data augmentation tricks such as inverted License Plates (LPs) and flipped characters. The resulting ALPR approach achieved impressive results in two datasets. First, in the SSIG dataset, composed of 2,000 frames from 101 vehicle videos, our system achieved a recognition rate of 93.53% and 47 Frames Per Second (FPS), performing better than both Sighthound and OpenALPR commercial systems (89.80% and 93.03%, respectively) and considerably outperforming previous results (81.80%). Second, targeting a more realistic scenario, we introduce a larger public dataset, called UFPR-ALPR dataset, designed to ALPR. This dataset contains 150 videos and 4,500 frames captured when both camera and vehicles are moving and also contains different types of vehicles (cars, motorcycles, buses and trucks). In our proposed dataset, the trial versions of commercial systems achieved recognition rates below 70%. On the other hand, our system performed better, with recognition rate of 78.33% and 35 FPS.Comment: Accepted for presentation at the International Joint Conference on Neural Networks (IJCNN) 201

Empirical Study of Car License Plates Recognition

The number of vehicles on the road has increased drastically in recent years. The license plate is an identity card for a vehicle. It can map to the owner and further information about vehicle. License plate information is useful to help traffic management systems. For example, traffic management systems can check for vehicles moving at speeds not permitted by law and can also be installed in parking areas to se-cure the entrance or exit way for vehicles. License plate recognition algorithms have been proposed by many researchers. License plate recognition requires license plate detection, segmentation, and charac-ters recognition. The algorithm detects the position of a license plate and extracts the characters. Various license plate recognition algorithms have been implemented, and each algorithm has its strengths and weaknesses. In this research, I implement three algorithms for detecting license plates, three algorithms for segmenting license plates, and two algorithms for recognizing license plate characters. I evaluate each of these algorithms on the same two datasets, one from Greece and one from Thailand. For detecting li-cense plates, the best result is obtained by a Haar cascade algorithm. After the best result of license plate detection is obtained, for the segmentation part a Laplacian based method has the highest accuracy. Last, the license plate recognition experiment shows that a neural network has better accuracy than other algo-rithm. I summarize and analyze the overall performance of each method for comparison