Vehicle-Rear: A New Dataset to Explore Feature Fusion for Vehicle Identification Using Convolutional Neural Networks

This work addresses the problem of vehicle identification through non-overlapping cameras. As our main contribution, we introduce a novel dataset for vehicle identification, called Vehicle-Rear, that contains more than three hours of high-resolution videos, with accurate information about the make, model, color and year of nearly 3,000 vehicles, in addition to the position and identification of their license plates. To explore our dataset we design a two-stream CNN that simultaneously uses two of the most distinctive and persistent features available: the vehicle's appearance and its license plate. This is an attempt to tackle a major problem: false alarms caused by vehicles with similar designs or by very close license plate identifiers. In the first network stream, shape similarities are identified by a Siamese CNN that uses a pair of low-resolution vehicle patches recorded by two different cameras. In the second stream, we use a CNN for OCR to extract textual information, confidence scores, and string similarities from a pair of high-resolution license plate patches. Then, features from both streams are merged by a sequence of fully connected layers for decision. In our experiments, we compared the two-stream network against several well-known CNN architectures using single or multiple vehicle features. The architectures, trained models, and dataset are publicly available at https://github.com/icarofua/vehicle-rear

Licence Plate Detection Using Machine Learning

License Plate Recognition (LPR) is one of the tough tasks in the field of computer vision. Although it has been around for quite a while, there still lies the challenges when we have to deal with; the harsh environmental conditions like snowy, rainfall, windy, low light conditions etc. as well as the condition of the plates which includes the bent, rotated, broken plates. The performance of the recognition and detection frameworks take a significant hit when it is concerned with these conditional effects on the license plate. In this paper, we introduced a model to improve our accuracy based on the Chinese Car Parking Dataset (CCPD) using 2 separate convolutional neural networks. The first CNN will be able to detect the bounding boxes for the license plate detection using Non-Maximus Suppression (NMS) to find the most probable bounding area whereas the second one will take these bounding boxes and use the spatial attenuation network and character recognition model to successfully recognize the license plate. First, we train the CNN to detect the license plates, then use the second CNN to recognize the characters. The overall recognition accuracy was found to be 89% in the CCPD dataset

SHINE: Deep Learning-Based Accessible Parking Management System

The ongoing expansion of urban areas facilitated by advancements in science and technology has resulted in a considerable increase in the number of privately owned vehicles worldwide, including in South Korea. However, this gradual increment in the number of vehicles has inevitably led to parking-related issues, including the abuse of disabled parking spaces (hereafter referred to as accessible parking spaces) designated for individuals with disabilities. Traditional license plate recognition (LPR) systems have proven inefficient in addressing such a problem in real-time due to the high frame rate of surveillance cameras, the presence of natural and artificial noise, and variations in lighting and weather conditions that impede detection and recognition by these systems. With the growing concept of parking 4.0, many sensors, IoT and deep learning-based approaches have been applied to automatic LPR and parking management systems. Nonetheless, the studies show a need for a robust and efficient model for managing accessible parking spaces in South Korea. To address this, we have proposed a novel system called, Shine, which uses the deep learning-based object detection algorithm for detecting the vehicle, license plate, and disability badges (referred to as cards, badges, or access badges hereafter) and verifies the rights of the driver to use accessible parking spaces by coordinating with the central server. Our model, which achieves a mean average precision of 92.16%, is expected to address the issue of accessible parking space abuse and contributes significantly towards efficient and effective parking management in urban environments

Text detection and recognition in natural scene images

This thesis addresses the problem of end-to-end text detection and recognition in natural scene images based on deep neural networks. Scene text detection and recognition aim to find regions in an image that are considered as text by human beings, generate a bounding box for each word and output a corresponding sequence of characters. As a useful task in image analysis, scene text detection and recognition attract much attention in computer vision field. In this thesis, we tackle this problem by taking advantage of the success in deep learning techniques. Car license plates can be viewed as a spacial case of scene text, as they both consist of characters and appear in natural scenes. Nevertheless, they have their respective specificities. During the research progress, we start from car license plate detection and recognition. Then we extend the methods to general scene text, with additional ideas proposed. For both tasks, we develop two approaches respectively: a stepwise one and an integrated one. Stepwise methods tackle text detection and recognition step by step by respective models; while integrated methods handle both text detection and recognition simultaneously via one model. All approaches are based on the powerful deep Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), considering the tremendous breakthroughs they brought into the computer vision community. To begin with, a stepwise framework is proposed to tackle text detection and recognition, with its application to car license plates and general scene text respectively. A character CNN classifier is well trained to detect characters from an image in a sliding window manner. The detected characters are then grouped together as license plates or text lines according to some heuristic rules. A sequence labeling based method is proposed to recognize the whole license plate or text line without character level segmentation. On the basis of the sequence labeling based recognition method, to accelerate the processing speed, an integrated deep neural network is then proposed to address car license plate detection and recognition concurrently. It integrates both CNNs and RNNs in one network, and can be trained end-to-end. Both car license plate bounding boxes and their labels are generated in a single forward evaluation of the network. The whole process involves no heuristic rule, and avoids intermediate procedures like image cropping or feature recalculation, which not only prevents error accumulation, but also reduces computation burden. Lastly, the unified network is extended to simultaneous general text detection and recognition in natural scene. In contrast to the one for car license plates, some innovations are proposed to accommodate the special characteristics of general text. A varying-size RoI encoding method is proposed to handle the various aspect ratios of general text. An attention-based sequence-to-sequence learning structure is adopted for word recognition. It is expected that a character-level language model can be learnt in this manner. The whole framework can be trained end-to-end, requiring only images, the ground-truth bounding boxes and text labels. Through end-to-end training, the learned features can be more discriminative, which improves the overall performance. The convolutional features are calculated only once and shared by both detection and recognition, which saves the processing time. The proposed method has achieved state-of-the-art performance on several standard benchmark datasets.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 201

Towards End-to-end Car License Plate Location and Recognition in Unconstrained Scenarios

Benefiting from the rapid development of convolutional neural networks, the performance of car license plate detection and recognition has been largely improved. Nonetheless, challenges still exist especially for real-world applications. In this paper, we present an efficient and accurate framework to solve the license plate detection and recognition tasks simultaneously. It is a lightweight and unified deep neural network, that can be optimized end-to-end and work in real-time. Specifically, for unconstrained scenarios, an anchor-free method is adopted to efficiently detect the bounding box and four corners of a license plate, which are used to extract and rectify the target region features. Then, a novel convolutional neural network branch is designed to further extract features of characters without segmentation. Finally, recognition task is treated as sequence labelling problems, which are solved by Connectionist Temporal Classification (CTC) directly. Several public datasets including images collected from different scenarios under various conditions are chosen for evaluation. A large number of experiments indicate that the proposed method significantly outperforms the previous state-of-the-art methods in both speed and precision

Multi-Object Tracking based Roadside Parking Behavior Recognition

Roadside parking spaces can alleviate the shortage of parking spaces, but there are some shortcomings to the charges for roadside parking. The popular charging methods at present mainly include manual charging, geomagnetic detection charging, meter charging, etc. These methods have certain limitations, such as high cost, difficult deployment, and low acceptance of people. To solve the shortcomings of roadside parking charges, this thesis proposes a scheme based on deep learning and image recognition. More specifically, the thesis proposes a scheme for detecting and tracking vehicles, recognizing license plates, recognizing vehicle parking behavior, and recording vehicle parking periods through the monocular camera to solve the problem of roadside parking charges. The scheme has the advantages of convenient deployment, low labor cost, high efficiency, and high accuracy. The main work of this thesis is as follows: 1. Based on the You Only Look Once (YOLO) algorithm, this thesis proposes a trapezoidal convolution algorithm to detect objects and improve the detection efficiency for the problem that the vehicle is far and small in the image. 2. Proposes a one-stage license plate recognition scheme based on YOLO, aiming to simplify the license plate recognition process. 3. Depending on the characteristics of the vehicle, this thesis proposes a feature extraction model of the vehicle, called the horizontal and vertical separation model, which use to combine with the deep Simple Online and Real-time Tracking (SORT) object tracking framework to track the vehicle and improve the tracking efficiency. 4. Uses a Long Short-Term Memory (LSTM) model to classify the behavior of the vehicle into three types: Park, leave, and no behavior. 5. Groups these modules together, and the engineering code is debugged a lot to realize a complete Roadside Parking Behavior Recognition (RPBR) system