11,303 research outputs found
Text detection and recognition in natural scene images
This thesis addresses the problem of end-to-end text detection and recognition in
natural scene images based on deep neural networks. Scene text detection and recognition
aim to find regions in an image that are considered as text by human beings,
generate a bounding box for each word and output a corresponding sequence of
characters. As a useful task in image analysis, scene text detection and recognition
attract much attention in computer vision field. In this thesis, we tackle this problem
by taking advantage of the success in deep learning techniques.
Car license plates can be viewed as a spacial case of scene text, as they both consist
of characters and appear in natural scenes. Nevertheless, they have their respective
specificities. During the research progress, we start from car license plate detection
and recognition. Then we extend the methods to general scene text, with additional
ideas proposed.
For both tasks, we develop two approaches respectively: a stepwise one and
an integrated one. Stepwise methods tackle text detection and recognition step by
step by respective models; while integrated methods handle both text detection and
recognition simultaneously via one model. All approaches are based on the powerful
deep Convolutional Neural Networks (CNNs) and Recurrent Neural Networks
(RNNs), considering the tremendous breakthroughs they brought into the computer
vision community.
To begin with, a stepwise framework is proposed to tackle text detection and
recognition, with its application to car license plates and general scene text respectively.
A character CNN classifier is well trained to detect characters from an image
in a sliding window manner. The detected characters are then grouped together as
license plates or text lines according to some heuristic rules. A sequence labeling
based method is proposed to recognize the whole license plate or text line without
character level segmentation.
On the basis of the sequence labeling based recognition method, to accelerate the
processing speed, an integrated deep neural network is then proposed to address
car license plate detection and recognition concurrently. It integrates both CNNs
and RNNs in one network, and can be trained end-to-end. Both car license plate
bounding boxes and their labels are generated in a single forward evaluation of the
network. The whole process involves no heuristic rule, and avoids intermediate
procedures like image cropping or feature recalculation, which not only prevents
error accumulation, but also reduces computation burden.
Lastly, the unified network is extended to simultaneous general text detection and
recognition in natural scene. In contrast to the one for car license plates, some innovations
are proposed to accommodate the special characteristics of general text. A
varying-size RoI encoding method is proposed to handle the various aspect ratios of general text. An attention-based sequence-to-sequence learning structure is adopted
for word recognition. It is expected that a character-level language model can be
learnt in this manner. The whole framework can be trained end-to-end, requiring
only images, the ground-truth bounding boxes and text labels. Through end-to-end
training, the learned features can be more discriminative, which improves the overall
performance. The convolutional features are calculated only once and shared by both
detection and recognition, which saves the processing time. The proposed method
has achieved state-of-the-art performance on several standard benchmark datasets.Thesis (Ph.D.) -- University of Adelaide, School of Computer Science, 201
Vehicle make and model recognition for intelligent transportation monitoring and surveillance.
Vehicle Make and Model Recognition (VMMR) has evolved into a significant subject of study due to its importance in numerous Intelligent Transportation Systems (ITS), such as autonomous navigation, traffic analysis, traffic surveillance and security systems. A highly accurate and real-time VMMR system significantly reduces the overhead cost of resources otherwise required. The VMMR problem is a multi-class classification task with a peculiar set of issues and challenges like multiplicity, inter- and intra-make ambiguity among various vehicles makes and models, which need to be solved in an efficient and reliable manner to achieve a highly robust VMMR system. In this dissertation, facing the growing importance of make and model recognition of vehicles, we present a VMMR system that provides very high accuracy rates and is robust to several challenges. We demonstrate that the VMMR problem can be addressed by locating discriminative parts where the most significant appearance variations occur in each category, and learning expressive appearance descriptors. Given these insights, we consider two data driven frameworks: a Multiple-Instance Learning-based (MIL) system using hand-crafted features and an extended application of deep neural networks using MIL. Our approach requires only image level class labels, and the discriminative parts of each target class are selected in a fully unsupervised manner without any use of part annotations or segmentation masks, which may be costly to obtain. This advantage makes our system more intelligent, scalable, and applicable to other fine-grained recognition tasks. We constructed a dataset with 291,752 images representing 9,170 different vehicles to validate and evaluate our approach. Experimental results demonstrate that the localization of parts and distinguishing their discriminative powers for categorization improve the performance of fine-grained categorization. Extensive experiments conducted using our approaches yield superior results for images that were occluded, under low illumination, partial camera views, or even non-frontal views, available in our real-world VMMR dataset. The approaches presented herewith provide a highly accurate VMMR system for rea-ltime applications in realistic environments.\\ We also validate our system with a significant application of VMMR to ITS that involves automated vehicular surveillance. We show that our application can provide law inforcement agencies with efficient tools to search for a specific vehicle type, make, or model, and to track the path of a given vehicle using the position of multiple cameras
Learning with Weak Annotations for Text in the Wild Detection and Recognition
V tĂ©to práci pĹ™edstavujeme metodu vyuĹľĂvajĂcĂ slabÄ› anotovanĂ© obrázky pro vylepšenĂ systĂ©mĹŻ pro extrakci textu. Slabá antoace spoÄŤĂvá v seznamu textĹŻ, kterĂ© se v danĂ©m obrázku mohou vyskytovat, ale nevĂme kde. Metoda pouĹľĂvá libovolnĂ˝ existujĂcĂ systĂ©m pro rozpoznávánĂ textu k zĂskánĂ oblastĂ, kde se pravdÄ›podobnÄ› vyskytuje text, spolu s ne nutnÄ› správnĂ˝m pĹ™episem. VĂ˝sledkem procesu zahrnujĂcĂho párovánĂ nepĹ™esnĂ˝ch pĹ™episĹŻ se slabĂ˝mi anotacemi a prohledávánĂ okolĂ vedenĂ© Levenshtein vzdálenostĂ jsou skoro bezchybnÄ› lokalizovanĂ© texty, se kterĂ˝mi dále zacházĂme jako s pseudo-anotacemi vyuĹľĂvanĂ˝mi k uÄŤenĂ. AplikovánĂ metody na dva slabÄ› anotovanĂ© datasety a douÄŤenĂ pouĹľitĂ©ho systĂ©mu pomocĂ zĂskanĂ˝ch pseudo-anotacĂ ukazuje, Ĺľe námi navrĹľenĂ˝ proces konzistentnÄ› zlepšuje pĹ™esnost rozpoznávánĂ na rĹŻznĂ˝ch datasetech (jinĂ˝ch domĂ©nách) běžnÄ› vyuĹľĂvanĂ˝ch k testovánĂ a velmi vĂ˝raznÄ› zvyšuje pĹ™esnost na stejnĂ©m datasetu. Metodu lze pouĹľĂt iterativnÄ›.In this work, we present a method for exploiting weakly annotated images to improve text extraction pipelines. The weak annotation of an image is a list of texts that are likely to appear in the image without any information about the location. An arbitrary existing end-to-end text recognition system is used to obtain text region proposals and their, possibly erroneous, transcriptions. A process that includes imprecise transcription to annotation matching and edit distance guided neighbourhood search produces nearly error-free, localised instances of scene text, which we treat as ``pseudo ground truth'' used for training. We apply the method to two weakly-annotated datasets and use the obtained pseudo ground truth to re-train the end-to-end system. The process consistently improves the accuracy of a state of the art recognition model across different benchmark datasets (image domains) as well as providing a significant performance boost on the same dataset, further improving when applied iteratively
Designing a labeling application for image object detection
We seek to build a large collection of images with ground truth labels to be used for training object detection and recognition algorithms. Such data is useful for supervised learning and quantitative evaluation. To achieve this, we developed a user interface tool that allows easy image annotation.
The tool provides functionalities such as drawing boxes, querying images, and browsing the database.
Using this annotation tool, we can collect a large dataset that spans many object categories, often containing multiple instances over a wide variety of images.
We quantify the contents of an existing dataset and compare against other state of the art datasets used for object recognition and detection.
Also, we show how to extend our dataset to automatically enhance object labels with WordNet, discover object parts, and increase the number of labels using minimal user supervisionope
- …