Development of a text reading system on video images

Since the early days of computer science researchers sought to devise a machine which could automatically read text to help people with visual impairments. The problem of extracting and recognising text on document images has been largely resolved, but reading text from images of natural scenes remains a challenge. Scene text can present uneven lighting, complex backgrounds or perspective and lens distortion; it usually appears as short sentences or isolated words and shows a very diverse set of typefaces. However, video sequences of natural scenes provide a temporal redundancy that can be exploited to compensate for some of these deficiencies. Here we present a complete end-to-end, real-time scene text reading system on video images based on perspective aware text tracking. The main contribution of this work is a system that automatically detects, recognises and tracks text in videos of natural scenes in real-time. The focus of our method is on large text found in outdoor environments, such as shop signs, street names and billboards. We introduce novel efficient techniques for text detection, text aggregation and text perspective estimation. Furthermore, we propose using a set of Unscented Kalman Filters (UKF) to maintain each text region¿s identity and to continuously track the homography transformation of the text into a fronto-parallel view, thereby being resilient to erratic camera motion and wide baseline changes in orientation. The orientation of each text line is estimated using a method that relies on the geometry of the characters themselves to estimate a rectifying homography. This is done irrespective of the view of the text over a large range of orientations. We also demonstrate a wearable head-mounted device for text reading that encases a camera for image acquisition and a pair of headphones for synthesized speech output. Our system is designed for continuous and unsupervised operation over long periods of time. It is completely automatic and features quick failure recovery and interactive text reading. It is also highly parallelised in order to maximize the usage of available processing power and to achieve real-time operation. We show comparative results that improve the current state-of-the-art when correcting perspective deformation of scene text. The end-to-end system performance is demonstrated on sequences recorded in outdoor scenarios. Finally, we also release a dataset of text tracking videos along with the annotated ground-truth of text regions

Object Detection Methodologies for Blind People

Vision is the most important sense. Image plays vital role in the human perception of the surrounding environment. However there are visually impaired people, industry has created a variety of computer vision products and services by developing new electronic technologies for the blind in order to overcome the difficulties. Digital image processing is the field which processes the digital image by using digital computer. An increasing interest in developing technologies attempts to help visually impaired people in their daily lives. It is shown that the object identification is the difficult task for visually impaired people . Although there are many applications that can be used for this task, there are still limitations that require more improving. For this reason, this paper provides the survey and an analysis of various evaluations for the technologies that used in the object identification task. For the visually impaired the idea of sensory substitution can be used

FingerReader: A Wearable Device to Explore Printed Text on the Go

Accessing printed text in a mobile context is a major challenge for the blind. A preliminary study with blind people reveals numerous difficulties with existing state-of-the-art technologies including problems with alignment, focus, accuracy, mobility and efficiency. In this paper, we present a finger-worn device, FingerReader, that assists blind users with reading printed text on the go. We introduce a novel computer vision algorithm for local-sequential text scanning that enables reading single lines, blocks of text or skimming the text with complementary, multimodal feedback. This system is implemented in a small finger-worn form factor, that enables a more manageable eyes-free operation with trivial setup. We offer findings from three studies performed to determine the usability of the FingerReader.SUTD-MIT International Design Centr

Aspects of n-tuple character recognition for a blind reading aid

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel University.This thesis reports research conducted into a character recognition system suitable for use in a reading aid for the blind. A brief review of blind reading aids is given, showing the need for a device which is cheap, simple and effective. The structure of a proposed reading aid fulfilling these needs is outlined, with a list of the desired characteristics of each of its subsystems. The remainder of the thesis is concerned with research into just two of these subsystems: the input device and the character recognizer. A detailed review of pattern recognition by the n-tuple method is presented, followed by a description of the experimental techniques used in obtaining real data from a camera system, and in simulating various recognizer structures. The camera system and computer programs developed specifically for the research are described in detail. Several series of experiments are reported, concerned mainly with investigating problems associated directly with the blind reading aid, namely accommodation of multifont printed text and of the tracking errors inherent in data from a hand-held probe. A further series of experiments, aimed at improving the performance of the recognizer within fixed size constraints, i. e., optimisation, has a wider field of application. Finally suggestions are made as to how the recognizer might be implemented in a reading aid, using RAMs, ROMs, or PLAs as the main storage elements.Science Research Counci

FingerReader: A Wearable Device to Support Text Reading on the Go

Visually impaired people report numerous difficulties with accessing printed text using existing technology, including problems with alignment, focus, accuracy, mobility and efficiency. We present a finger worn device that assists the visually impaired with effectively and efficiently reading paper-printed text. We introduce a novel, local-sequential manner for scanning text which enables reading single lines, blocks of text or skimming the text for important sections while providing real-time auditory and tactile feedback. The design is motivated by preliminary studies with visually impaired people, and it is small-scale and mobile, which enables a more manageable operation with little setup

Cognitive Information Processing

Contains research objectives and reports on two research projects.Joint Services Electronics Program by the U. S. Army Research Office, Durham, under Contract DA 36-039-AMC-03200(E)National Science Foundation (Grant GP-2495)National Institutes of Health (Grant MH-04737-05)National Aeronautics and Space Administration (Grant NsG-496