287 research outputs found

    An Application of Voice Mail: Email Services for the Visually Challenged Individual

    Get PDF
    Communication plays a crucial role in every field in one’s life. It is an integration of the communicating technologies with the help of internet. But this facility is not for blind people. Hence, we aimed to develop an Android based email application that can facilitate visually challenged people to use email services for communication. The application will work solely on voice commands spoken by the user which will enable them to communicate with the world. They can send and receive any mails whether it is a text document, picture, audio, video, etc. using this system using the internet. By providing the platform in which they can speak the operation and can able to send and receive the messages. The system will be build using Google Text-to-Speech and Speech-to-Text APIs, which will make it efficient, accurate to a certain limit and user friendly

    Eyes-Free Vision-Based Scanning of Aligned Barcodes and Information Extraction from Aligned Nutrition Tables

    Get PDF
    Visually impaired (VI) individuals struggle with grocery shopping and have to rely on either friends, family or grocery store associates for shopping. ShopMobile 2 is a proof-of-concept system that allows VI shoppers to shop independently in a grocery store using only their smartphone. Unlike other assistive shopping systems that use dedicated hardware, this system is a software only solution that relies on fast computer vision algorithms. It consists of three modules - an eyes free barcode scanner, an optical character recognition (OCR) module, and a tele-assistance module. The eyes-free barcode scanner allows VI shoppers to locate and retrieve products by scanning barcodes on shelves and on products. The OCR module allows shoppers to read nutrition facts on products and the tele-assistance module allows them to obtain help from sighted individuals at remote locations. This dissertation discusses, provides implementations of, and presents laboratory and real-world experiments related to all three modules

    A review of the current trends and future directions of camera barcode reading

    Get PDF
    Modern mobile phones or smartphones have become a pervasive and affordable device for users at different levels of age around the world. Smartphones equipped with many useful sensors, including camera, barometer, accelerometer, and digital compass. The sensors on smartphones attracted researchers and developers to develop mobile applications (apps) and study the potential use of the sensors to support daily life activities. Unlike other types of sensor, the smartphone camera has been underutilized. Analysis of the literature suggested that smartphone camera mainly serves for personal and social photography. Practically, a smartphone camera can be used as an imaging device for reading a barcode. Although barcode has been used for identifying products and items, the use of a smartphone camera as a reading device has not been explored thoroughly. Further, scholarly resources describing the fundamental knowledge of smartphone camera barcode reading is not available in the literature which could be the reason contributed to slow research progress of the domain. Therefore, this study aims to review the current trends and future directions of smartphone camera for barcode reading. Specifically, the study reviews the literature on the types of applications that are currently available and run on the standard mobile platform for reading a barcode. It also analyzes the necessary components that made up barcode reading apps. Further, the review identifies technical and non-technical issues that are critical for the development of the apps. The contributions of this work are twofold, first, it provides the fundamental knowledge on the building blocks of camera barcode reading apps, and second, it explores the issues in the current camera barcode reading apps that could encourage exploration towards addressing the issues. Practically, the findings could spark new research ideas to address the current issues related to the use of smartphone camera for barcode reading in the near future

    Vision Based Extraction of Nutrition Information from Skewed Nutrition Labels

    Get PDF
    An important component of a healthy diet is the comprehension and retention of nutritional information and understanding of how different food items and nutritional constituents affect our bodies. In the U.S. and many other countries, nutritional information is primarily conveyed to consumers through nutrition labels (NLs) which can be found in all packaged food products. However, sometimes it becomes really challenging to utilize all this information available in these NLs even for consumers who are health conscious as they might not be familiar with nutritional terms or find it difficult to integrate nutritional data collection into their daily activities due to lack of time, motivation, or training. So it is essential to automate this data collection and interpretation process by integrating Computer Vision based algorithms to extract nutritional information from NLs because it improves the user’s ability to engage in continuous nutritional data collection and analysis. To make nutritional data collection more manageable and enjoyable for the users, we present a Proactive NUTrition Management System (PNUTS). PNUTS seeks to shift current research and clinical practices in nutrition management toward persuasion, automated nutritional information processing, and context-sensitive nutrition decision support. PNUTS consists of two modules, firstly a barcode scanning module which runs on smart phones and is capable of vision-based localization of One Dimensional (1D) Universal Product Code (UPC) and International Article Number (EAN) barcodes with relaxed pitch, roll, and yaw camera alignment constraints. The algorithm localizes barcodes in images by computing Dominant Orientations of Gradients (DOGs) of image segments and grouping smaller segments with similar DOGs into larger connected components. Connected components that pass given morphological criteria are marked as potential barcodes. The algorithm is implemented in a distributed, cloud-based system. The system’s front end is a smartphone application that runs on Android smartphones with Android 4.2 or higher. The system’s back end is deployed on a five node Linux cluster where images are processed. The algorithm was evaluated on a corpus of 7,545 images extracted from 506 videos of bags, bottles, boxes, and cans in a supermarket. The DOG algorithm was coupled to our in-place scanner for 1D UPC and EAN barcodes. The scanner receives from the DOG algorithm the rectangular planar dimensions of a connected component and the component’s dominant gradient orientation angle referred to as the skew angle. The scanner draws several scan lines at that skew angle within the component to recognize the barcode in place without any rotations. The scanner coupled to the localizer was tested on the same corpus of 7,545 images. Laboratory experiments indicate that the system can localize and scan barcodes of any orientation in the yaw plane, of up to 73.28 degrees in the pitch plane, and of up to 55.5 degrees in the roll plane. The videos have been made public for all interested research communities to replicate our findings or to use them in their own research. The front end Android application is available for free download at Google Play under the title of NutriGlass. This module is also coupled to a comprehensive NL database from which nutritional information can be retrieved on demand. Currently our NL database consists of more than 230,000 products. The second module of PNUTS is an algorithm whose objective is to determine the text skew angle of an NL image without constraining the angle’s magnitude. The horizontal, vertical, and diagonal matrices of the (Two Dimensional) 2D Haar Wavelet Transform are used to identify 2D points with significant intensity changes. The set of points is bounded with a minimum area rectangle whose rotation angle is the text’s skew. The algorithm’s performance is compared with the performance of five text skew detection algorithms on 1001 U.S. nutrition label images and 2200 single- and multi-column document images in multiple languages. To ensure the reproducibility of the reported results, the source code of the algorithm and the image data have been made publicly available. If the skew angle is estimated correctly, optical character recognition (OCR) techniques can be used to extract nutrition information

    BLINDSHOPPING: NAVIGATION SYSTEM

    Get PDF
    The QR trail is an android application that designed to encourage visually challenged person to participate in more normal activities as normal person does. Moreover, this application can be used by normal person as well to navigate around places when the person lost in a way. The main purpose of the project is to provide a navigation system for the visually challenged person to move around autonomously in supermarkets or hypermarkets and do some shopping. The application will provide a guidance for visually impaired person through voice command from the smartphone as the user need to scan QR codes on the floor which contains the details of current location and instruction to move from one point of the shopping mall to another point. The development of this application will use Eclipse development tool. The programming language that will be used the development process in Java language and ZXing library. The rapid application development methodology is applied in development process of this application which consists 4 stages which are system design, prototype cycle, system testing and implication

    A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research

    Get PDF
    This paper summarizes recent developments in audio and tactile feedback based assistive technologies targeting the blind community. Current technology allows applications to be efficiently distributed and run on mobile and handheld devices, even in cases where computational requirements are significant. As a result, electronic travel aids, navigational assistance modules, text-to-speech applications, as well as virtual audio displays which combine audio with haptic channels are becoming integrated into standard mobile devices. This trend, combined with the appearance of increasingly user- friendly interfaces and modes of interaction has opened a variety of new perspectives for the rehabilitation and training of users with visual impairments. The goal of this paper is to provide an overview of these developments based on recent advances in basic research and application development. Using this overview as a foundation, an agenda is outlined for future research in mobile interaction design with respect to users with special needs, as well as ultimately in relation to sensor-bridging applications in genera

    Skip Trie Matching for Real-Time OCR Output Error Corrrection on Smartphones

    Get PDF
    Many Visually Impaired individuals are managing their daily activities with the help of smartphones. While there are many vision-based mobile applications to identify products, there is a relative dearth of applications for extracting useful nutrition information. In this report, we study the performance of existing OCR systems available for the Android platform, and choose the best to extract the nutrition facts information from U.S grocery store packages. We then provide approaches to improve the results of text strings produced by the Tesseract OCR engine on image segments of nutrition tables automatically extracted by an Android 2.3.6 smartphone application using real-time video streams of grocery products. We also present an algorithm, called Skip Trie Matching (STM), for real-time OCR output error correction on smartphones. The algorithm’s performance is compared with Apache Lucene’s spell checker. Our evaluation indicates that the average run time of the STM algorithm is lower than Lucene’s. (68 pages

    Mobile Apps Catalog

    Get PDF
    The Mobile Apps Catalog is a collection of emergency management and assistive mobile applications intended to assist first responders, emergency managers, and the public, specifically people with disabilities or others with access and function needs. Highlighted in this catalog are readily available preparedness and response apps that can be accessed by wireless devices, as well as assistive resources to advance the usability of wireless devices for consumers with disabilities. The apps are also helpful for the whole community. “Federal Emergency Management Agency (FEMA)’s “Whole Community” approach to emergency management recognizes that individuals, families and communities are assets and keys to success (Fugate 2011).
    corecore