1,815 research outputs found

    Fine-Grained Product Class Recognition for Assisted Shopping

    Full text link
    Assistive solutions for a better shopping experience can improve the quality of life of people, in particular also of visually impaired shoppers. We present a system that visually recognizes the fine-grained product classes of items on a shopping list, in shelves images taken with a smartphone in a grocery store. Our system consists of three components: (a) We automatically recognize useful text on product packaging, e.g., product name and brand, and build a mapping of words to product classes based on the large-scale GroceryProducts dataset. When the user populates the shopping list, we automatically infer the product class of each entered word. (b) We perform fine-grained product class recognition when the user is facing a shelf. We discover discriminative patches on product packaging to differentiate between visually similar product classes and to increase the robustness against continuous changes in product design. (c) We continuously improve the recognition accuracy through active learning. Our experiments show the robustness of the proposed method against cross-domain challenges, and the scalability to an increasing number of products with minimal re-training.Comment: Accepted at ICCV Workshop on Assistive Computer Vision and Robotics (ICCV-ACVR) 201

    Product recognition in store shelves as a sub-graph isomorphism problem

    Full text link
    The arrangement of products in store shelves is carefully planned to maximize sales and keep customers happy. However, verifying compliance of real shelves to the ideal layout is a costly task routinely performed by the store personnel. In this paper, we propose a computer vision pipeline to recognize products on shelves and verify compliance to the planned layout. We deploy local invariant features together with a novel formulation of the product recognition problem as a sub-graph isomorphism between the items appearing in the given image and the ideal layout. This allows for auto-localizing the given image within the aisle or store and improving recognition dramatically.Comment: Slightly extended version of the paper accepted at ICIAP 2017. More information @project_page --> http://vision.disi.unibo.it/index.php?option=com_content&view=article&id=111&catid=7

    SANIP: Shopping Assistant and Navigation for the visually impaired

    Full text link
    The proposed shopping assistant model SANIP is going to help blind persons to detect hand held objects and also to get a video feedback of the information retrieved from the detected and recognized objects. The proposed model consists of three python models i.e. Custom Object Detection, Text Detection and Barcode detection. For object detection of the hand held object, we have created our own custom dataset that comprises daily goods such as Parle-G, Tide, and Lays. Other than that we have also collected images of Cart and Exit signs as it is essential for any person to use a cart and also notice the exit sign in case of emergency. For the other 2 models proposed the text and barcode information retrieved is converted from text to speech and relayed to the Blind person. The model was used to detect objects that were trained on and was successful in detecting and recognizing the desired output with a good accuracy and precision.Comment: 6 pages, 8 figures. arXiv admin note: text overlap with arXiv:2011.04244 by other author

    Iterative Design and Prototyping of Computer Vision Mediated Remote Sighted Assistance

    Get PDF
    Remote sighted assistance (RSA) is an emerging navigational aid for people with visual impairments (PVI). Using scenario-based design to illustrate our ideas, we developed a prototype showcasing potential applications for computer vision to support RSA interactions. We reviewed the prototype demonstrating real-world navigation scenarios with an RSA expert, and then iteratively refined the prototype based on feedback. We reviewed the refined prototype with 12 RSA professionals to evaluate the desirability and feasibility of the prototyped computer vision concepts. The RSA expert and professionals were engaged by, and reacted insightfully and constructively to the proposed design ideas. We discuss what we learned about key resources, goals, and challenges of the RSA prosthetic practice through our iterative prototype review, as well as implications for the design of RSA systems and the integration of computer vision technologies into RSA

    Planogram Compliance Checking Based on Detection of Recurring Patterns

    Get PDF
    In this paper, a novel method for automatic planogram compliance checking in retail chains is proposed without requiring product template images for training. Product layout is extracted from an input image by means of unsupervised recurring pattern detection and matched via graph matching with the expected product layout specified by a planogram to measure the level of compliance. A divide and conquer strategy is employed to improve the speed. Specifically, the input image is divided into several regions based on the planogram. Recurring patterns are detected in each region respectively and then merged together to estimate the product layout. Experimental results on real data have verified the efficacy of the proposed method. Compared with a template-based method, higher accuracies are achieved by the proposed method over a wide range of products.Comment: Accepted by MM (IEEE Multimedia Magazine) 201

    IoT enabled intelligent stick for visually impaired people for obstacle recognition

    Get PDF
    Producción CientíficaThis paper presents the design, development, and testing of an IoT-enabled smart stick for visually impaired people to navigate the outside environment with the ability to detect and warn about obstacles. The proposed design employs ultrasonic sensors for obstacle detection, a water sensor for sensing the puddles and wet surfaces in the user’s path, and a high-definition video camera integrated with object recognition. Furthermore, the user is signaled about various hindrances and objects using voice feedback through earphones after accurately detecting and identifying objects. The proposed smart stick has two modes; one uses ultrasonic sensors for detection and feedback through vibration motors to inform about the direction of the obstacle, and the second mode is the detection and recognition of obstacles and providing voice feedback. The proposed system allows for switching between the two modes depending on the environment and personal preference. Moreover, the latitude/longitude values of the user are captured and uploaded to the IoT platform for effective tracking via global positioning system (GPS)/global system for mobile communication (GSM) modules, which enable the live location of the user/stick to be monitored on the IoT dashboard. A panic button is also provided for emergency assistance by generating a request signal in the form of an SMS containing a Google maps link generated with latitude and longitude coordinates and sent through an IoT-enabled environment. The smart stick has been designed to be lightweight, waterproof, size adjustable, and has long battery life. The overall design ensures energy efficiency, portability, stability, ease of access, and robust features

    Skip Trie Matching for Real-Time OCR Output Error Corrrection on Smartphones

    Get PDF
    Many Visually Impaired individuals are managing their daily activities with the help of smartphones. While there are many vision-based mobile applications to identify products, there is a relative dearth of applications for extracting useful nutrition information. In this report, we study the performance of existing OCR systems available for the Android platform, and choose the best to extract the nutrition facts information from U.S grocery store packages. We then provide approaches to improve the results of text strings produced by the Tesseract OCR engine on image segments of nutrition tables automatically extracted by an Android 2.3.6 smartphone application using real-time video streams of grocery products. We also present an algorithm, called Skip Trie Matching (STM), for real-time OCR output error correction on smartphones. The algorithm’s performance is compared with Apache Lucene’s spell checker. Our evaluation indicates that the average run time of the STM algorithm is lower than Lucene’s. (68 pages

    Overcoming barriers and increasing independence: service robots for elderly and disabled people

    Get PDF
    This paper discusses the potential for service robots to overcome barriers and increase independence of elderly and disabled people. It includes a brief overview of the existing uses of service robots by disabled and elderly people and advances in technology which will make new uses possible and provides suggestions for some of these new applications. The paper also considers the design and other conditions to be met for user acceptance. It also discusses the complementarity of assistive service robots and personal assistance and considers the types of applications and users for which service robots are and are not suitable

    Secure Navigation System for Visually Impaired

    Get PDF
    A cane is commonly used to help blind people navigate on a path. A stick is used to inform them about any pits, obstacles or elevation. This paper presents a secure system which detects any obstacles using an Ultrasonic sensor. GPS-GSM module is used that gives us the location of the person and sends a message alert on the phone of their families on regular intervals. Also, obstacle detection warning is given through a voice command using speaker or headphones. This system is designed to help visually impaired navigate a path safely and without much difficulties. In the present ongoing systems the security and safety of the visually impaired person is not adequately taken care of. So we will be preparing an Electronic Travelling Aid (ETA) with ultrasonic sensor and GPS for increasing the security and safety of the visually impaired person. This system will be more efficient and cost effective as compared to its former systems
    corecore