Design, modeling and analysis of object localization through acoustical signals for cognitive electronic travel aid for blind people

El objetivo de la tesis consiste en el estudio y análisis de la localización de objetos en el entorno real mediante sonidos, así como la posterior integración y ensayo de un dispositivo real basado en tal técnica y destinado a personas con discapacidad visual. Con el propósito de poder comprender y analizar la localización de objetos se ha realizado un profundo estado de arte sobre los Sistemas de Navegación desarrollados durante las últimas décadas y orientados a personas con distintos grados de discapacidad visual. En el citado estado del arte, se han analizado y estructurado los dispositivos de navegación existentes, clasificándolos de acuerdo con los componentes de adquisición de datos del entorno utilizados. A este respecto, hay que señalar que, hasta el momento, se conocen tres clases de dispositivos de navegación: 'detectores de obstáculos', que se basan en dispositivos de ultrasonidos y sensores instalados en los dispositivos electrónicos de navegación con el objetivo de detectar los objetos que aparecen en el área de trabajo del sistema; 'sensores del entorno' - que tienen como objetivo la detección del objeto y del usuario. Esta clase de dispositivos se instalan en las estaciones de autobús, metro, tren, pasos de peatones etc., de forma que cuando el sensor del usuario penetra en el área de alcance de los sensores instalados en la estación, éstos informan al usuario sobre la presencia de la misma. Asimismo, el sensor del usuario detecta también los medios de transporte que tienen instalado el correspondiente dispositivo basado en láser o ultrasonidos, ofreciendo al usuario información relativa a número de autobús, ruta etc La tercera clase de sistemas electrónicos de navegación son los 'dispositivos de navegación'. Estos elementos se basan en dispositivos GPS, indicando al usuario tanto su locación, como la ruta que debe seguir para llegar a su punto de destino. Tras la primera etapa de elaboración del estaDunai ., L. (2010). Design, modeling and analysis of object localization through acoustical signals for cognitive electronic travel aid for blind people [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/8441Palanci

Real-Time Obstacle Detection System in Indoor Environment for the Visually Impaired Using Microsoft Kinect Sensor

Any mobility aid for the visually impaired people should be able to accurately detect and warn about nearly obstacles. In this paper, we present a method for support system to detect obstacle in indoor environment based on Kinect sensor and 3D-image processing. Color-Depth data of the scene in front of the user is collected using the Kinect with the support of the standard framework for 3D sensing OpenNI and processed by PCL library to extract accurate 3D information of the obstacles. The experiments have been performed with the dataset in multiple indoor scenarios and in different lighting conditions. Results showed that our system is able to accurately detect the four types of obstacle: walls, doors, stairs, and a residual class that covers loose obstacles on the floor. Precisely, walls and loose obstacles on the floor are detected in practically all cases, whereas doors are detected in 90.69% out of 43 positive image samples. For the step detection, we have correctly detected the upstairs in 97.33% out of 75 positive images while the correct rate of downstairs detection is lower with 89.47% from 38 positive images. Our method further allows the computation of the distance between the user and the obstacles

A comparative study in real-time scene sonification for visually impaired people

In recent years, with the development of depth cameras and scene detection algorithms, a wide variety of electronic travel aids for visually impaired people have been proposed. However, it is still challenging to convey scene information to visually impaired people efficiently. In this paper, we propose three different auditory-based interaction methods, i.e., depth image sonification, obstacle sonification as well as path sonification, which convey raw depth images, obstacle information and path information respectively to visually impaired people. Three sonification methods are compared comprehensively through a field experiment attended by twelve visually impaired participants. The results show that the sonification of high-level scene information, such as the direction of pathway, is easier to learn and adapt, and is more suitable for point-to-point navigation. In contrast, through the sonification of low-level scene information, such as raw depth images, visually impaired people can understand the surrounding environment more comprehensively. Furthermore, there is no interaction method that is best suited for all participants in the experiment, and visually impaired individuals need a period of time to find the most suitable interaction method. Our findings highlight the features and the differences of three scene detection algorithms and the corresponding sonification methods. The results provide insights into the design of electronic travel aids, and the conclusions can also be applied in other fields, such as the sound feedback of virtual reality applications

Unifying terrain awareness for the visually impaired through real-time semantic segmentation.

Navigational assistance aims to help visually-impaired people to ambulate the environment safely and independently. This topic becomes challenging as it requires detecting a wide variety of scenes to provide higher level assistive awareness. Vision-based technologies with monocular detectors or depth sensors have sprung up within several years of research. These separate approaches have achieved remarkable results with relatively low processing time and have improved the mobility of impaired people to a large extent. However, running all detectors jointly increases the latency and burdens the computational resources. In this paper, we put forward seizing pixel-wise semantic segmentation to cover navigation-related perception needs in a unified way. This is critical not only for the terrain awareness regarding traversable areas, sidewalks, stairs and water hazards, but also for the avoidance of short-range obstacles, fast-approaching pedestrians and vehicles. The core of our unification proposal is a deep architecture, aimed at attaining efficient semantic understanding. We have integrated the approach in a wearable navigation system by incorporating robust depth segmentation. A comprehensive set of experiments prove the qualified accuracy over state-of-the-art methods while maintaining real-time speed. We also present a closed-loop field test involving real visually-impaired users, demonstrating the effectivity and versatility of the assistive framework

Ayuda técnica para la autonomía en el desplazamiento

The project developed in this thesis involves the design, implementation and evaluation of a new technical assistance aiming to ease the mobility of people with visual impairments. By using processing and sounds synthesis, the users can hear the sonification protocol (through bone conduction) informing them, after training, about the position and distance of the various obstacles that may be on their way, avoiding eventual accidents. In this project, surveys were conducted with experts in the field of rehabilitation, blindness and techniques of image processing and sound, which defined the user requirements that served as guideline for the design. The thesis consists of three self-contained blocks: (i) image processing, where 4 processing algorithms are proposed for stereo vision, (ii) sonification, which details the proposed sound transformation of visual information, and (iii) a final central chapter on integrating the above and sequentially evaluated in two versions or implementation modes (software and hardware). Both versions have been tested with both sighted and blind participants, obtaining qualitative and quantitative results, which define future improvements to the project. ---------------------------------------------------------------------------------------------------------------------------------------------El proyecto desarrollado en la presente tesis doctoral consiste en el diseño, implementación y evaluación de una nueva ayuda técnica orientada a facilitar la movilidad de personas con discapacidad visual. El sistema propuesto consiste en un procesador de estereovisión y un sintetizador de sonidos, mediante los cuales, las usuarias y los usuarios pueden escuchar un código de sonidos mediante transmisión ósea que les informa, previo entrenamiento, de la posición y distancia de los distintos obstáculos que pueda haber en su camino, evitando accidentes. En dicho proyecto, se han realizado encuestas a expertos en el campo de la rehabilitación, la ceguera y en las técnicas y tecnologías de procesado de imagen y sonido, mediante las cuales se definieron unos requisitos de usuario que sirvieron como guía de propuesta y diseño. La tesis está compuesta de tres grandes bloques autocontenidos: (i) procesado de imagen, donde se proponen 4 algoritmos de procesado de visión estéreo, (ii) sonificación, en el cual se detalla la propuesta de transformación a sonido de la información visual, y (iii) un último capítulo central sobre integración de todo lo anterior en dos versiones evaluadas secuencialmente, una software y otra hardware. Ambas versiones han sido evaluadas con usuarios tanto videntes como invidentes, obteniendo resultados cualitativos y cuantitativos que permiten definir mejoras futuras sobre el proyecto finalmente implementado

A Sound Approach Toward a Mobility Aid for Blind and Low-Vision Individuals

Reduced independent mobility of blind and low-vision individuals (BLVIs) cause considerable societal cost, burden on relatives, and reduced quality of life for the individuals, including increased anxiety, depression symptoms, need of assistance, risk of falls, and mortality. Despite the numerous electronic travel aids proposed since at least the 1940’s, along with ever-advancing technology, the mobility issues persist. A substantial reason for this is likely several and severe shortcomings of the field, both in regards to aid design and evaluation.In this work, these shortcomings are addressed with a generic design model called Desire of Use (DoU), which describes the desire of a given user to use an aid for a given activity. It is then applied on mobility of BLVIs (DoU-MoB), to systematically illuminate and structure possibly all related aspects that such an aid needs to aptly deal with, in order for it to become an adequate aid for the objective. These aspects can then both guide user-centered design as well as choice of test methods and measures.One such measure is then demonstrated in the Desire of Use Questionnaire for Mobility of Blind and Low-Vision Individuals (DoUQ-MoB), an aid-agnostic and comprehensive patient-reported outcome measure. The question construction originates from the DoU-MoB to ensure an encompassing focus on mobility of BLVIs, something that has been missing in the field. Since it is aid-agnostic it facilitates aid comparison, which it also actively promotes. To support the reliability of the DoUQ-MoB, it utilizes the best known practices of questionnaire design and has been validated once with eight orientation and mobility professionals, and six BLVIs. Based on this, the questionnaire has also been revised once.To allow for relevant and reproducible methodology, another tool presented herein is a portable virtual reality (VR) system called the Parrot-VR. It uses a hybrid control scheme of absolute rotation by tracking the user’s head in reality, affording intuitive turning; and relative movement where simple button presses on a controller moves the virtual avatar forward and backward, allowing for large-scale traversal while not walking physically. VR provides excellent reproducibility, making various aggregate movement analysis feasible, while it is also inherently safe. Meanwhile, the portability of the system facilitates testing near the participants, substantially increasing the number of potential blind and low-vision recruits for user tests.The thesis also gives a short account on the state of long-term testing in the field; it being short is mainly due to that there is not much to report. It then provides an initial investigation into possible outcome measures for such tests by taking instruments in use by Swedish orientation and mobility professionals as a starting point. Two of these are also piloted in an initial single-session trial with 19 BLVIs, and could plausibly be used for long-term tests after further evaluation.Finally, a discussion is presented regarding the Audomni project — the development of a primary mobility aid for BLVIs. Audomni is a visuo-auditory sensory supplementation device, which aims to take visual information and translate it to sound. A wide field-of-view, 3D-depth camera records the environment, which is then transformed to audio through the sonification algorithms of Audomni, and finally presented in a pair of open-ear headphones that do not block out environmental sounds. The design of Audomni leverages the DoU-MoB to ensure user-centric development and evaluation, in the aim of reaching an aid with such form and function that it grants the users better mobility, while the users still want to use it.Audomni has been evaluated with user tests twice, once in pilot tests with two BLVIs, and once in VR with a heterogenous set of 19 BLVIs, utilizing the Parrot-VR and the DoUQ-MoB. 76 % of responders (13 / 17) answered that it was very or extremely likely that they would want use Audomni along with their current aid. This might be the first result in the field demonstrating a majority of blind and low-vision participants reporting that they actually want to use a new electronic travel aid. This shows promise that eventual long-term tests will demonstrate an increased mobility of blind and low-vision users — the overarching project aim. Such results would ultimately mean that Audomni can become an aid that alleviates societal cost, reduces burden on relatives, and improves users’ quality of life and independence