A Systematic Review of Extended Reality (XR) for Understanding and Augmenting Vision Loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on augmentation of a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the last decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the suitability and usability of different XR-based accessibility aids. By broadening end-user participation to early stages of the design process and shifting the focus from behavioral performance to qualitative assessments of usability, future research has the potential to develop XR technologies that may not only allow for studying vision loss, but also enable novel visual accessibility aids with the potential to impact the lives of millions of people living with vision loss

Pedestrian Detection with Wearable Cameras for the Blind: A Two-way Perspective

Blind people have limited access to information about their surroundings, which is important for ensuring one's safety, managing social interactions, and identifying approaching pedestrians. With advances in computer vision, wearable cameras can provide equitable access to such information. However, the always-on nature of these assistive technologies poses privacy concerns for parties that may get recorded. We explore this tension from both perspectives, those of sighted passersby and blind users, taking into account camera visibility, in-person versus remote experience, and extracted visual information. We conduct two studies: an online survey with MTurkers (N=206) and an in-person experience study between pairs of blind (N=10) and sighted (N=40) participants, where blind participants wear a working prototype for pedestrian detection and pass by sighted participants. Our results suggest that both of the perspectives of users and bystanders and the several factors mentioned above need to be carefully considered to mitigate potential social tensions.Comment: The 2020 ACM CHI Conference on Human Factors in Computing Systems (CHI 2020

Auxilio: A Sensor-Based Wireless Head-Mounted Mouse for People with Upper Limb Disability

Upper limb disability may be caused either due to accidents, neurological disorders, or even birth defects, imposing limitations and restrictions on the interaction with a computer for the concerned individuals using a generic optical mouse. Our work proposes the design and development of a working prototype of a sensor-based wireless head-mounted Assistive Mouse Controller (AMC), Auxilio, facilitating interaction with a computer for people with upper limb disability. Combining commercially available, low-cost motion and infrared sensors, Auxilio solely utilizes head and cheek movements for mouse control. Its performance has been juxtaposed with that of a generic optical mouse in different pointing tasks as well as in typing tasks, using a virtual keyboard. Furthermore, our work also analyzes the usability of Auxilio, featuring the System Usability Scale. The results of different experiments reveal the practicality and effectiveness of Auxilio as a head-mounted AMC for empowering the upper limb disabled community.Comment: 28 pages, 9 figures, 5 table

Computational Personalization through Physical and Aesthetic Featured Digital Fabrication

Thesis (Master of Science in Informatics)--University of Tsukuba, no. 41269, 2019.3.2

Towards Accessible News Reading Design in Virtual Reality for Low Vision

International audienceLow-vision conditions resulting in partial loss of the central visual field strongly affect patients' daily tasks and routines, and none more prominently than the ability to access text. Though vision aids such as magnifiers, digital screens, and text-to-speech devices can improve overall accessibility to text, news media, which is non-linear and has complex and volatile formatting, is still inaccessible, barring low-vision patients from easy access to essential news content. This position paper proposes virtual reality as a promising solution towards accessible and enjoyable news reading for low vision. We first provide an extensive review into existing research on low-vision reading technologies and visual accessibility solutions for modern news media. From previous research and studies, we then conduct an analysis into the advantages of virtual reality for low-vision reading and propose comprehensive guidelines for visual accessibility design in virtual reality, with a focus on reading. This is coupled with a hands-on survey of eight reading applications in virtual reality to evaluate how accessibility design is currently implemented in existing products. Finally, we present an open toolbox using browser-based graphics (WebGL) that implements the design principles from our study. A proof-of-concept is created using this toolbox to demonstrate the feasibility of our proposal with modern virtual reality technology

Vers une conception accessible de la lecture d'actualités en réalité virtuelle pour la basse vision

Low-vision conditions resulting in partial loss of the visual field strongly affect patients' daily tasks and routines, and none more prominently than the ability to access text. Though vision aids such as magnifiers, digital screens, and text-to-speech devices can improve overall accessibility to text, news media, which is non-linear and has complex and volatile formatting, is still inaccessible, barring low-vision patients from easy access to essential news content. This paper positions virtual reality as the next step towards accessible and enjoyable news reading for the low vision. We first conduct an extensive review into existing research on low-vision reading technologies and accessibility for modern news media. From previous research and studies, we then conduct an analysis into the advantages of virtual reality for low-vision reading and propose comprehensive guidelines for visual accessibility design in virtual reality, with a focus on reading. This is coupled with a hands-on study of eight reading applications in virtual reality to evaluate how accessibility design is currently implemented in existing products. Finally, we present a framework that integrates the design principles resulting from our analysis and study, and implement a proof-of-concept for this framework using browser-based graphics to demonstrate the feasibility of our proposal with modern virtual reality technology.Les conditions de basse vision entraînant une perte partielle du champ visuel affectent fortement les tâches et les routines quotidiennes des patients, et aucune de celles-ci n'est plus notable que la possibilité d'accéder à du texte. Bien que les aides visuelles telles que les loupes, les écrans numériques et les dispositifs de synthèse vocale puissent améliorer l’accessibilité globale au texte, les médias d’information, qui sont non linéaires et présentent un formatage complexe et volatile, sont toujours inaccessibles, empêchant ainsi les patients malvoyants d’avoir accès aux contenus des nouvelles.Ce papier positionne la réalité virtuelle comme la prochaine étape vers une lecture de nouvelles pour la basse vision, qui soit accessible et agréable. Nous menons d’abord une étude approfondie des recherches existantes sur les technologies de lecture en basse vision et l’accessibilité des médias d’information modernes. À partir de recherches et d’études antérieures, nous analysons ensuite les avantages de la réalité virtuelle pour la lecture en basse vision et proposons des directives globales pour la conception de l’accessibilité visuelle en réalité virtuelle, en mettant l’accent sur la lecture. Ceci est associé à une étude pratique de huit applications de lecture en réalité virtuelle pour évaluer comment cette conception de l'accessibilité est actuellement mise en œuvre dans les produits existants. Enfin, nous présentons un cadre qui intègre les principes de conception résultant de nos analyses et études et nous implémentons une preuve de concept en utilisant des graphiques basés sur un navigateur afin de démontrer la faisabilité de notre proposition avec la technologie moderne de réalité virtuelle

Head-mounted displays and dynamic text presentation to aid reading in macular disease

The majority of individuals living with significant sight loss have residual vision which can be enhanced using low vision aids. Smart glasses and smartphone-based headsets, both increasing in prevalence, are proposed as a low vision aid platform. Three novel tests for measuring the visibility of displays to partially sighted users are described, along with a questionnaire for assessing subjective preference. Most individuals tested, save those with the weakest vision, were able to see and read from both a smart glasses screen and a smartphone screen mounted in a headset. The scheme for biomimetic scrolling, a text presentation strategy which translates natural eye movement into text movement, is described. It is found to enable the normally sighted to read at a rate five times that of continuous scrolling and is faster than rapid serial visual presentation for individuals with macular disease. With text presentation on the smart glasses optimised to the user, individuals with macular disease read on average 65% faster than when using their habitual optical aid. It is concluded that this aid demonstrates clear benefit over the commonly used devices and is thus recommended for further development towards widespread availability

AdaptiX -- A Transitional XR Framework for Development and Evaluation of Shared Control Applications in Assistive Robotics

With the ongoing efforts to empower people with mobility impairments and the increase in technological acceptance by the general public, assistive technologies, such as collaborative robotic arms, are gaining popularity. Yet, their widespread success is limited by usability issues, specifically the disparity between user input and software control along the autonomy continuum. To address this, shared control concepts provide opportunities to combine the targeted increase of user autonomy with a certain level of computer assistance. This paper presents the free and open-source AdaptiX XR framework for developing and evaluating shared control applications in a high-resolution simulation environment. The initial framework consists of a simulated robotic arm with an example scenario in Virtual Reality (VR), multiple standard control interfaces, and a specialized recording/replay system. AdaptiX can easily be extended for specific research needs, allowing Human-Robot Interaction (HRI) researchers to rapidly design and test novel interaction methods, intervention strategies, and multi-modal feedback techniques, without requiring an actual physical robotic arm during the early phases of ideation, prototyping, and evaluation. Also, a Robot Operating System (ROS) integration enables the controlling of a real robotic arm in a PhysicalTwin approach without any simulation-reality gap. Here, we review the capabilities and limitations of AdaptiX in detail and present three bodies of research based on the framework. AdaptiX can be accessed at https://adaptix.robot-research.de.Comment: Accepted submission at The 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS'24

HABITAT: An IoT Solution for Independent Elderly

In this work, a flexible and extensive digital platform for Smart Homes is presented, exploiting the most advanced technologies of the Internet of Things, such as Radio Frequency Identification, wearable electronics, Wireless Sensor Networks, and Artificial Intelligence. Thus, the main novelty of the paper is the system-level description of the platform flexibility allowing the interoperability of different smart devices. This research was developed within the framework of the operative project HABITAT (Home Assistance Based on the Internet of Things for the Autonomy of Everybody), aiming at developing smart devices to support elderly people both in their own houses and in retirement homes, and embedding them in everyday life objects, thus reducing the expenses for healthcare due to the lower need for personal assistance, and providing a better life quality to the elderly users