Accessible Autonomy: Exploring Inclusive Autonomous Vehicle Design and Interaction for People who are Blind and Visually Impaired

Autonomous vehicles are poised to revolutionize independent travel for millions of people experiencing transportation-limiting visual impairments worldwide. However, the current trajectory of automotive technology is rife with roadblocks to accessible interaction and inclusion for this demographic. Inaccessible (visually dependent) interfaces and lack of information access throughout the trip are surmountable, yet nevertheless critical barriers to this potentially lifechanging technology. To address these challenges, the programmatic dissertation research presented here includes ten studies, three published papers, and three submitted papers in high impact outlets that together address accessibility across the complete trip of transportation. The first paper began with a thorough review of the fully autonomous vehicle (FAV) and blind and visually impaired (BVI) literature, as well as the underlying policy landscape. Results guided prejourney ridesharing needs among BVI users, which were addressed in paper two via a survey with (n=90) transit service drivers, interviews with (n=12) BVI users, and prototype design evaluations with (n=6) users, all contributing to the Autonomous Vehicle Assistant: an award-winning and accessible ridesharing app. A subsequent study with (n=12) users, presented in paper three, focused on prejourney mapping to provide critical information access in future FAVs. Accessible in-vehicle interactions were explored in the fourth paper through a survey with (n=187) BVI users. Results prioritized nonvisual information about the trip and indicated the importance of situational awareness. This effort informed the design and evaluation of an ultrasonic haptic HMI intended to promote situational awareness with (n=14) participants (paper five), leading to a novel gestural-audio interface with (n=23) users (paper six). Strong support from users across these studies suggested positive outcomes in pursuit of actionable situational awareness and control. Cumulative results from this dissertation research program represent, to our knowledge, the single most comprehensive approach to FAV BVI accessibility to date. By considering both pre-journey and in-vehicle accessibility, results pave the way for autonomous driving experiences that enable meaningful interaction for BVI users across the complete trip of transportation. This new mode of accessible travel is predicted to transform independent travel for millions of people with visual impairment, leading to increased independence, mobility, and quality of life

Human-powered smartphone assistance for blind people

Mobile devices are fundamental tools for inclusion and independence. Yet, there are still many open research issues in smartphone accessibility for blind people (Grussenmeyer and Folmer 2017). Currently, learning how to use a smartphone is non-trivial, especially when we consider that the need to learn new apps and accommodate to updates never ceases. When first transitioning from a basic feature-phone, people have to adapt to new paradigms of interaction. Where feature phones had a finite set of applications and functions, users can extend the possible functions and uses of a smartphone by installing new 3rd party applications. Moreover, the interconnectivity of these applications means that users can explore a seemingly endless set of workflows across applications. To that end, the fragmented nature of development on these devices results in users needing to create different mental models for each application. These characteristics make smartphone adoption a demanding task, as we found from our eight-week longitudinal study on smartphone adoption by blind people. We conducted multiple studies to characterize the smartphone challenges that blind people face, and found people often require synchronous, co-located assistance from family, peers, friends, and even strangers to overcome the different barriers they face. However, help is not always available, especially when we consider the disparity in each barrier, individual support network and current location. In this dissertation we investigated if and how in-context human-powered solutions can be leveraged to improve current smartphone accessibility and ease of use. Building on a comprehensive knowledge of the smartphone challenges faced and coping mechanisms employed by blind people, we explored how human-powered assistive technologies can facilitate use. The thesis of this dissertation is: Human-powered smartphone assistance by non-experts is effective and impacts perceptions of self-efficacy

Augmenting the Spatial Perception Capabilities of Users Who Are Blind

People who are blind face a series of challenges and limitations resulting from their lack of being able to see, forcing them to either seek the assistance of a sighted individual or work around the challenge by way of a inefficient adaptation (e.g. following the walls in a room in order to reach a door rather than walking in a straight line to the door). These challenges are directly related to blind users' lack of the spatial perception capabilities normally provided by the human vision system. In order to overcome these spatial perception related challenges, modern technologies can be used to convey spatial perception data through sensory substitution interfaces. This work is the culmination of several projects which address varying spatial perception problems for blind users. First we consider the development of non-visual natural user interfaces for interacting with large displays. This work explores the haptic interaction space in order to find useful and efficient haptic encodings for the spatial layout of items on large displays. Multiple interaction techniques are presented which build on prior research (Folmer et al. 2012), and the efficiency and usability of the most efficient of these encodings is evaluated with blind children. Next we evaluate the use of wearable technology in aiding navigation of blind individuals through large open spaces lacking tactile landmarks used during traditional white cane navigation. We explore the design of a computer vision application with an unobtrusive aural interface to minimize veering of the user while crossing a large open space. Together, these projects represent an exploration into the use of modern technology in augmenting the spatial perception capabilities of blind users

Accessible On-Body Interaction for People With Visual Impairments

While mobile devices offer new opportunities to gain independence in everyday activities for people with disabilities, modern touchscreen-based interfaces can present accessibility challenges for low vision and blind users. Even with state-of-the-art screenreaders, it can be difficult or time-consuming to select specific items without visual feedback. The smooth surface of the touchscreen provides little tactile feedback compared to physical button-based phones. Furthermore, in a mobile context, hand-held devices present additional accessibility issues when both of the users’ hands are not available for interaction (e.g., on hand may be holding a cane or a dog leash). To improve mobile accessibility for people with visual impairments, I investigate on-body interaction, which employs the user’s own skin surface as the input space. On-body interaction may offer an alternative or complementary means of mobile interaction for people with visual impairments by enabling non-visual interaction with extra tactile and proprioceptive feedback compared to a touchscreen. In addition, on-body input may free users’ hands and offer efficient interaction as it can eliminate the need to pull out or hold the device. Despite this potential, little work has investigated the accessibility of on-body interaction for people with visual impairments. Thus, I begin by identifying needs and preferences of accessible on-body interaction. From there, I evaluate user performance in target acquisition and shape drawing tasks on the hand compared to on a touchscreen. Building on these studies, I focus on the design, implementation, and evaluation of an accessible on-body interaction system for visually impaired users. The contributions of this dissertation are: (1) identification of perceived advantages and limitations of on-body input compared to a touchscreen phone, (2) empirical evidence of the performance benefits of on-body input over touchscreen input in terms of speed and accuracy, (3) implementation and evaluation of an on-body gesture recognizer using finger- and wrist-mounted sensors, and (4) design implications for accessible non-visual on-body interaction for people with visual impairments

Pedestrian Detection with Wearable Cameras for the Blind: A Two-way Perspective

Blind people have limited access to information about their surroundings, which is important for ensuring one's safety, managing social interactions, and identifying approaching pedestrians. With advances in computer vision, wearable cameras can provide equitable access to such information. However, the always-on nature of these assistive technologies poses privacy concerns for parties that may get recorded. We explore this tension from both perspectives, those of sighted passersby and blind users, taking into account camera visibility, in-person versus remote experience, and extracted visual information. We conduct two studies: an online survey with MTurkers (N=206) and an in-person experience study between pairs of blind (N=10) and sighted (N=40) participants, where blind participants wear a working prototype for pedestrian detection and pass by sighted participants. Our results suggest that both of the perspectives of users and bystanders and the several factors mentioned above need to be carefully considered to mitigate potential social tensions.Comment: The 2020 ACM CHI Conference on Human Factors in Computing Systems (CHI 2020

The Graphical Access Challenge for People with Visual Impairments: Positions and Pathways Forward

Graphical access is one of the most pressing challenges for individuals who are blind or visually impaired. This chapter discusses some of the factors underlying the graphics access challenge, reviews prior approaches to addressing this long-standing information access barrier, and describes some promising new solutions. We specifically focus on touchscreen-based smart devices, a relatively new class of information access technologies, which our group believes represent an exemplary model of user-centered, needs-based design. We highlight both the challenges and the vast potential of these technologies for alleviating the graphics accessibility gap and share the latest results in this line of research. We close with recommendations on ideological shifts in mindset about how we approach solving this vexing access problem, which will complement both technological and perceptual advancements that are rapidly being uncovered through a growing research community in this domain

Designing wearable interfaces for blind people

Tese de mestrado, Engenharia Informática (Arquitectura, Sistemas e Redes de Computadores), Universidade de Lisboa, faculdade de Ciências, 2015Hoje em dia os dispositivos com ecrã táctil, estão cada vez mais onipresentes. Até recentemente, a maioria dos ecrãs sensíveis ao toque forneciam poucos recursos de acessibilidade para deficientes visuais, deixando-os inutilizáveis. Sendo uma tecnologia tão presente no nosso quotidiano, como em telemóveis e tablets. Estes dispositivos são cada vez mais essenciais para a nossa vida, uma vez que, guardam muita informação pessoal, por exemplo, o pagamento através carteiras eletrónicas. A falta de acessibilidade deste tipo de ecrãs devem-se ao facto de estas interfaces serem baseadas no que os utilizadores veem no ecrã e em tocar no conteúdo apresentado neste. Isso torna-se num grande problema quando uma pessoa deficiente visual tenta usar estas interfaces. No mercado existem algumas soluções mas são quase todas baseadas em retorno áudio. Esta solução não é a melhor quando se trata de informação pessoal que a pessoa deseja manter privada. Por exemplo quando um utilizador está num autocarro e recebe uma mensagem, esta é lida por um leitor de ecrã através das colunas do dispositivo. Esta solução é prejudicial para a privacidade do utilizador, pois todas a pessoas `a sua volta irão ouvir o conteúdo da mensagem. Uma solução para este problema, poderá ser a utilização de vibração e de teclas físicas, que retiram a necessidade da utilização de leitores de ecrã. Contudo, para a navegação em menus a problemática mantém-se. Uma maneira de resolver este problema é através da utilização de uma interface baseada em gestos. Este tipo de interface é uma forma flexível e intuitiva de interação com este dispositivos. Até hoje, muitas abordagens têm vindo a apresentar soluções, no entanto não resolvem todos os pontos referidos. De uma maneira ou de outra estas abordagens terão de ser complementadas com outros dispositivos. Guerreiro e colegas (2012), apresentaram um protótipo que possibilita a leitura texto através de vibração, mas todo o impacto de uma utilização no dia a dia não é tido em conta. Um outro estudo realizado por Myung-Chul Cho (2002) apresenta um par de luvas para escrita codificada pelo alfabeto Braile, contudo não é testado para uma utilização com integração de uma componente de leitura, sem ser o retorno áudio. Dois outros estudos destacam-se, relativamente à utilização de gestos para navegação no dispositivo. Ruiz (2011), efetuou uma elicitação de gestos no ar, no entanto, eles não incluem pessoas invisuais no estudo, o que poderá levar à exclusão de tais utilizadores. Outro estudo apresentado por Kane (2011), inclui pessoas invisuais e destina-se a interações com gestos mas exigindo contacto físico com os ecrãs tácteis. A abordagem apresentada neste estudo integra as melhores soluções apresentadas num único dispositivo. O nosso objectivo principal é tornar os dispositivos de telemóveis mais acessíveis a pessoas invisuais, de forma serem integrados no seu quotidiano. Para isso, desenvolvemos uma interface baseada num par de luvas. O utilizador pode usá-las e com elas ler e escrever mensagens e ainda fazer gestos para outras tarefas. Este par de luvas aproveita o conhecimento sobre Braille por parte dos utilizadores para ler e escrever informação textual. Para a característica de leitura instalámos seis motores de vibração nos dedos da luva, no dedo indicador, no dedo do meio e no dedo anelar, de ambas as mãos. Estes motores simulam a configuração das teclas de uma máquina de escrever Braille, por exemplo, a Perkins Brailler. Para a parte de escrita, instalámos botões de pressão na ponta destes mesmos dedos, sendo cada um representante de um ponto de uma célula de Braille. Para a detecção de gestos optámos por uma abordagem através de um acelerómetro. Este encontra-se colocado nas costas da mão da luva. Para uma melhor utilização a luva é composta por duas camadas, e desta forma é possível instalar todos os componente entre as duas camadas de tecido, permitindo ao utilizador calçar e descalçar as luvas sem se ter que preocupar com os componentes eletrónicos. A construção das luvas assim como todos os testes realizados tiveram a participação de um grupo de pessoas invisuais, alunos e professores, da Fundação Raquel e Martin Sain. Para avaliarmos o desempenho do nosso dispositivo por invisuais realizámos alguns teste de recepcão (leitura) e de envio de mensagens (escrita). No teste de leitura foi realizado com um grupo apenas de pessoas invisuais. O teste consistiu em, receber letras em Braille, onde o utilizador replicava as vibrações sentidas, com os botões das luvas. Para isso avaliámos as taxas de reconhecimento de caracteres. Obtivemos uma média de 31 %, embora estes resultados sejam altamente dependentes das habilidades dos utilizadores. No teste de escrita, foi pedido uma letra ao utilizador e este escrevia em braille utilizando as luvas. O desempenho nesta componente foi em média 74 % de taxa de precisão. A maioria dos erros durante este teste estão ligados a erros, onde a diferença entre a palavra inicial e a escrita pelo utilizador, é de apenas um dedo. Estes testes foram bastante reveladores, relativamente à possível utilização destas luvas por pessoas invisuais. Indicaram-nos que os utilizadores devem ser treinados previamente para serem maximizados os resultados, e que pode ser necessário um pouco de experiencia com o dispositivo. O reconhecimento de gestos permite ao utilizador executar várias tarefas com um smartphone, tais como, atender/rejeitar uma chamada e navegar em menus. Para avaliar que gestos os utilizadores invisuais e normovisuais sugerem para a execução de tarefas em smartphones, realizámos um estudo de elicitação. Este estudo consiste em pedir aos utilizadores que sugiram gestos para a realização de tarefas. Descobrimos que a maioria dos gestos inventados pelos participantes tendem a ser físicos, em contexto, discreto e simples, e que utilizam apenas um ´unico eixo espacial. Concluímos também que existe um consenso, entre utilizadores, para todas as tarefas propostas. Além disso, o estudo de elicitação revelou que as pessoas invisuais preferem gestos mais simples, opondo-se a uma preferência por gestos mais complexos por parte de pessoas normovisuais. Sendo este um dispositivo que necessita de treino para reconhecimento de gestos, procurámos saber qual o tipo de treino é mais indicado para a sua utilização. Com os resultados obtidos no estudo de elicitação, comparámos treinos dos utilizadores individuais, treinos entre as das populações (invisuais e normovisuais) e um treino com ambas as populações (global). Descobrimos que um treino personalizado, ou seja, feito pelo próprio utilizador, é muito mais eficaz que um treino da população e um treino global. O facto de o utilizador poder enviar e receber mensagens, sem estar dependente de vários dispositivos e/ou aplicações contorna, as tão levantadas, questões de privacidade. Com o mesmo dispositivo o utilizador pode, ainda, navegar nos menus do seu smartphone, através de gestos simples e intuitivos. Os nossos resultados sugerem que será possível a utilização de um dispositivo wearable, no seio da comunidade invisual. Com o crescimento exponencial do mercado wearable e o esforço que a comunidade académica está a colocar nas tecnologias de acessibilidade, ainda existe uma grande margem para melhorar. Com este projeto, espera-se que os dispositivos portáteis de apoio irão desempenhar um papel importante na integração social das pessoas com deficiência, criando com isto uma sociedade mais igualitária e justa.Nowadays touch screens are ubiquitous, present in almost all modern devices. Most touch screens provide few accessibility features for blind people, leaving them partly unusable. There are some solutions, based on audio feedback, that help blind people to use touch screens in their daily tasks. The problem with those solutions raises privacy issues, since the content on screen is transmitted through the device speakers. Also, these screen readers make the interaction slow, and they are not easy to use. The main goal of this project is to develop a new wearable interface that allows blind people to interact with smartphones. We developed a pair of gloves that is capable to recognise mid-air gestures, and also allows the input and output of text. To evaluate the usability of input and output, we conducted a user study to assess character recognition and writing performance. Character recognition rates were highly user-dependent, and writing performance showed some problems, mostly related to one-finger issues. Then, we conducted an elicitation study to assess what type of gestures blind and sighted people suggest. Sighted people suggested more complex gestures, compared with blind people. However, all the gestures tend to be physical, in-context, discrete and simple, and use only a single axis. We also found that a training based on the user’s gestures is better for recognition accuracy. Nevertheless, the input and output text components still require new approaches to improve users performance. Still, this wearable interface seems promising for simple actions that do not require cognitive load. Overall, our results suggest that we are on track to make possible blind people interact with mobile devices in daily life

Making Spatial Information Accessible on Touchscreens for Users who are Blind and Visually Impaired

Touchscreens have become a de facto standard of input for mobile devices as they most optimally use the limited input and output space that is imposed by their form factor. In recent years, people who are blind and visually impaired have been increasing their usage of smartphones and touchscreens. Although basic access is available, there are still many accessibility issues left to deal with in order to bring full inclusion to this population. One of the important challenges lies in accessing and creating of spatial information on touchscreens. The work presented here provides three new techniques, using three different modalities, for accessing spatial information on touchscreens. The first system makes geometry and diagram creation accessible on a touchscreen through the use of text-to-speech and gestural input. This first study is informed by a qualitative study of how people who are blind and visually impaired currently access and create graphs and diagrams. The second system makes directions through maps accessible using multiple vibration sensors without any sound or visual output. The third system investigates the use of binaural sound on a touchscreen to make various types of applications accessible such as physics simulations, astronomy, and video games

Do we enjoy what we sense and perceive?:A dissociation between aesthetic appreciation and basic perception of environmental objects or events

This integrative review rearticulates the notion of human aesthetics by critically appraising the conventional definitions, offerring a new, more comprehensive definition, and identifying the fundamental components associated with it. It intends to advance holistic understanding of the notion by differentiating aesthetic perception from basic perceptual recognition, and by characterizing these concepts from the perspective of information processing in both visual and nonvisual modalities. To this end, we analyze the dissociative nature of information processing in the brain, introducing a novel local-global integrative model that differentiates aesthetic processing from basic perceptual processing. This model builds on the current state of the art in visual aesthetics as well as newer propositions about nonvisual aesthetics. This model comprises two analytic channels: aesthetics-only channel and perception-to-aesthetics channel. The aesthetics-only channel primarily involves restricted local processing for quality or richness (e.g., attractiveness, beauty/prettiness, elegance, sublimeness, catchiness, hedonic value) analysis, whereas the perception-to-aesthetics channel involves global/extended local processing for basic feature analysis, followed by restricted local processing for quality or richness analysis. We contend that aesthetic processing operates independently of basic perceptual processing, but not independently of cognitive processing. We further conjecture that there might be a common faculty, labeled as aesthetic cognition faculty, in the human brain for all sensory aesthetics albeit other parts of the brain can also be activated because of basic sensory processing prior to aesthetic processing, particularly during the operation of the second channel. This generalized model can account not only for simple and pure aesthetic experiences but for partial and complex aesthetic experiences as well.</p

Self-Confidence Levels in Sequential Learning Versus Structured Discovery Cane Travel, Post Orientation and Mobility Instruction: A Comparison Study

Sequential Learning (SL), the medical model of Orientation and Mobility (O&M)was designed for blinded WWII veterans in the 1940s. This preeminent curriculum monopolized the O&M profession, creating a paradigm paralysis, until Structured Discovery Cane Travel (SDCT) made its official debut in 1997. The conceptual framework for this study is Glasser\u27s choice theory (1998) whereby ideas or systems of belief direct or oversee behavior, and this principle holds true for both O&M professionals and individuals who are blind or visually impaired (consumers). A comparison study answered the research question; that is, at what distance and frequency do consumers travel independently post-instruction and how does this differ between the two curriculums? Data was collected through a quantitative study in which 40 participants (20 SL, 20 SDCT) voluntarily responded to an electronic survey. Because of their increased frequency and distances traveled and their decreased need for additional training, study results revealed SDCT consumers’ self-confidence is higher than SL consumers by 32%. In addition, this study discovered when sighted guide instruction commences prior to introduction of the long, white cane (as in the SL curriculum); self-confidence is hindered and leads consumers toward the Custodial Paradigm. However, when instruction of the long, white cane and problem-solving is paramount (as in the SDCT curriculum); the foundation for ongoing successful O&M post-instruction is likely whereby consumers are lead toward the Independence Paradigm