Pedestrian Detection with Wearable Cameras for the Blind: A Two-way Perspective

Blind people have limited access to information about their surroundings, which is important for ensuring one's safety, managing social interactions, and identifying approaching pedestrians. With advances in computer vision, wearable cameras can provide equitable access to such information. However, the always-on nature of these assistive technologies poses privacy concerns for parties that may get recorded. We explore this tension from both perspectives, those of sighted passersby and blind users, taking into account camera visibility, in-person versus remote experience, and extracted visual information. We conduct two studies: an online survey with MTurkers (N=206) and an in-person experience study between pairs of blind (N=10) and sighted (N=40) participants, where blind participants wear a working prototype for pedestrian detection and pass by sighted participants. Our results suggest that both of the perspectives of users and bystanders and the several factors mentioned above need to be carefully considered to mitigate potential social tensions.Comment: The 2020 ACM CHI Conference on Human Factors in Computing Systems (CHI 2020

Towards Making Videos Accessible for Low Vision Screen Magnifier Users

People with low vision who use screen magnifiers to interact with computing devices find it very challenging to interact with dynamically changing digital content such as videos, since they do not have the luxury of time to manually move, i.e., pan the magnifier lens to different regions of interest (ROIs) or zoom into these ROIs before the content changes across frames. In this paper, we present SViM, a first of its kind screen-magnifier interface for such users that leverages advances in computer vision, particularly video saliency models, to identify salient ROIs in videos. SViM\u27s interface allows users to zoom in/out of any point of interest, switch between ROIs via mouse clicks and provides assistive panning with the added flexibility that lets the user explore other regions of the video besides the ROIs identified by SViM. Subjective and objective evaluation of a user study with 13 low vision screen magnifier users revealed that overall the participants had a better user experience with SViM over extant screen magnifiers, indicative of the former\u27s promise and potential for making videos accessible to low vision screen magnifier users

On the conception and mechanical design of customized glasses for augmented reality

Wearable devices such as smart glasses are one of the main challenges of augmented reality (AR) adoption in the consumer market. Despite the significant advances in recent years, the hardware available presents a bulky form factor and does not bring significant value to everyday life. LusoVU is focused on developing new AR smart glasses accessible to the general population with their HIDO technology. This dissertation combines the areas of product devel-opment and augmented reality for the development of a new prototype for LusoVU. The aim is to integrate the electrical and optical elements of the HIDO technology into a frame design that aligns with society’s standards of an acceptable everyday eyewear device, whilst consider-ing all the constraints and tradeoffs of the optical technology. The research hereafter presented followed the steps of a product development, namely: internal and external benchmarking, identification of client needs; product specifications; concept generation and selection; mod-elling and prototyping. It presents and describes the many iterations to achieve the final de-sign. The final design meets the product specifications and customer needs and will now be submitted in the coming months to the biometric analysis.Aparelhos portáteis para uso quotidiano, tais como óculos inteligentes, são um dos principais desafios da adoção da realidade aumentada no mercado do consumidor. Apesar dos grandes avanços nos últimos anos, o equipamento disponível apresenta um formato volumoso e ainda não acrescenta valor significativo ao dia a dia. A LusoVU está focada no desenvolvimento de novos óculos inteligentes, acessíveis à população em geral, com a sua tecnologia HIDO. Esta tese combina as áreas de desenvolvimento de produto e de realidade aumentada para o desenvolvimento de um novo protótipo para LusoVU. O objetivo é a conceção e o projeto de armações de óculos, alinhado com os padrões estéticos da sociedade, integrando os elementos óticos e eletrónicos da tecnologia HIDO e considerando todas as suas restrições. O presente trabalho segue as etapas do desenvolvimento do produto: benchmarking interno e externo, identificação das necessidades do cliente; especificações do produto; geração e seleção de conceitos; modelação e prototipagem. A dissertação apresenta e descreve as várias iterações necessárias para alcançar o projeto final. O projeto final atende às especificações do produto e às necessidades do cliente, e será agora submetido nos próximos meses à análise biométrica

MRnews: Design Explorations into Accessibility and News

Creating accessible technology and content is generally seen as beneficial for all users. This is particularly important when the content has a significant societal impact, such as news stories. To find new and innovative ways to engage users, digital news outlets are faced with challenges related to accessibility. In the case of Mixed Reality (MR) technology, the increasing interest emphasizes the need for the technology to be inclusive and accessible. The embodied nature and affordances of MR technology enable users to manipulate virtual objects using real-world knowledge and in real-time and enable them to utilize a wide range of skills when interacting with such systems. In turn, leveraging these affordances can enhance the accessibility of the task at hand. Contributions to developing accessibility guidelines have been made, and the use of MR applications to enhance accessibility is on the rise. However, these contributions are most prominent in education and not for leisurely use. This research project investigates the affordances of MR and of the Augmented Reality (AR) Head Mounted Display (HMD), HoloLens 2 (HL2) in particular, and how these can be leveraged to enhance accessibility when reading digital news. This is a Research through Design (RtD) project carried out in participation with users by conducting design activities and user evaluations. The RtD-process is supported by prototypes developed through an iterative process. MRnews is an application built for Microsoft’s AR HMD, the HL2. The implemented design showcases how news content creators and developers can leverage the affordances of MR technology to achieve accessibility in news stories. The results point toward direct manipulation of virtual content utilizing the spatial nature of MR technology and the use of sensory cues to keep the user oriented and focused impact accessibility.Masteroppgave i medie- og interaksjonsdesignMIX350MASV-MI

Aplicación dos produtos de apoio de alta tecnoloxía no desempeño ocupacional das persoas con discapacidade visual: scoping review

[Resumo] Introdución: As persoas con discapacidade visual poden experimentar dificultades na realización de diferentes ocupacións cotiás. A tecnoloxía de apoio (TA) debe actuar coma un facilitador na participación desta poboación en actividades desexadas. Obxectivo: Coñecer se os produtos de apoio de alta tecnoloxía dan resposta ás dificultades no desempeño ocupacional coas que se atopan máis frecuentemente as persoas con discapacidade visual no seu día a día. Metodoloxía: Seguiuse unha metodoloxía de revisión de alcance. A busca bibliográfica foi levada a cabo nas bases de datos CINAHL, PubMed, Scopus e Web of Science. As variables temáticas establecéronse seguindo os compoñentes propostos polo Modelo da Actividade Humana e da Tecnoloxía de Apoio. Para a súa análise empregouse un enfoque mixto. Resultados: Na mostra incluíronse un total de 44 estudos. En moitos casos, non se especificou certa información relativa ás persoas participantes. A TA analizada centrouse principalmente na detección e na evitación de obstáculos, facilitando a mobilidade funcional. Os dispositivos reciben a información mediante sistemas de cámaras para transmitirlla á persoa por son. A TA está deseñada para contextos tanto interiores coma exteriores da vida diaria, e foi testada maiormente en ambientes reais. Conclusión: Os produtos de apoio de alta tecnoloxía están dirixidos principalmente a facilitar a mobilidade funcional, a participación social, e a lectura das persoas con discapacidade visual. En menor medida, abordaron a educación, o deporte e as tarefas do fogar. Así, non se centraron noutras actividades relevantes como a condución ou o emprego.[Resumen] Introducción: Las personas con discapacidad visual pueden experimentan dificultades en la realización de diferentes ocupaciones cotidianas. La tecnología de apoyo (TA) debe actuar como un facilitador en la participación de esta población en actividades deseadas. Objetivo: Conocer si los productos de apoyo de alta tecnología dan respuesta a las dificultades en el desempeño ocupacional con las que se encuentran más frecuentemente las personas con discapacidad visual en su día a día. Metodología: Se siguió una metodología de revisión de alcance. La búsqueda bibliográfica se llevó a cabo en las bases de datos CINAHL, PubMed, Scopus y Web of Science. Las variables temáticas se establecieron siguiendo los componentes propuestos por el Modelo de la Actividad Humana y la Tecnología de Apoyo. Para su análisis se empleó un enfoque mixto. Resultados: En la muestra se incluyeron un total de 44 estudios. En muchos casos, no se especificó cierta información relativa a las personas participantes. La TA analizada se centró principalmente en la detección y evitación de obstáculos, facilitando la movilidad funcional. Los dispositivos reciben la información mediante sistemas de cámaras para transmitírsela a la persona por audio. La TA está diseñada para contextos tanto interiores como exteriores de la vida diaria, y fue testada mayormente en ambientes reales. Conclusión: Los productos de apoyo de alta tecnología están dirigidos principalmente a facilitar la movilidad funcional, la participación social, y la lectura de las personas con discapacidad visual. En menor medida, abordaron la educación, el deporte, o las tareas del hogar. Así, no se centraron en otras actividades relevantes como la conducción o el empleo.[Abstract] Background: People with visual impairment can have difficulties in performing different daily occupations. Assistive technology (AT) should be an enabler in the participation of these population’s desired activities. Aim: To know if high-tech assistive devices respond to difficulties in the occupational performance that people with visual impairment most frequently experiment in their day-to-day lives. Methodology: A methodology of Scoping Review was used. The bibliographic search was carried out in the databases CINAHL, PubMed, Scopus y Web of Science. Thematic variables were established following the components proposed by the Human Activity Assistive Technology Model. For their analyses, a mixed methods approach was used. Results: A total of 44 studies were included in the sample. In many cases, certain information regarding the participants was not specified. The analyzed AT focused mainly on the detection and avoidance of obstacles, facilitating functional mobility. The devices receive the information through camera systems to transmit it to the person by audio. The AT is designed for indoor and outdoor contexts of everyday life, and it was mostly tested in real-world environments. Conclusion: The high-tech assistive devices are primarily aimed at facilitating the functional mobility, social participation, and reading of people with visual impairment. To a lesser extent, they addressed education, sports, or housework. However, they did not focus on other relevant activities such as driving or working.Traballo fin de mestrado (UDC.FCS). Asistencia e investigación sanitaria. Especialidade en Reeducación Funcional, Autonomía Persoal e Calidade de Vida. Curso 2020-202

HandSight: A Touch-Based Wearable System to Increase Information Accessibility for People with Visual Impairments

Many activities of daily living such as getting dressed, preparing food, wayfinding, or shopping rely heavily on visual information, and the inability to access that information can negatively impact the quality of life for people with vision impairments. While numerous researchers have explored solutions for assisting with visual tasks that can be performed at a distance, such as identifying landmarks for navigation or recognizing people and objects, few have attempted to provide access to nearby visual information through touch. Touch is a highly attuned means of acquiring tactile and spatial information, especially for people with vision impairments. By supporting touch-based access to information, we may help users to better understand how a surface appears (e.g., document layout, clothing patterns), thereby improving the quality of life. To address this gap in research, this dissertation explores methods to augment a visually impaired user’s sense of touch with interactive, real-time computer vision to access information about the physical world. These explorations span three application areas: reading and exploring printed documents, controlling mobile devices, and identifying colors and visual textures. At the core of each application is a system called HandSight that uses wearable cameras and other sensors to detect touch events and identify surface content beneath the user’s finger. To create HandSight, we designed and implemented the physical hardware, developed signal processing and computer vision algorithms, and designed real-time feedback that enables users to interpret visual or digital content. We involve visually impaired users throughout the design and development process, conducting several user studies to assess usability and robustness and to improve our prototype designs. The contributions of this dissertation include: (i) developing and iteratively refining HandSight, a novel wearable system to assist visually impaired users in their daily lives; (ii) evaluating HandSight across a diverse set of tasks, and identifying tradeoffs of a finger-worn approach in terms of physical design, algorithmic complexity and robustness, and usability; and (iii) identifying broader design implications for future wearable systems and for the fields of accessibility, computer vision, augmented and virtual reality, and human-computer interaction