Touch- and Walkable Virtual Reality to Support Blind and Visually Impaired Peoples‘ Building Exploration in the Context of Orientation and Mobility

Der Zugang zu digitalen Inhalten und Informationen wird immer wichtiger für eine erfolgreiche Teilnahme an der heutigen, zunehmend digitalisierten Zivilgesellschaft. Solche Informationen werden meist visuell präsentiert, was den Zugang für blinde und sehbehinderte Menschen einschränkt. Die grundlegendste Barriere ist oft die elementare Orientierung und Mobilität (und folglich die soziale Mobilität), einschließlich der Erlangung von Kenntnissen über unbekannte Gebäude vor deren Besuch. Um solche Barrieren zu überbrücken, sollten technische Hilfsmittel entwickelt und eingesetzt werden. Es ist ein Kompromiss zwischen technologisch niedrigschwellig zugänglichen und verbreitbaren Hilfsmitteln und interaktiv-adaptiven, aber komplexen Systemen erforderlich. Die Anpassung der Technologie der virtuellen Realität (VR) umfasst ein breites Spektrum an Entwicklungs- und Entscheidungsoptionen. Die Hauptvorteile der VR-Technologie sind die erhöhte Interaktivität, die Aktualisierbarkeit und die Möglichkeit, virtuelle Räume und Modelle als Abbilder von realen Räumen zu erkunden, ohne dass reale Gefahren und die begrenzte Verfügbarkeit von sehenden Helfern auftreten. Virtuelle Objekte und Umgebungen haben jedoch keine physische Beschaffenheit. Ziel dieser Arbeit ist es daher zu erforschen, welche VR-Interaktionsformen sinnvoll sind (d.h. ein angemessenes Verbreitungspotenzial bieten), um virtuelle Repräsentationen realer Gebäude im Kontext von Orientierung und Mobilität berührbar oder begehbar zu machen. Obwohl es bereits inhaltlich und technisch disjunkte Entwicklungen und Evaluationen zur VR-Technologie gibt, fehlt es an empirischer Evidenz. Zusätzlich bietet diese Arbeit einen Überblick über die verschiedenen Interaktionen. Nach einer Betrachtung der menschlichen Physiologie, Hilfsmittel (z.B. taktile Karten) und technologischen Eigenschaften wird der aktuelle Stand der Technik von VR vorgestellt und die Anwendung für blinde und sehbehinderte Nutzer und der Weg dorthin durch die Einführung einer neuartigen Taxonomie diskutiert. Neben der Interaktion selbst werden Merkmale des Nutzers und des Geräts, der Anwendungskontext oder die nutzerzentrierte Entwicklung bzw. Evaluation als Klassifikatoren herangezogen. Begründet und motiviert werden die folgenden Kapitel durch explorative Ansätze, d.h. im Bereich 'small scale' (mit sogenannten Datenhandschuhen) und im Bereich 'large scale' (mit einer avatargesteuerten VR-Fortbewegung). Die folgenden Kapitel führen empirische Studien mit blinden und sehbehinderten Nutzern durch und geben einen formativen Einblick, wie virtuelle Objekte in Reichweite der Hände mit haptischem Feedback erfasst werden können und wie verschiedene Arten der VR-Fortbewegung zur Erkundung virtueller Umgebungen eingesetzt werden können. Daraus werden geräteunabhängige technologische Möglichkeiten und auch Herausforderungen für weitere Verbesserungen abgeleitet. Auf der Grundlage dieser Erkenntnisse kann sich die weitere Forschung auf Aspekte wie die spezifische Gestaltung interaktiver Elemente, zeitlich und räumlich kollaborative Anwendungsszenarien und die Evaluation eines gesamten Anwendungsworkflows (d.h. Scannen der realen Umgebung und virtuelle Erkundung zu Trainingszwecken sowie die Gestaltung der gesamten Anwendung in einer langfristig barrierefreien Weise) konzentrieren.Access to digital content and information is becoming increasingly important for successful participation in today's increasingly digitized civil society. Such information is mostly presented visually, which restricts access for blind and visually impaired people. The most fundamental barrier is often basic orientation and mobility (and consequently, social mobility), including gaining knowledge about unknown buildings before visiting them. To bridge such barriers, technological aids should be developed and deployed. A trade-off is needed between technologically low-threshold accessible and disseminable aids and interactive-adaptive but complex systems. The adaptation of virtual reality (VR) technology spans a wide range of development and decision options. The main benefits of VR technology are increased interactivity, updatability, and the possibility to explore virtual spaces as proxies of real ones without real-world hazards and the limited availability of sighted assistants. However, virtual objects and environments have no physicality. Therefore, this thesis aims to research which VR interaction forms are reasonable (i.e., offering a reasonable dissemination potential) to make virtual representations of real buildings touchable or walkable in the context of orientation and mobility. Although there are already content and technology disjunctive developments and evaluations on VR technology, there is a lack of empirical evidence. Additionally, this thesis provides a survey between different interactions. Having considered the human physiology, assistive media (e.g., tactile maps), and technological characteristics, the current state of the art of VR is introduced, and the application for blind and visually impaired users and the way to get there is discussed by introducing a novel taxonomy. In addition to the interaction itself, characteristics of the user and the device, the application context, or the user-centered development respectively evaluation are used as classifiers. Thus, the following chapters are justified and motivated by explorative approaches, i.e., in the group of 'small scale' (using so-called data gloves) and in the scale of 'large scale' (using an avatar-controlled VR locomotion) approaches. The following chapters conduct empirical studies with blind and visually impaired users and give formative insight into how virtual objects within hands' reach can be grasped using haptic feedback and how different kinds of VR locomotion implementation can be applied to explore virtual environments. Thus, device-independent technological possibilities and also challenges for further improvements are derived. On the basis of this knowledge, subsequent research can be focused on aspects such as the specific design of interactive elements, temporally and spatially collaborative application scenarios, and the evaluation of an entire application workflow (i.e., scanning the real environment and exploring it virtually for training purposes, as well as designing the entire application in a long-term accessible manner)

μGraph: Haptic Exploration and Editing of 3D Chemical Diagrams

People with visual impairments or blindness (VIB) encounter diffculties in exploring graphical representations that are widely used for the study of STEM subjects. In particular, graphs are used to represent many different scientifc notations: fowcharts, automata, cognitive maps, and more. Among these, structural chemical formulae are characterized by a complex, often 3-dimensional structure, which makes them hard to access and author with traditional assistive tools. We propose \ub5Graph, a multimodal system that combines haptic and speech feedback to enable people with VIB to explore and edit structural chemical formulae. Two main contributions are presented: (i) a novel, non-visual interaction paradigm for exploring graphs and its implementation in the \ub5Graph system, and (ii) an extensive evaluation of the proposed system with 10 participants with VIB showing that \ub5Graph is thoroughly accessible and that the haptic feedback enhances understanding of the geometric properties of a graph

A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research

This paper summarizes recent developments in audio and tactile feedback based assistive technologies targeting the blind community. Current technology allows applications to be efficiently distributed and run on mobile and handheld devices, even in cases where computational requirements are significant. As a result, electronic travel aids, navigational assistance modules, text-to-speech applications, as well as virtual audio displays which combine audio with haptic channels are becoming integrated into standard mobile devices. This trend, combined with the appearance of increasingly user- friendly interfaces and modes of interaction has opened a variety of new perspectives for the rehabilitation and training of users with visual impairments. The goal of this paper is to provide an overview of these developments based on recent advances in basic research and application development. Using this overview as a foundation, an agenda is outlined for future research in mobile interaction design with respect to users with special needs, as well as ultimately in relation to sensor-bridging applications in genera

Digital technologies applied to the accessible management of museums. The first experiments carried out at the Museum of Oriental Art of Turin

[EN] The proposed research focuses on the museum context of the city of Turin (Italy), within an ongoing framework agreement between the Politecnico di Torino and the Museum of Oriental Art (MAO). It aims at the construction of a model for the management of the workflow that includes the digital survey, the 3D virtual modelling and the digital fabrication of the tactile models of the artworks and the exhibition spaces. The paper starts with an overview on the methods of conservation and dissemination of cultural and architectural heritage, on the accessibility of museums and on tactile perception and graphics. It presents the first outcomes of the designed workflow with a special focus on the digital fabrication experiments on the vaulted system of the atrium of Palazzo Mazzonis (MAO home). The founding idea is that tactile exploration of artifacts helps the visitor engagement, making the museum experience more educationally incisive and more inclusive, in a “Design for All” perspective.[ES] La investigación propuesta se centra en el contexto museístico de la ciudad de Turín (Italia), dentro de un acuerdo marco en curso entre el Politecnico di Torino y el Museo de Arte Oriental (MAO). Su objetivo es la construcción de un modelo para la gestión del flujo de trabajo que parte de la digitalización de las obras de arte y de los espacios y termina con la realización de modelos táctiles. El artículo comienza con una visión general sobre los métodos de conservación y difusión del patrimonio cultural y arquitectónico sobre la percepción y los gráficos táctiles. Presenta los primeros resultados del flujo de trabajo diseñado, centrándose especialmente en los experimentos de fabricación digital en el sistema abovedado del atrio del Palazzo Mazzonis (casa del MAO). La idea básica es que la exploración táctil de los artefactos ayuda a la participación del visitante, haciendo que la experiencia del museo sea más incisiva desde el punto de vista educativo y más inclusiva, en una perspectiva de “Diseño para todos”.This research is carried out within the framework of an agreement between the Museo d’Arte Orientale and the Politecnico di Torino. I would like to thank Dr Marco Guglielminotti Trivel, Director of the Museum, Dr Claudia Ramasso, museum curator and Mrs Patrizia Bosio, from the Technical and Security Office for having favored the current research. In addition, I would like to thank Dr Franco Lepore Disability Manager of the Municipality of Turin, and Arch. Rocco Rolli, founder of Tactile Vision Onlus, for their willingness to help me in the management of the model testing phase that will take place soon.Ronco, F. (2021). Digital technologies applied to the accessible management of museums. The first experiments carried out at the Museum of Oriental Art of Turin. EGE Revista de Expresión Gráfica en la Edificación. 0(14):49-61. https://doi.org/10.4995/ege.2021.15661OJS4961014Amoruso, G. 2019. "UID PhD Summer School. Rilievo del patrimonio culturale e rappresentazione inclusiva". Diségno, n. 4, p. 261-264. ISSN 2533-2899Anagnostakis, G., Antoniou, M., Kardamitsi, E., Sachinidis, T., Koutsabasis, P., Stavrakis, S., Vosinakis, M., Zissis, D. 2016. "Accessible museum collections for the visually impaired: combining tactile exploration, audio descriptions and mobile gestures". In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16). New York: Association for Computing Machinery, p. 1021-1025. https://doi.org/10.1145/2957265.2963118Balletti, C., Ballarin, M. 2019. "An Application of Integrated 3D Technologies for Replicas in Cultural Heritage". ISPRS Int. J. Geo-Inf., n. 8, p. 285-313. https://doi.org/10.3390/ijgi8060285Barberà Giné, A. 2018. "Fotogrametría para la conservación-restauración de bienes culturales". Unicum, n. 17, p. 153-162. ISSN: 1579-3613.Bo, P., Pottmann, H., Kilian, M., Wang, W., Wallner, J. 2011. "Circular arc structures". ACM Trans. Graph., vol. 30, n. 4, p. 101:1-101:11. ISSN 0730-0301. https://doi.org/10.1145/2010324.1964996Brie, M., Morice, J. C. 1996. "Il disegno in rilievo: oggetto di conoscenza". XY, Dimensioni del Disegno, n. 26, p. 38-51.Bruno, A., Ricca, F. 2010. Il Museo d'Arte Orientale MAO. Torino: Allemandi, p. 3-24. ISBN 978-88-422-1699-5.Bruno, Ivana. 2019. "Comunicazione e accessibilità culturale. L'esperienza di Museo Facile". Il capitale culturale, n. 20, p. 297-325. http://dx.doi.org/10.13138/2039-2362/2068.Candling, F. 2010. Art, Museums and Touch. Manchester: Manchester University Press. ISBN 978-0719079337Clini, P., Frapiccini, N., Quattrini, R., Nespeca, R. 2018. "Toccare l'arte e guardare con altri occhi. Una via digitale per la rinascita dei musei archeologici nell'epoca della riproducibilità dell'opera d'arte". In Luigini, Alessandro and Panciroli, Chiara eds. Ambienti digitali per l'educazione all'arte e al patrimonio. Milano: FrancoAngeli s.r.l., p. 97-113. ISBN 9788891773333. http://library.oapen.org/handle/20.500.12657/25364 [accessed on 2020/05/04].D'Agnano, F., Balletti, C., Guerra, F., Vernier, P.. 2015. "Tooteko: A case study of augmented reality for an accessible cultural heritage. Digitization, 3D printing and sensors for an audio-tactile experience". International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 5W4, p. 207-213. https://doi.org/10.5194/isprsarchives-XL-5-W4-207-2015De Rubertis, R. (ed.) 1996. "Il disegno oscuro". XY Dimensioni del Disegno, n. 26Díaz Gómez, F., Jiménez,Peiró J., Barreda Benavent, A., Asensi Recuenco, B., Hervás Juan, J. 2015. " Modelado 3D para la generación de patrimonio virtual. 3D modeling for the generation of virtual heritage". Virtual Archaeology Review, vol. 6, no 12, p. 29-37. ISSN 1989-9947. https://doi.org/10.4995/var.2015.4150Empler, T. 1996. Il "disegno in rilievo negli Istituti di Ricerca italiani e francesi". XY, Dimensioni del Disegno, n. 26, p. 5-7.Empler, T. 2013. "Universal Design: ruolo del Disegno e Rilievo". Disegnare. Idee Immagini, n. 46, p. 52-63. ISBN 978-88-492-1835-0.Empler, T., Fusinetti, A. 2019. "Rappresentazione visuo-tattile: comunicazione tattile per i disabili visivi". In Belardi, Paolo ed. UID per il disegno:2019. Riflessioni: l'arte del disegno/ il disegno dell'arte. Roma : Gangemi, p 1563-1572. ISBN 978-88-492-3762-7. https://www.torrossa.com/it/resources/an/4557553 [accessed on 2020/05/04].García Lucerga, M. A. 1993. El acceso de las personas deficientes visuales al mundo de los museos. Madrid: Organización Nacional de Ciegos Españoles, Sección de Cultura D.L. ISBN 84-87277-25-X. https://portal.once.es/bibliotecas/fondo-bibliografico-discapacidad-visual [accessed on 2020/05/04].Levi, F., Rolli, R. 1994. Disegnare per le mani. Manuale di disegno in rilievo. Torino: Silvio Zamorani Editore. ISBN: 9788871580388.Montella, M., Petraroia, P., Manacorda, Daniele, Di Macco, Michela. 2016. In Feliciati, Pierluigi, ed. La valorizzazione dell'eredità culturale in Italia, Atti del Convegno (Macerata 5-6 novembre 2015), Supplemento, n. 5. Macerata: eum edizioni, p. 13-36. ISBN 978-88-6056-485-6.Nasini, L., Isawi, H. 2006. Una geometria per comprendere lo spazio senza percepirlo visivamente. Roma: Officina Edizioni. ISBN: 9788860490070.Pérez de Andrés, C., Ramos Fuentes, A. 1994. Museos abiertos a todos los sentidos : acoger mejor a las personas minusválidas. Salamanca: Ministerio de Cultura, Organización Nacional de Ciegos Españoles, Sección de Cultura 1994. ISBN 84-87277-40-3. https://portal.once.es/bibliotecas/fondo-bibliografico-discapacidad-visual [accessed on 2020/05/04].Petrelli, D., Ciolfi, L., Van DiJk, D., Horneker, E., Not, E., Schmidt A. 2013. "Integrating material and digital: A new way for cultural heritage". Interactions: new visions of human-computer, n. 20, p. 58-63. https://doi.org/10.1145/2486227.2486239Photoworks: 3D Photo Printing Products. URL: https://www.3dphotoworks.com/product [accessed on 2020/05/04].Reichinger, A., Neumüller, M., Rist, F., Maierhofer, S., Purgathofer, W. 2012. "Computer-Aided Design of Tactile Models. Taxonomy and Case Studies". In Miesenberger, Klaus et al. eds. ICCHP 2012, Part II, LNCS 7383, p. 497-504. ISBN 978-3-642-31533-6. https://doi.org/10.1007/978-3-642-31534-3_73Reichinger, A., Fuhrmann, A., Maierhofer, S., Purgathofer, W. 2016. "Gesture-Based Interactive Audio Guide on Tactile Reliefs". In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). New York: Association for Computing Machinery, p. 91-100. https://doi.org/10.1145/2982142.2982176Samaroudi, M., Echavarria, K.R., Rodriguez, Song, R., Evans, R. 2017. "The Fabricated Diorama: Tactile Relief and Context-aware Technology for Visually Impaired Audiences". In Proceedings of the Eurographics Workshop on Graphics and Cultural Heritage (GCH '17). Goslar: Eurographics Association, p 201-206. https://doi.org/10.2312/gch.20171315.Sdegno, A. 2018. "Rappresentare l'opera d'arte con tecnologie digitali: dalla realtà aumentata alle esperienze tattili". In Luigini, Alessandro and Panciroli, Chiara eds. Ambienti digitali per l'educazione all'arte e al patrimonio. Milano: FrancoAngeli s.r.l., p. 257-272. ISBN 9788891773333Secchi, L. 2018. "Toccare con gli occhi e vedere con le mani. Funzioni cognitive e conoscitive dell'educazione estetica". Ocula, vol.19, n.19, p. 15-31. https://doi.org/10.12977/ocula2018-9Shi, L., Zelzer, I., Feng, C., Azenkot, S. 2016. "Tickers and Talker: An Accessible Labeling Toolkit for 3D Printed Models". In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). New York: Association for Computing Machinery, p. 4896-4907. https://doi.org/10.1145/2858036.2858507Spallone, R., Vitali, M. 2017. Star-shaped and Planterian Vaults in the Baroque Atria of Turin. Ariccia: Aracne. ISBN 978-88-255-0472-9Wilson, P. F., Stott, J., Warnett, J. M., Attridge, A., Smith, M. P., Williams, M. A. 2017. "Evaluation of touchable 3D-printed replicas in museums". Curator. The Museum Journal, vol. 60, n. 4, p. 445-465. https://doi.org/10.1111/cura.1224

Designing Accessible Nonvisual Maps

Access to nonvisual maps has long required special equipment and training to use; Google Maps, ESRI, and other commonly used digital maps are completely visual and thus inaccessible to people with visual impairments. This project presents the design and evaluation of an easy to use digital auditory map and 3D model interactive map. A co-design was also undertaken to discover tools for an ideal nonvisual navigational experience. Baseline results of both studies are presented so future work can improve on the designs. The user evaluation revealed that both prototypes were moderately easy to use. An ideal nonvisual navigational experience, according to these participants, consists of both an accurate turn by turn navigational system, and an interactive map. Future work needs to focus on the development of appropriate tools to enable this ideal experience

Using wrist vibrations to guide hand movement and whole body navigation

International audienceIn the absence of vision, mobility and orientation are challenging. Audio and tactile feedback can be used to guide visually impaired people. In this paper, we present two complementary studies on the use of vibrational cues for hand guidance during the exploration of itineraries on a map, and whole body-guidance in a virtual environment. Concretely, we designed wearable Arduino bracelets integrating a vibratory motor producing multiple patterns of pulses. In a first study, this bracelet was used for guiding the hand along unknown routes on an interactive tactile map. A wizard-of-Oz study with six blindfolded participants showed that tactons, vibrational patterns, may be more efficient than audio cues for indicating directions. In a second study, this bracelet was used by blindfolded participants to navigate in a virtual environment. The results presented here show that it is possible to significantly decrease travel distance with vibrational cues. To sum up, these preliminary but complementary studies suggest the interest of vibrational feedback in assistive technology for mobility and orientation for blind people

Haptic Reassurance in the Pitch Black for an Immersive Theatre Experience

An immersive theatre experience was designed to raise awareness and question perceptions of ‘blindness’, through enabling both sighted and blind members to experience a similar reality. A multimodal experience was created, comprising ambient sounds and narratives – heard through headphones – and an assortment of themed tactile objects, intended to be felt. In addition, audience members were each provided with a novel haptic device that was designed to enhance their discovery of a pitch-black space. An in the wild study of the cultural experience showed how blind and sighted audience members had different ‘felt’ experiences, but that neither was a lesser one. Furthermore, the haptic device was found to encourage enactive exploration and provide reassurance of the environment for both sighted and blind people, rather than acting simply as a navigation guide. We discuss the potential of using haptic feedback to create cultural experiences for both blind and sighted people; rethinking current utilitarian framing of it as assistive technology

LIMITATIONS AND ADVANTAGES OF COMPUTER TECHNOLOGY IN COMMUNICATION OF PERSONS WITH IMPAIRED VISION

Modern society requires a constant keeping up with innovative trends in the field of information literacy and knowledge of new computer technologies. In order for each individual to be fully integrated into social life, to progress in education and to socialize successfully, it is necessary to master the basics of computer literacy. People with visual impairments tend to fit into the educational and social environment with the help of computer technology, but they mostly encounter difficulties due to insufficient knowledge of the individual needs of each individual. It is necessary to ensure accessibility, equal conditions of use for all persons and thus enable them to successfully establish and maintain communicatio

Principles and Guidelines for Advancement of Touchscreen-Based Non-visual Access to 2D Spatial Information

Graphical materials such as graphs and maps are often inaccessible to millions of blind and visually-impaired (BVI) people, which negatively impacts their educational prospects, ability to travel, and vocational opportunities. To address this longstanding issue, a three-phase research program was conducted that builds on and extends previous work establishing touchscreen-based haptic cuing as a viable alternative for conveying digital graphics to BVI users. Although promising, this approach poses unique challenges that can only be addressed by schematizing the underlying graphical information based on perceptual and spatio-cognitive characteristics pertinent to touchscreen-based haptic access. Towards this end, this dissertation empirically identified a set of design parameters and guidelines through a logical progression of seven experiments. Phase I investigated perceptual characteristics related to touchscreen-based graphical access using vibrotactile stimuli, with results establishing three core perceptual guidelines: (1) a minimum line width of 1mm should be maintained for accurate line-detection (Exp-1), (2) a minimum interline gap of 4mm should be used for accurate discrimination of parallel vibrotactile lines (Exp-2), and (3) a minimum angular separation of 4mm should be used for accurate discrimination of oriented vibrotactile lines (Exp-3). Building on these parameters, Phase II studied the core spatio-cognitive characteristics pertinent to touchscreen-based non-visual learning of graphical information, with results leading to the specification of three design guidelines: (1) a minimum width of 4mm should be used for supporting tasks that require tracing of vibrotactile lines and judging their orientation (Exp-4), (2) a minimum width of 4mm should be maintained for accurate line tracing and learning of complex spatial path patterns (Exp-5), and (3) vibrotactile feedback should be used as a guiding cue to support the most accurate line tracing performance (Exp-6). Finally, Phase III demonstrated that schematizing line-based maps based on these design guidelines leads to development of an accurate cognitive map. Results from Experiment-7 provide theoretical evidence in support of learning from vision and touch as leading to the development of functionally equivalent amodal spatial representations in memory. Findings from all seven experiments contribute to new theories of haptic information processing that can guide the development of new touchscreen-based non-visual graphical access solutions