Touch- and Walkable Virtual Reality to Support Blind and Visually Impaired Peoples‘ Building Exploration in the Context of Orientation and Mobility

Der Zugang zu digitalen Inhalten und Informationen wird immer wichtiger für eine erfolgreiche Teilnahme an der heutigen, zunehmend digitalisierten Zivilgesellschaft. Solche Informationen werden meist visuell präsentiert, was den Zugang für blinde und sehbehinderte Menschen einschränkt. Die grundlegendste Barriere ist oft die elementare Orientierung und Mobilität (und folglich die soziale Mobilität), einschließlich der Erlangung von Kenntnissen über unbekannte Gebäude vor deren Besuch. Um solche Barrieren zu überbrücken, sollten technische Hilfsmittel entwickelt und eingesetzt werden. Es ist ein Kompromiss zwischen technologisch niedrigschwellig zugänglichen und verbreitbaren Hilfsmitteln und interaktiv-adaptiven, aber komplexen Systemen erforderlich. Die Anpassung der Technologie der virtuellen Realität (VR) umfasst ein breites Spektrum an Entwicklungs- und Entscheidungsoptionen. Die Hauptvorteile der VR-Technologie sind die erhöhte Interaktivität, die Aktualisierbarkeit und die Möglichkeit, virtuelle Räume und Modelle als Abbilder von realen Räumen zu erkunden, ohne dass reale Gefahren und die begrenzte Verfügbarkeit von sehenden Helfern auftreten. Virtuelle Objekte und Umgebungen haben jedoch keine physische Beschaffenheit. Ziel dieser Arbeit ist es daher zu erforschen, welche VR-Interaktionsformen sinnvoll sind (d.h. ein angemessenes Verbreitungspotenzial bieten), um virtuelle Repräsentationen realer Gebäude im Kontext von Orientierung und Mobilität berührbar oder begehbar zu machen. Obwohl es bereits inhaltlich und technisch disjunkte Entwicklungen und Evaluationen zur VR-Technologie gibt, fehlt es an empirischer Evidenz. Zusätzlich bietet diese Arbeit einen Überblick über die verschiedenen Interaktionen. Nach einer Betrachtung der menschlichen Physiologie, Hilfsmittel (z.B. taktile Karten) und technologischen Eigenschaften wird der aktuelle Stand der Technik von VR vorgestellt und die Anwendung für blinde und sehbehinderte Nutzer und der Weg dorthin durch die Einführung einer neuartigen Taxonomie diskutiert. Neben der Interaktion selbst werden Merkmale des Nutzers und des Geräts, der Anwendungskontext oder die nutzerzentrierte Entwicklung bzw. Evaluation als Klassifikatoren herangezogen. Begründet und motiviert werden die folgenden Kapitel durch explorative Ansätze, d.h. im Bereich 'small scale' (mit sogenannten Datenhandschuhen) und im Bereich 'large scale' (mit einer avatargesteuerten VR-Fortbewegung). Die folgenden Kapitel führen empirische Studien mit blinden und sehbehinderten Nutzern durch und geben einen formativen Einblick, wie virtuelle Objekte in Reichweite der Hände mit haptischem Feedback erfasst werden können und wie verschiedene Arten der VR-Fortbewegung zur Erkundung virtueller Umgebungen eingesetzt werden können. Daraus werden geräteunabhängige technologische Möglichkeiten und auch Herausforderungen für weitere Verbesserungen abgeleitet. Auf der Grundlage dieser Erkenntnisse kann sich die weitere Forschung auf Aspekte wie die spezifische Gestaltung interaktiver Elemente, zeitlich und räumlich kollaborative Anwendungsszenarien und die Evaluation eines gesamten Anwendungsworkflows (d.h. Scannen der realen Umgebung und virtuelle Erkundung zu Trainingszwecken sowie die Gestaltung der gesamten Anwendung in einer langfristig barrierefreien Weise) konzentrieren.Access to digital content and information is becoming increasingly important for successful participation in today's increasingly digitized civil society. Such information is mostly presented visually, which restricts access for blind and visually impaired people. The most fundamental barrier is often basic orientation and mobility (and consequently, social mobility), including gaining knowledge about unknown buildings before visiting them. To bridge such barriers, technological aids should be developed and deployed. A trade-off is needed between technologically low-threshold accessible and disseminable aids and interactive-adaptive but complex systems. The adaptation of virtual reality (VR) technology spans a wide range of development and decision options. The main benefits of VR technology are increased interactivity, updatability, and the possibility to explore virtual spaces as proxies of real ones without real-world hazards and the limited availability of sighted assistants. However, virtual objects and environments have no physicality. Therefore, this thesis aims to research which VR interaction forms are reasonable (i.e., offering a reasonable dissemination potential) to make virtual representations of real buildings touchable or walkable in the context of orientation and mobility. Although there are already content and technology disjunctive developments and evaluations on VR technology, there is a lack of empirical evidence. Additionally, this thesis provides a survey between different interactions. Having considered the human physiology, assistive media (e.g., tactile maps), and technological characteristics, the current state of the art of VR is introduced, and the application for blind and visually impaired users and the way to get there is discussed by introducing a novel taxonomy. In addition to the interaction itself, characteristics of the user and the device, the application context, or the user-centered development respectively evaluation are used as classifiers. Thus, the following chapters are justified and motivated by explorative approaches, i.e., in the group of 'small scale' (using so-called data gloves) and in the scale of 'large scale' (using an avatar-controlled VR locomotion) approaches. The following chapters conduct empirical studies with blind and visually impaired users and give formative insight into how virtual objects within hands' reach can be grasped using haptic feedback and how different kinds of VR locomotion implementation can be applied to explore virtual environments. Thus, device-independent technological possibilities and also challenges for further improvements are derived. On the basis of this knowledge, subsequent research can be focused on aspects such as the specific design of interactive elements, temporally and spatially collaborative application scenarios, and the evaluation of an entire application workflow (i.e., scanning the real environment and exploring it virtually for training purposes, as well as designing the entire application in a long-term accessible manner)

Modelo multissensorial desenvolvido por tecnologias 3d para o auxílio na percepção da forma de peças museológicas por pessoas com deficiência visual

O uso de tecnologias 3D aliadas ao design oferece novas oportunidades para o campo da tecnologia assistiva. Em museus, há um grande potencial para o desenvolvimento de réplicas táteis que visem, principalmente, incluir o público com deficiência visual. Porém, são escassos os estudos de sistemas multissensoriais aplicados nessas réplicas, por exemplo, combinando tato e audição. Nesse contexto, essa pesquisa tem por objetivo o desenvolvimento e a avaliação de um sistema multissensorial, desenvolvido por tecnologias 3D, para o auxílio a pessoas com deficiência visual na percepção da forma de peças museológicas. Para tanto, o estudo foi conduzido junto ao Museu de Porto Alegre, tendo início com a digitalização 3D de uma peça do acervo e sua respectiva conversão em um modelo virtual, o qual foi posteriormente impresso em 3D. Em paralelo, a fim de possibilitar a interação entre a réplica tátil e o usuário, foi selecionado um sensor de toque capacitivo como o responsável pelo feedback de áudio, bem como foi desenvolvida a programação do sistema. Em meio a isso, desenvolveu-se uma tinta condutora para atuar como o sensor de toque capacitivo supracitado, a qual foi aplicada em regiões selecionadas para serem descritas aos usuários. Assim que o sistema foi montado junto ao modelo finalizado, o roteiro das audiodescrições foi escrito, gravado e incluído no sistema. Por fim, o modelo multissensorial foi avaliado por meio de um teste piloto e, posteriormente, foi realizada uma consultoria de audiodescrição junto a um profissional com deficiência visual, no intuito de otimizar as audiodescrições, bem como colocar o modelo multissensorial à prova, visando observar o desempenho, pontos positivos e negativos. Como resultado, o sistema tornou-se autossuficiente e o usuário livre para utilizar o sistema de múltiplas formas, seja seguindo as orientações sugeridas, seja explorando-o conforme desejado, bem como retomando informações que queira ouvir novamente. Assim, os resultados indicam que o sistema desenvolvido é eficiente quanto ao auxílio na percepção da forma por uma pessoa com deficiência visual e já poderia ser implementado em um museu para promover a acessibilidade.The application of 3D technology in design brings in new opportunities to the field of assistive technology. In museums, there is vast potential in the development of tactile replicas with the goal to include visually impaired people. However, there is a significant lack of research approaching multi-sensory systems applied in these replicas combining touch and hearing, for example. In this context, this paper intends to both elaborate and assess a multi-sensory system, developed with the use of 3D technology, for the assistance of people with visual deficiency in the perception of museum exhibits. For this purpose, this study was carried out closely with the Museum of Porto Alegre, beginning from the 3D digitization of an exhibit and its conversion to a virtual model, which was afterward printed in 3D. Simultaneously, in order to enable the interaction between the tactile replica and the user, a capacitive touch sensor was set to be responsible for the audio feedback, and the system programming was also consolidated. A conductive paint was conceived to act as the touch capacitive sensor mentioned above, which was applied in previously selected regions to be described to the users. Once the system was assembled together with the finalized model, the script for the audio descriptions was written, recorded and incorporated into the system. Lastly, the multi-sensory model was assessed by means of a trial, and then a visually impaired professional was consulted on the audio description, with the purpose of optimizing the audio descriptions as well as to put the multi-sensory model to the test, aiming to analyze its performance. As a result, the system could be declared self sufficient, and the user was able to operate the system in multiple ways, whether following the instructions suggested, or exploring it as desired, as well as to revisit information if they perchance wanted to listen again. Thus, the results indicate that the developed system is efficient in regards to aiding the perception of shape to a person with visual deficiency and could already be implemented in a museum so as to amplify the accessibility