O ambiente virtual áudio-háptico como instrumento para a Aprendizagem de geometria : estudo das formas para estudantes cegos

O desenvolvimento das habilidades para os objetos de conhecimento da geometria está relacionado com as formas de organização de aprendizagem matemática e com os recursos didáticos utilizados para a construção de competências para o pensamento matemático (BRASIL, 2017). Esta tese se propõe a investigar como um ambiente virtual áudio-háptico pode contribuir na aprendizagem de geometria para estudantes cegos no Ensino Fundamental, com base em uma sequência de tarefas que contemplem os objetivos de conhecimento e habilidades previstos na Base Nacional Comum Curricular (BNCC). O referencial teórico proposto para esta pesquisa destaca os seguintes temas: a Teoria da Abstração Reflexionante de Piaget (1995), que traz contribuições para a compreensão da construção de conhecimentos; a percepção háptica humana na perspectiva de Lederman e Klatzky (1987), que propõem um conjunto de procedimentos exploratórios (Exploratory Procedure – EPs) para uma pessoa examinar um objeto com ou sem visão no intuito de perceber propriedades por intermédio do tato, bem como discutem as sensações que são convertidas pelo cérebro em informações cutâneas e cinestésica; a tecnologia háptica de hardware, em que se apresentam os dispositivos que permitem a interação de pessoas com ambientes virtuais por meio do tato e feedback de força; e o estudo da geometria, evidenciando a importância da aprendizagem dos conceitos geométricos no Ensino Fundamental, e a Tecnologia Assistiva digital com foco no sentido tátil-cinestésico para inclusão de estudantes cegos no estudo da geometria. A pesquisa apresenta abordagem qualitativa de natureza aplicada e foi realizada no Instituto Benjamin Constant, instituição de ensino para deficientes visuais localizada no bairro da Urca, na cidade e no estado do Rio de Janeiro. Para a coleta de dados, foram utilizadas as técnicas de observação participante, gravação de vídeo e Think Aloud, com o intuito de explorar os fatores de eficácia e eficiência e o mecanismo de abstração reflexionante na construção de conhecimentos geométricos. Os dados coletados foram analisados por meio da técnica de análise categorial prevista no método de análise de conteúdo (BARDIN, 2016). Acredita-se que esta tese possa contribuir como um recurso assistivo que apoie a aprendizagem de geometria – estudo das formas para estudantes cegos no Ensino Fundamental.The development of skills for the objects of knowledge of geometry is related to the forms of organization of mathematical learning and the didactic resources used to build competencies for mathematical thinking (BRASIL, 2017). This thesis aims to investigate how an audiohaptic virtual environment can contribute to Geometry Learning for Blind Elementary School Students, based on a sequence of tasks that address the objectives of knowledge and skills foreseen in the Common National Curriculum Base (BNCC). The theoretical framework proposed for this research highlights the following themes: Piaget's theory of reflect abstraction, Piaget (1995), which brings contributions to the understanding of the construction of knowledge; the human pain perception from the perspective of Lederman and Klatzky (1987), who propose a set of Exploratory Procedures (EPs) for a person to examine an object with or without vision in order to perceive properties through touch, as well as discuss the sensations that are converted by the brain into cutaneous and kinesthetic information; the political technology of hardware, in which the devices that allow the interaction of people with virtual environments through touch and force feedback are presented; and the study of geometry, evidencing the importance of learning geometric concepts in Elementary School, and digital assistive technology focused on the tactile-kinesthetic sense for the inclusion of blind students in the study of geometry. The research presents a qualitative approach of an applied nature and was held at the Benjamin Constant Institute, an educational institution for the visually impaired located in the Urca neighborhood, in the city and in the state of Rio de Janeiro. For data collection of this investigation, participant observation, video recording and Think Aloud techniques were used in order to explore the factors of efficacy and efficiency and the mechanism of reflecting abstraction in the construction of geometric knowledge. The collected data were analyzed using the category analysis technique provided for in the content analysis method (BARDIN, 2016). It is believed that this can contribute as an assistive resource that supports the learning of geometry – study of forms for blind students in Elementary School

Virtual envonments and spatial ability to people with special educational needs (SEN)/disabilities

Σημείωση: διατίθεται συμπληρωματικό υλικό σε ξεχωριστό αρχείο

Uso de los Entornos Virtuales en la comprensión del espacio en personas con ceguera y discapacidad visual

La instrucción en habilidades para el desplazamiento autónomo (denominada en el ámbito científico como Orientación y movilidad, O&M) es fundamental dentro del proceso de rehabilitación e inclusión social de las personas con ceguera y discapacidad visual (DV). Durante los últimos años, ha comenzado a utilizarse software de Entornos Virtuales (EV) interactivos representados a través de interfaces de audio y/o háptica para el aprendizaje de habilidades de navegación en niños y adultos con ceguera y DV. Paralelamente, los métodos de exploración neurológica han avanzado, permitiendo la observación de la actividad neurológica del sujeto mientras realiza tareas que implican habilidades espaciales. En el presente trabajo se realizó una revisión bibliográfica sobre el uso de los EV en la comprensión del espacio en personas con ceguera y DV, además de estudiar algunos aspectos generales de la cognición espacial y de la neuroplasticidad en las personas con ceguera y DV. Las principales conclusiones fueron que las personas con ceguera y DV pueden desarrollar mapas cognitivos espaciales y que su desarrollo cognitivo para estas tareas es similar al de las personas con visión, además, que el cerebro recluta las zonas de la visión para este tipo de tareas y que los EV basados en audio y/o háptica permiten el desarrollo de habilidades de O&M, así como la construcción de mapas cognitivos espaciales precisos en personas con ceguera y DV, los cuales pueden ser transferidos a tareas de navegación en el entorno real.Máster en Rehabilitación Visua

Touch- and Walkable Virtual Reality to Support Blind and Visually Impaired Peoples‘ Building Exploration in the Context of Orientation and Mobility

Der Zugang zu digitalen Inhalten und Informationen wird immer wichtiger für eine erfolgreiche Teilnahme an der heutigen, zunehmend digitalisierten Zivilgesellschaft. Solche Informationen werden meist visuell präsentiert, was den Zugang für blinde und sehbehinderte Menschen einschränkt. Die grundlegendste Barriere ist oft die elementare Orientierung und Mobilität (und folglich die soziale Mobilität), einschließlich der Erlangung von Kenntnissen über unbekannte Gebäude vor deren Besuch. Um solche Barrieren zu überbrücken, sollten technische Hilfsmittel entwickelt und eingesetzt werden. Es ist ein Kompromiss zwischen technologisch niedrigschwellig zugänglichen und verbreitbaren Hilfsmitteln und interaktiv-adaptiven, aber komplexen Systemen erforderlich. Die Anpassung der Technologie der virtuellen Realität (VR) umfasst ein breites Spektrum an Entwicklungs- und Entscheidungsoptionen. Die Hauptvorteile der VR-Technologie sind die erhöhte Interaktivität, die Aktualisierbarkeit und die Möglichkeit, virtuelle Räume und Modelle als Abbilder von realen Räumen zu erkunden, ohne dass reale Gefahren und die begrenzte Verfügbarkeit von sehenden Helfern auftreten. Virtuelle Objekte und Umgebungen haben jedoch keine physische Beschaffenheit. Ziel dieser Arbeit ist es daher zu erforschen, welche VR-Interaktionsformen sinnvoll sind (d.h. ein angemessenes Verbreitungspotenzial bieten), um virtuelle Repräsentationen realer Gebäude im Kontext von Orientierung und Mobilität berührbar oder begehbar zu machen. Obwohl es bereits inhaltlich und technisch disjunkte Entwicklungen und Evaluationen zur VR-Technologie gibt, fehlt es an empirischer Evidenz. Zusätzlich bietet diese Arbeit einen Überblick über die verschiedenen Interaktionen. Nach einer Betrachtung der menschlichen Physiologie, Hilfsmittel (z.B. taktile Karten) und technologischen Eigenschaften wird der aktuelle Stand der Technik von VR vorgestellt und die Anwendung für blinde und sehbehinderte Nutzer und der Weg dorthin durch die Einführung einer neuartigen Taxonomie diskutiert. Neben der Interaktion selbst werden Merkmale des Nutzers und des Geräts, der Anwendungskontext oder die nutzerzentrierte Entwicklung bzw. Evaluation als Klassifikatoren herangezogen. Begründet und motiviert werden die folgenden Kapitel durch explorative Ansätze, d.h. im Bereich 'small scale' (mit sogenannten Datenhandschuhen) und im Bereich 'large scale' (mit einer avatargesteuerten VR-Fortbewegung). Die folgenden Kapitel führen empirische Studien mit blinden und sehbehinderten Nutzern durch und geben einen formativen Einblick, wie virtuelle Objekte in Reichweite der Hände mit haptischem Feedback erfasst werden können und wie verschiedene Arten der VR-Fortbewegung zur Erkundung virtueller Umgebungen eingesetzt werden können. Daraus werden geräteunabhängige technologische Möglichkeiten und auch Herausforderungen für weitere Verbesserungen abgeleitet. Auf der Grundlage dieser Erkenntnisse kann sich die weitere Forschung auf Aspekte wie die spezifische Gestaltung interaktiver Elemente, zeitlich und räumlich kollaborative Anwendungsszenarien und die Evaluation eines gesamten Anwendungsworkflows (d.h. Scannen der realen Umgebung und virtuelle Erkundung zu Trainingszwecken sowie die Gestaltung der gesamten Anwendung in einer langfristig barrierefreien Weise) konzentrieren.Access to digital content and information is becoming increasingly important for successful participation in today's increasingly digitized civil society. Such information is mostly presented visually, which restricts access for blind and visually impaired people. The most fundamental barrier is often basic orientation and mobility (and consequently, social mobility), including gaining knowledge about unknown buildings before visiting them. To bridge such barriers, technological aids should be developed and deployed. A trade-off is needed between technologically low-threshold accessible and disseminable aids and interactive-adaptive but complex systems. The adaptation of virtual reality (VR) technology spans a wide range of development and decision options. The main benefits of VR technology are increased interactivity, updatability, and the possibility to explore virtual spaces as proxies of real ones without real-world hazards and the limited availability of sighted assistants. However, virtual objects and environments have no physicality. Therefore, this thesis aims to research which VR interaction forms are reasonable (i.e., offering a reasonable dissemination potential) to make virtual representations of real buildings touchable or walkable in the context of orientation and mobility. Although there are already content and technology disjunctive developments and evaluations on VR technology, there is a lack of empirical evidence. Additionally, this thesis provides a survey between different interactions. Having considered the human physiology, assistive media (e.g., tactile maps), and technological characteristics, the current state of the art of VR is introduced, and the application for blind and visually impaired users and the way to get there is discussed by introducing a novel taxonomy. In addition to the interaction itself, characteristics of the user and the device, the application context, or the user-centered development respectively evaluation are used as classifiers. Thus, the following chapters are justified and motivated by explorative approaches, i.e., in the group of 'small scale' (using so-called data gloves) and in the scale of 'large scale' (using an avatar-controlled VR locomotion) approaches. The following chapters conduct empirical studies with blind and visually impaired users and give formative insight into how virtual objects within hands' reach can be grasped using haptic feedback and how different kinds of VR locomotion implementation can be applied to explore virtual environments. Thus, device-independent technological possibilities and also challenges for further improvements are derived. On the basis of this knowledge, subsequent research can be focused on aspects such as the specific design of interactive elements, temporally and spatially collaborative application scenarios, and the evaluation of an entire application workflow (i.e., scanning the real environment and exploring it virtually for training purposes, as well as designing the entire application in a long-term accessible manner)

Proceedings of the 9th international conference on disability, virtual reality and associated technologies (ICDVRAT 2012)

The proceedings of the conferenc

Development of the multimodal system of educational game for partially sighted and blind children

У дисертацији су приказани развој мултимодалног система за образовну игру Луграм и истраживања везана за његову примену у области редовног и специјализованог основношколског образовања. Извршено је испитивање утицаја Луграма на повишење ефеката учења геометрије у редовној разредној настави, испитивање да ли његову прилагођену мултимодалну верзију могу да користе слабовида и слепа деца и да ли се она може користити за њихову обуку за самосталну употребу рачунарске аудио верзије Луграма. Утврђено је да Луграм има утицаја на пораст успеха у учењу геометрије и да се прилагођена аудио- тактилна верзија може користити у сврху обуке слабовиде и слепе деце за самостално коришћење аудио верзије игре. Резултати истраживања су показали оправданост приступа развоју Луграма као мултимодалног система за игру и усмерили његов даљи развој ка асистивном мултимодалном систему који поседује способност говорне интеракције са корисником и могућност прилагођења различитим категоријама корисника.U disertaciji su prikazani razvoj multimodalnog sistema za obrazovnu igru Lugram i istraživanja vezana za njegovu primenu u oblasti redovnog i specijalizovanog osnovnoškolskog obrazovanja. Izvršeno je ispitivanje uticaja Lugrama na povišenje efekata učenja geometrije u redovnoj razrednoj nastavi, ispitivanje da li njegovu prilagođenu multimodalnu verziju mogu da koriste slabovida i slepa deca i da li se ona može koristiti za njihovu obuku za samostalnu upotrebu računarske audio verzije Lugrama. Utvrđeno je da Lugram ima uticaja na porast uspeha u učenju geometrije i da se prilagođena audio- taktilna verzija može koristiti u svrhu obuke slabovide i slepe dece za samostalno korišćenje audio verzije igre. Rezultati istraživanja su pokazali opravdanost pristupa razvoju Lugrama kao multimodalnog sistema za igru i usmerili njegov dalji razvoj ka asistivnom multimodalnom sistemu koji poseduje sposobnost govorne interakcije sa korisnikom i mogućnost prilagođenja različitim kategorijama korisnika.The dissertation presents the development of the multimodal system for the educational game Lugram and the research related to its application in the regular and specialized primary education. The examination of the effect which Lugram has on the improvement in learning geometry in the primary education is done, as well as the testing weather its adapted multimodal version can be used for the training of partially sighted and blind children for the independent use of the computer audio version of Lugram. It is established that Lugram has an influence on the improvement in learning geometry and that adapted audio-tactile version can be used for the training purposes for partially sighted and blind children. Results of the research have justified the approach in the development of Lugram as a multimodal system for the game. Moreover, they focus further development toward assistive multimodal system which has ability of the voice interaction with the user and also ability of adaptation to the different kind of users

Assistance à la navigation pour les non-voyants : vers un positionnement, un SIG et un suivi adaptés

Le déplacement est une des plus grandes difficultés éprouvées par les déficients visuels dans leur vie quotidienne. Avec l'essor des TIC, plusieurs aides électroniques à la navigation ont vu le jour. L'objectif de ces systèmes est d'améliorer l'autonomie des déficients visuels dans leurs déplacements. Cependant, les aides au déplacement commercialisées aujourd'hui reposent sur une simple adaptation des dispositifs conçus pour les automobilistes et sont donc inadaptés ou mal adaptés à un piéton, de surcroit déficient visuel. Ces systèmes reposent généralement sur un positionnement peu précis et utilisent des cartographies et des stratégies de guidage mal ou non adaptées. Dans ce travail de doctorat, nous proposons la conception et le développement d'un système d'aide à la navigation destiné aux piétons déficients visuels. Cette thèse s'articule autour de trois problématiques identifiées comme critiques à savoir le positionnement, les systèmes d'informations géographiques et le suivi d'itinéraire. Le positionnement développé dans NAVIG repose sur la fusion multi capteurs dans le but d'en améliorer la précision. Cette fusion se base sur un module de vision par ordinateur, un système GPS et des capteurs inertiels. Les résultats obtenus montrent la faisabilité de ces techniques dans un contexte de navigation piétonne. Pour la cartographie, nous proposons une classification de données environnementales permettant d'assurer la perception de l'environnement et le calcul d'itinéraire, notamment pour un utilisateur privé de vision. Un modèle conceptuel adéquat pour le stockage des données est aussi proposé. Le suivi d'itinéraire est une fonction qui permet au système de savoir sur quelle section de l'itinéraire se trouve l'utilisateur à tout instant. C'est un processus important, car il permet d'assurer un guidage de qualité. L'amélioration du guidage passe donc par un suivi adapté à un déplacement piéton. Nous avons proposé quatre stratégies de suivi. L'évaluation des performances de ces différentes stratégies de suivi s'est déroulée dans un simulateur de guidage développé au cours de cette thèse. Il s'agit d'un environnement virtuel multimodal qui permet l'évaluation systématique des différents algorithmes proposés dans un environnement contrôlé et sécurisé. Finalement, un prototype du système NAVIG a été réalisé et a été testé auprès de deux utilisateurs déficients visuels, ce qui a mis en évidence un ensemble de verrous toujours présents. Pour conclure, bien que ce travail de doctorat doive être complété par un ensemble d'évaluations plus systématiques, il a apporté un ensemble de réponses sur un positionnement et un suivi d'itinéraire adaptés à des systèmes d'assistance à la navigation pour déficients visuels. Il ouvre des questions très actuelles sur la disponibilité et le partage des données géographiques et l'utilisation de la vision embarquée. Il ouvre aussi des perspectives intéressantes sur le guidage et la description de l'espace adapté à des utilisateurs piétons et déficients visuels.Navigation, especially in unknown areas, is a major problem for the visually impaired (VI). Over the past 50 years, a number of electronic travel aids (ETAs) have been developed with the aim of improving mobility of the VI. Despite many research efforts, these systems are rarely used. Although the explanation is likely to be incomplete, it is possible to identify three important factors : (1) positioning accuracy provided by these devices is not sufficient to guide a VI pedestrian, (2) these systems are based on Geographical Information Systems not adapted to pedestrian mobility, and (3) the guidance methods should be adapted to the task of VI pedestrian wayfinding. All these three components are sources of usability issues. In this thesis, committed in the collaborative research project called NAVIG, we present the design and implementation of an electronic navigation aid for the blind. In this work, we relied on the analysis of the needs of the visually impaired to propose solutions for improving positioning and guidance. First, we present a solution based on real-time fusion of A-GPS and embedded artificial vision positioning signals. The benefit of our device is two-fold : 1/ it provides a more accurate positioning, compatible with Blind mobility and guidance ; 2/ it matches the needs of Blind users in terms of space perception. Second, we define a classification of objects that should be included in every geographical information system (GIS) that is used in a navigation aid. This classification was based on multiple brainstorming and interviews with blind people and orientation and mobility (O&M) instructors. We present a database scheme integrating the principal classes proposed in this classification. We also propose a methodology allowing the selection of the most appropriate route, based on user needs, and relying on the proposed classification of geographical data. Finally, regarding pedestrian tracking, we propose 3 new strategies adapted to pedestrian navigation. The evaluation of those strategies was performed into a virtual environment framework. To do so, we designed a multimodal (input and output) Virtual Environment (VE) that simulates different interactions that could be used for space perception and guidance in an ETA. This platform subserves two goals : help designers to systematically test guidance strategies (i.e. for the development of new ETAs) and train blind people to use interactive ETAs, with an emphasis on cognitive mapping enhancement. Using this platform we performed several evaluations with 16 users to define the best tracking strategies. To conclude, the combined positioning (vision, GPS) was successfully evaluated in two real environments (Toulouse University campus, and a district in the Toulouse center). Results from evaluations of tracking strategies shown that it is very important to adapt such strategies to pedestrian navigation