Spatial representation and visual impairement - Developmental trends and new technological tools for assessment and rehabilitation

It is well known that perception is mediated by the five sensory modalities (sight, hearing, touch, smell and taste), which allows us to explore the world and build a coherent spatio-temporal representation of the surrounding environment. Typically, our brain collects and integrates coherent information from all the senses to build a reliable spatial representation of the world. In this sense, perception emerges from the individual activity of distinct sensory modalities, operating as separate modules, but rather from multisensory integration processes. The interaction occurs whenever inputs from the senses are coherent in time and space (Eimer, 2004). Therefore, spatial perception emerges from the contribution of unisensory and multisensory information, with a predominant role of visual information for space processing during the first years of life. Despite a growing body of research indicates that visual experience is essential to develop spatial abilities, to date very little is known about the mechanisms underpinning spatial development when the visual input is impoverished (low vision) or missing (blindness). The thesis's main aim is to increase knowledge about the impact of visual deprivation on spatial development and consolidation and to evaluate the effects of novel technological systems to quantitatively improve perceptual and cognitive spatial abilities in case of visual impairments. Chapter 1 summarizes the main research findings related to the role of vision and multisensory experience on spatial development. Overall, such findings indicate that visual experience facilitates the acquisition of allocentric spatial capabilities, namely perceiving space according to a perspective different from our body. Therefore, it might be stated that the sense of sight allows a more comprehensive representation of spatial information since it is based on environmental landmarks that are independent of body perspective. Chapter 2 presents original studies carried out by me as a Ph.D. student to investigate the developmental mechanisms underpinning spatial development and compare the spatial performance of individuals with affected and typical visual experience, respectively visually impaired and sighted. Overall, these studies suggest that vision facilitates the spatial representation of the environment by conveying the most reliable spatial reference, i.e., allocentric coordinates. However, when visual feedback is permanently or temporarily absent, as in the case of congenital blindness or blindfolded individuals, respectively, compensatory mechanisms might support the refinement of haptic and auditory spatial coding abilities. The studies presented in this chapter will validate novel experimental paradigms to assess the role of haptic and auditory experience on spatial representation based on external (i.e., allocentric) frames of reference. Chapter 3 describes the validation process of new technological systems based on unisensory and multisensory stimulation, designed to rehabilitate spatial capabilities in case of visual impairment. Overall, the technological validation of new devices will provide the opportunity to develop an interactive platform to rehabilitate spatial impairments following visual deprivation. Finally, Chapter 4 summarizes the findings reported in the previous Chapters, focusing the attention on the consequences of visual impairment on the developmental of unisensory and multisensory spatial experience in visually impaired children and adults compared to sighted peers. It also wants to highlight the potential role of novel experimental tools to validate the use to assess spatial competencies in response to unisensory and multisensory events and train residual sensory modalities under a multisensory rehabilitation

OPTIMALISASI INDERA PERABA DAN PENDENGARAN DALAM PENGUASAAN ONE-HANDED OVERARM THROW ANAK DENGAN GANGGUAN PENGLIHATAN

Anak tunanetra memiliki performa aktivitas fisik harian yang lebih rendah dibandingkan dengan sebaya mereka yang memiliki penglihatan, termasuk olahraga. Anak dengan gangguan penglihatan memiliki gerakan motorik yang tidak ideal karena indera penglihatan penting untuk keakuratan gerakan secara optimal. Faktor yang mempengaruhi derajat tidak idealnya gerakan adalah tingkat gangguan penglihatan, status kelahiran (cukup bulan atau prematur) dan latar belakang pengetahuan tentang kemampuan atau konsep gerakan. Penguasaan keterampilan motorik yang memadai, terutama keterampilan lokomotor dan motorik kasar, berkaitan dengan peningkatan tingkat aktivitas fisik selama prasekolah, anak, dan remaja. Penelitian ini dilakukan di SLB A YPAB Surabaya di mana dalam mengajarkan gerakan melempar, guru hanya memberikan instruksi secara lisan dan sesekali mengarahkan tangan ke arah bola untuk mengambil bola, namun tidak untuk menuntun anak melakukan gerakan tersebut. Hal ini kemungkinan disebabkan karena tidak adanya tenaga pendidik pelajaran kesehatan jasmani sehingga kurang memiliki pengetahuan tentang gerakan tersebut. Penelitian ini bertujuan untuk mengeksplorasi keefektifitasan 2 metode pelatihan, yaitu pelatihan menggunakan indera pendengaran saja dan menggunakan indera pendengaran dan peraba pada tingkat penguasaan gerakan one-handed overarm throw serta respon psikologis subjek yang nantinya dapat digunakan sebagai bahan dasar pertimbangan penyusunan metode pengajaran gerakan pada siswa sekolah dasar dengan gangguan penglihatan. Pada penelitian ini ditemukan adanya tingkat penguasaan yang lebih baik pada siswa yang diberikan pelatihan menggunakan indera pendengaran dan peraba dibandingkan dengan pendengaran saja. Secara psikologis, siswa dengan pelatihan menggunakan indera peraba dan pendengaran menunjukkan respon yang lebih positif

Multisensory Perception and Learning: Linking Pedagogy, Psychophysics, and Human–Computer Interaction

In this review, we discuss how specific sensory channels can mediate the learning of properties of the environment. In recent years, schools have increasingly been using multisensory technology for teaching. However, it still needs to be sufficiently grounded in neuroscientific and pedagogical evidence. Researchers have recently renewed understanding around the role of communication between sensory modalities during development. In the current review, we outline four principles that will aid technological development based on theoretical models of multisensory development and embodiment to foster in-depth, perceptual, and conceptual learning of mathematics. We also discuss how a multidisciplinary approach offers a unique contribution to development of new practical solutions for learning in school. Scientists, engineers, and pedagogical experts offer their interdisciplinary points of view on this topic. At the end of the review, we present our results, showing that one can use multiple sensory inputs and sensorimotor associations in multisensory technology to improve the discrimination of angles, but also possibly for educational purposes. Finally, we present an application, the ‘RobotAngle’ developed for primary (i.e., elementary) school children, which uses sounds and body movements to learn about angles

From sensory perception to spatial cognition

To interact with the environmet, it is crucial to have a clear space representation. Several findings have shown that the space around our body is split in several portions, which are differentially coded by the brain. Evidences of such subdivision have been reported by studies on people affected by neglect, on space near (peripersonal) and far (extrapersonal) to the body position and considering space around specific different portion of the body. Moreover, recent studies showed that sensory modalities are at the base of important cognitive skills. However, it is still unclear if each sensory modality has a different role in the development of cognitive skills in the several portions of space around the body. Recent works showed that the visual modality is crucial for the development of spatial representation. This idea is supported by studies on blind individuals showing that visual information is fundamental for the development of auditory spatial representation. For example, blind individuals are not able to perform the spatial bisection task, a task that requires to build an auditory spatial metric, a skill that sighted children acquire around 6 years of age. Based these prior researches, we hypothesize that if different sensory modalities have a role on the devlopment of different cognitive skills, then we should be able to find a clear correlation between availability of the sensory modality and the cognitive skill associated. In particular we hypothesize that the visual information is crucial for the development of auditory space represnetation; if this is true, we should find different spatial skill between front and back spaces. In this thesis, I provide evidences that spaces around our body are differently influenced by sensory modalities. Our results suggest that visual input have a pivotal role in the development of auditory spatial representation and that this applies only to the frontal space. Indeed sighted people are less accurated in spatial task only in space where vision is not present (i.e. the back), while blind people show no differences between front and back spaces. On the other hand, people tend to report sounds in the back space, suggesting that the role of hearing in allertness could be more important in the back than frontal spaces. Finally, we show that natural training, stressing the integration of audio motor stimuli, can restore spatial cognition, opening new possibility for rehabilitation programs. Spatial cognition is a well studied topic. However, we think our findings fill the gap regarding how the different availibility of sensory information, across spaces, causes the development of different cognitive skills in these spaces. This work is the starting point to understand the strategies that the brain adopts to maximize its resources by processing, in the more efficient way, as much information as possible

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD

Visual Impairment and Blindness

Blindness and vision impairment affect at least 2.2 billion people worldwide with most individuals having a preventable vision impairment. The majority of people with vision impairment are older than 50 years, however, vision loss can affect people of all ages. Reduced eyesight can have major and long-lasting effects on all aspects of life, including daily personal activities, interacting with the community, school and work opportunities, and the ability to access public services. This book provides an overview of the effects of blindness and visual impairment in the context of the most common causes of blindness in older adults as well as children, including retinal disorders, cataracts, glaucoma, and macular or corneal degeneration

A Haptic Study to Inclusively Aid Teaching and Learning in the Discipline of Design

Designers are known to use a blend of manual and virtual processes to produce design prototype solutions. For modern designers, computer-aided design (CAD) tools are an essential requirement to begin to develop design concept solutions. CAD, together with augmented reality (AR) systems have altered the face of design practice, as witnessed by the way a designer can now change a 3D concept shape, form, color, pattern, and texture of a product by the click of a button in minutes, rather than the classic approach to labor on a physical model in the studio for hours. However, often CAD can limit a designer’s experience of being ‘hands-on’ with materials and processes. The rise of machine haptic1 (MH) tools have afforded a great potential for designers to feel more ‘hands-on’ with the virtual modeling processes. Through the use of MH, product designers are able to control, virtually sculpt, and manipulate virtual 3D objects on-screen. Design practitioners are well placed to make use of haptics, to augment 3D concept creation which is traditionally a highly tactile process. For similar reasoning, it could also be said that, non-sighted and visually impaired (NS, VI) communities could also benefit from using MH tools to increase touch-based interactions, thereby creating better access for NS, VI designers. In spite of this the use of MH within the design industry (specifically product design), or for use by the non-sighted community is still in its infancy. Therefore the full benefit of haptics to aid non-sighted designers has not yet been fully realised. This thesis empirically investigates the use of multimodal MH as a step closer to improving the virtual hands-on process, for the benefit of NS, VI and fully sighted (FS) Designer-Makers. This thesis comprises four experiments, embedded within four case studies (CS1-4). Case study 1and2 worked with self-employed NS, VI Art Makers at Henshaws College for the Blind and Visual Impaired. The study examined the effects of haptics on NS, VI users, evaluations of experience. Case study 3 and4, featuring experiments 3 and4, have been designed to examine the effects of haptics on distance learning design students at the Open University. The empirical results from all four case studies showed that NS, VI users were able to navigate and perceive virtual objects via the force from the haptically rendered objects on-screen. Moreover, they were assisted by the whole multimodal MH assistance, which in CS2 appeared to offer better assistance to NS versus FS participants. In CS3 and 4 MH and multimodal assistance afforded equal assistance to NS, VI, and FS, but haptics were not as successful in bettering the time results recorded in manual (M) haptic conditions. However, the collision data between M and MH showed little statistical difference. The thesis showed that multimodal MH systems, specifically used in kinesthetic mode have enabled human (non-disabled and disabled) to credibly judge objects within the virtual realm. It also shows that multimodal augmented tooling can improve the interaction and afford better access to the graphical user interface for a wider body of users

A Framework to Account for the Effects of Visual Loss on Human Auditory Abilities

Until recently, a commonly held view was that blindness resulted in enhanced auditory abilities, underpinned by the beneficial effects of cross-modal neuroplasticity. This viewpoint has been challenged by studies showing that blindness results in poorer performance for some auditory spatial tasks. It is now clear that visual loss does not result in a general increase or decrease in all auditory abilities. Although several hypotheses have been proposed to explain why certain auditory abilities are enhanced while others are degraded, these are often limited to a specific subset of tasks. A comprehensive explanation encompassing auditory abilities assessed in fully blind and partially sighted populations and spanning spatial and non-spatial cognition has not so far been proposed. The current article proposes a framework comprising a set of nine principles that can be used to predict whether auditory abilities are enhanced or degraded. The validity of these principles is assessed by comparing their predictions with a wide range of empirical evidence concerning the effects of visual loss on spatial and non-spatial auditory abilities. Developmental findings and the effects of early- versus late-onset visual loss are discussed. Ways of improving auditory abilities for individuals with visual loss and reducing auditory spatial deficits are summarized. A new Perceptual Restructuring Hypothesis is proposed within the framework, positing that the auditory system is restructured to provide the most accurate information possible given the loss of the visual signal and utilizing available cortical resources, resulting in different auditory abilities getting better or worse according to the nine principles

Haptic Interaction with a Guide Robot in Zero Visibility

Search and rescue operations are often undertaken in dark and noisy environment in which rescue team must rely on haptic feedback for exploration and safe exit. However, little attention has been paid specifically to haptic sensitivity in such contexts or the possibility of enhancing communicational proficiency in the haptic mode as a life-preserving measure. The potential of root swarms for search and rescue has been shown by the Guardians project (EU, 2006-2010); however the project also showed the problem of human robot interaction in smoky (non-visibility) and noisy conditions. The REINS project (UK, 2011-2015) focused on human robot interaction in such conditions. This research is a body of work (done as a part of he REINS project) which investigates the haptic interaction of a person wit a guide robot in zero visibility. The thesis firstly reflects upon real world scenarios where people make use of the haptic sense to interact in zero visibility (such as interaction among firefighters and symbiotic relationship between visually impaired people and guide dogs). In addition, it reflects on the sensitivity and trainability of the haptic sense, to be used for the interaction. The thesis presents an analysis and evaluation of the design of a physical interface (Designed by the consortium of the REINS project) connecting the human and the robotic guide in poor visibility conditions. Finally, it lays a foundation for the design of test cases to evaluate human robot haptic interaction, taking into consideration the two aspects of the interaction, namely locomotion guidance and environmental exploration