Using sound localization to gain depth perception for the visually impaired through sensory substitution

Abstract

The visually impaired do not have the visual ability to localize objects in three-dimensional space, rather, they rely on their other senses to gain depth perception. Sensory substitution is the concept of substituting one sense for another, normally substituting an impaired sense with a functioning sense. Visual-to-auditory sensory substitution substitutes an impaired visual sense with a functioning auditory sense. This research aimed to investigate and develop techniques for visualto-auditory sensory substitution – using sound localization as a sensory substitution for depth perception. The research started by investigating the characteristics of human audition with a particular focus on how humans localize sounds. It then looked at existing visual-to-auditory sensory substitution systems and the techniques they used. From the existing systems, a system known as MeloSee was chosen as a baseline for developing and evaluating further sensory substitution prototypes. The baseline prototype (0) was then implemented and a preliminary study performed. Based on the knowledge gained from the preliminary study, baseline implementation and the background research, a set of improvement recommendations were generated. The next iteration – Prototype 1 (1) – was then developed based on the recommendations. A comparative study between 0 and 1 was then performed. Based on the study, another set of improvement recommendations were generated. From the recommendations, a final prototype was developed – Prototype 2 (2). The last comparative study was then performed between 0 and 2, with a third set of recommendations being generated as a result. From the studies it was found that participants using 0 were able to identify when they were approaching large objects such as walls. 1 built on that, improving the ability to identify the quadrant of a nearby isolated object. 2 built on 0 and 1, achieving similar results to 1 for identifying the quadrant of nearby isolated objects, and improving on 0 and 1 with regard to depth discrimination – especially for navigation tasks where there were no obstacles. Based on the three sets of recommendations and what was learnt over the course of the research, a set of visual-to-auditory sensory substitution techniques were presented. The techniques aim to be useful for implementing visual-to-auditory sensory substitution systems, which would provide the visually impaired with the visual ability to localize objects in three-dimensional space through sound