Using Sonic Enhancement to Augment Non-Visual Tabular Navigation

More information is now readily available to computer users than at any time in human history; however, much of this information is often inaccessible to people with blindness or low-vision, for whom information must be presented non-visually. Currently, screen readers are able to verbalize on-screen text using text-to-speech (TTS) synthesis; however, much of this vocalization is inadequate for browsing the Internet. An auditory interface that incorporates auditory-spatial orientation was created and tested. For information that can be structured as a two-dimensional table, links can be semantically grouped as cells in a row within an auditory table, which provides a consistent structure for auditory navigation. An auditory display prototype was tested. Sixteen legally blind subjects participated in this research study. Results demonstrated that stereo panning was an effective technique for audio-spatially orienting non-visual navigation in a five-row, six-column HTML table as compared to a centered, stationary synthesized voice. These results were based on measuring the time- to-target (TTT), or the amount of time elapsed from the first prompting to the selection of each tabular link. Preliminary analysis of the TTT values recorded during the experiment showed that the populations did not conform to the ANOVA requirements of normality and equality of variances. Therefore, the data were transformed using the natural logarithm. The repeated-measures two-factor ANOVA results show that the logarithmically-transformed TTTs were significantly affected by the tonal variation method, F(1,15) = 6.194, p= 0.025. Similarly, the results show that the logarithmically transformed TTTs were marginally affected by the stereo spatialization method, F(1,15) = 4.240, p=0.057. The results show that the logarithmically transformed TTTs were not significantly affected by the interaction of both methods, F(1,15) = 1.381, p=0.258. These results suggest that some confusion may be caused in the subject when employing both of these methods simultaneously. The significant effect of tonal variation indicates that the effect is actually increasing the average TTT. In other words, the presence of preceding tones increases task completion time on average. The marginally-significant effect of stereo spatialization decreases the average log(TTT) from 2.405 to 2.264

The Design of Audio Mixing Software Displays to Support Critical Listening

PhDThe mixing desk metaphor found in Digital Audio Workstations (DAW) is built upon a specialised and technical knowledge of signal flow and audio engineering. However, since their inception the DAW has gained a far wider and less technically specialised user-base. Furthermore, the limited screen space of laptop and tablet computers, combined with potentially limitless tracks in current DAWs has resulted in the need for complex interface navigation during mixing which may inhibit a fluid and intuitive approach to mixing. The research outlined in this thesis explores novel designs for Graphical User Interfaces (GUIs) for mixing, which acknowledge the changing role of the user, the limited space of tablet and mobile computers screens and the limitations of human perception during cross modal activities (aural and visual). The author designs and conducts several experiments using non-expert participants drawn from several music technology courses, to assess and quantify the extent to which current DAW designs might influence mixing workflow, aiming our research especially at beginner and non-expert users. The results of our studies suggest that GUIs which load visual working memory, or force the user to mentally integrate visual information across the interface, can reduce the ability to hear subtle simultaneous changes to the audio. We use the analysis of these experiments to propose novel GUI designs that are better suited to human cross-modal perceptual limitations and which take into account the specific challenges and opportunities afforded by screen-based audio mixers. By so doing, we aim to support the user in achieving a more fluid and focused interaction while mixing, where the visual feedback supports and enhances the primary goal of attending to and modifying the audio content of the mix. In turn, it is hoped this will facilitate the artistic and creative approaches required by music computer users

Mobile Auditory Guidance for Public Transportation

Helsingin seudulla on vapaasti saatavilla kattavaa tietoa julkisesta liikenteestä. Tietoa välitetään käyttäjille monien eri sovellusten avulla. Yleensä nämä sovellukset toimittavat tiedon pelkästään visuaalisessa muodossa, jättäen äänen potentiaalin (lisä)tiedon välityksessä täysin vaille huomiota. Graafinen ratkaisu voi olla epäkäytännöllinen ja tiedon saanti vaatia paljon keskittymistä, kun kyseessä on mobiililaite ja informaatio tarvitaan liikenteessä. Tässä työssä suunniteltiin ja kehitettiin täysin toiminnallinen mobiilisovellus reittisuunnittelua ja -opastusta varten. Sovelluksen tavoitteena on minimoida visuaalinen huomiointi ja tarkkailu, toimittaen äänen avulla tietoa perustuen aikatauluihin, kellonaikaan sekä käyttäjän sijaintiin. Päämääränä on tarjota informaatio merkityksellisessä ja helposti ymmärrettävässä muodossa, hyödyntäen muuta ääntä kuin puhetta, vapauttaen käyttäjän silmät ja kädet sekä poistaen tarpeen seurata kellonaikaa. Käytettyjen merkkiäänten tunnistettavuutta ja arvioita niiden ärsyttävyydestä testattiin epämuodollisesti pienellä määrällä koehenkilöitä. Sovelluksen tarkoitus ja äänten metaforat selitettiin lyhyesti, jonka jälkeen suurin osa käytetyistä auditiivisista symboleista tunnistettiin hyvin. Ylipäänsä mitään ääniä hyödyntävän sovelluksen käyttö julkisissa paikoissa ilman kuulokkeita herätti joissakin testihenkilöissä epäilyksiä. Kaiken kaikkiaan sovelluksessa todettiin olevan selviä mahdollisuuksia hyödylliseksi apuvälineeksi ja useimmat koekuuntelijoista havaitsivat ominaisuuksia, joiden he kokivat olevan itselleen käytännöllisiä.Comprehensive information for using public transportation in the Helsinki region is freely available and there are different applications that provide this. These applications, however, typically only provide information in the visual modality, completely neglecting the potential of using audio for conveying (additional) messages. In the context of mobile devices and requiring information while on the move, this can be cumbersome and requires the user to pay a lot of attention to the application in order to get the information. A completely functional mobile application for journey planning and guidance for travelling along a found route was designed and developed in this thesis. The aim for the application is to reduce the attention requirements to a minimum by using sound to deliver useful information to the user based on schedules, time, and the user's location. The objective is to provide the information by meaningful and easily understandable non-speech auditory cues to free the user's eyes and hands while, at the same time, removing the need to pay attention to the time. The recognizability of the used sounds, and estimation of whether they would be useful or annoying, was tested informally by a small number of people. After a brief explanation of the purpose and metaphors of the sounds in the application, most auditory icons were recognized well. Using an application with any sort of sound in public without headphones raised doubts in some. Overall the application was found to have potential for being a helpful aid and most of the test subjects saw elements that would be useful for themselves

Attention-controlled acquisition of a qualitative scene model for mobile robots

Haasch A. Attention-controlled acquisition of a qualitative scene model for mobile robots. Bielefeld (Germany): Bielefeld University; 2007.Robots that are used to support humans in dangerous environments, e.g., in manufacture facilities, are established for decades. Now, a new generation of service robots is focus of current research and about to be introduced. These intelligent service robots are intended to support humans in everyday life. To achieve a most comfortable human-robot interaction with non-expert users it is, thus, imperative for the acceptance of such robots to provide interaction interfaces that we humans are accustomed to in comparison to human-human communication. Consequently, intuitive modalities like gestures or spontaneous speech are needed to teach the robot previously unknown objects and locations. Then, the robot can be entrusted with tasks like fetch-and-carry orders even without an extensive training of the user. In this context, this dissertation introduces the multimodal Object Attention System which offers a flexible integration of common interaction modalities in combination with state-of-the-art image and speech processing techniques from other research projects. To prove the feasibility of the approach the presented Object Attention System has successfully been integrated in different robotic hardware. In particular, the mobile robot BIRON and the anthropomorphic robot BARTHOC of the Applied Computer Science Group at Bielefeld University. Concluding, the aim of this work, to acquire a qualitative Scene Model by a modular component offering object attention mechanisms, has been successfully achieved as demonstrated on numerous occasions like reviews for the EU-integrated Project COGNIRON or demos

Subjective evaluation and electroacoustic theoretical validation of a new approach to audio upmixing

Audio signal processing systems for converting two-channel (stereo) recordings to four or five channels are increasingly relevant. These audio upmixers can be used with conventional stereo sound recordings and reproduced with multichannel home theatre or automotive loudspeaker audio systems to create a more engaging and natural-sounding listening experience. This dissertation discusses existing approaches to audio upmixing for recordings of musical performances and presents specific design criteria for a system to enhance spatial sound quality. A new upmixing system is proposed and evaluated according to these criteria and a theoretical model for its behavior is validated using empirical measurements.The new system removes short-term correlated components from two electronic audio signals using a pair of adaptive filters, updated according to a frequency domain implementation of the normalized-least-means-square algorithm. The major difference of the new system with all extant audio upmixers is that unsupervised time-alignment of the input signals (typically, by up to +/-10 ms) as a function of frequency (typically, using a 1024-band equalizer) is accomplished due to the non-minimum phase adaptive filter. Two new signals are created from the weighted difference of the inputs, and are then radiated with two loudspeakers behind the listener. According to the consensus in the literature on the effect of interaural correlation on auditory image formation, the self-orthogonalizing properties of the algorithm ensure minimal distortion of the frontal source imagery and natural-sounding, enveloping reverberance (ambiance) imagery.Performance evaluation of the new upmix system was accomplished in two ways: Firstly, using empirical electroacoustic measurements which validate a theoretical model of the system; and secondly, with formal listening tests which investigated auditory spatial imagery with a graphical mapping tool and a preference experiment. Both electroacoustic and subjective methods investigated system performance with a variety of test stimuli for solo musical performances reproduced using a loudspeaker in an orchestral concert-hall and recorded using different microphone techniques.The objective and subjective evaluations combined with a comparative study with two commercial systems demonstrate that the proposed system provides a new, computationally practical, high sound quality solution to upmixing

Sonic interactions in virtual environments

This book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments

Ways of Guided Listening: Embodied approaches to the design of interactive sonifications

This thesis presents three use cases for interactive feedback. In each case users interact with a system and receive feedback: the primary source of feedback is visual, while a second source of feedback is offered as sonification. The first use case comprised an interactive sonification system for use by pathologists in the triage stage of cancer diagnostics. Image features derived from computational homology are mapped to a soundscape with integrated auditory glance indicating potential regions of interests. The resulting prototype did not meet the requirements of a domain expert. In the second case this thesis presents an interactive sonification plug-in developed for a software package for interactive visualisation of macromolecular complexes. A framework for building different sonification methods in Python and an OSC-controlled sound producing software was established along with sonification methods and a general sonification plugin. It received generally positive feedback, but the mapping was deemed not very transparent. From these cases and ideas in sonification design literature, the Subject-Position-Based Sonification Design Framework (SPBDF) was developed. It explores an alternative conception of design: that working from a frame of reference encompassing a non-expert audience will lead towards sonifications that are more easily understood. A method for the analysis of sonifications according to its criteria is outlined and put into practice to evaluate a range of sonifications. This framework was evaluated in the third use case, a system for sonified feedback for an exercise device designed for back pain rehabilitation. Two different sonifications, one using SPBDF as basis of their design, were evaluated, indicating that interactive sonification can provide valuable feedback and improve task performance (decrease the mean speed) when the soundscape employed invokes an appropriate emotional response in the user

Sonic Interactions in Virtual Environments

This open access book tackles the design of 3D spatial interactions in an audio-centered and audio-first perspective, providing the fundamental notions related to the creation and evaluation of immersive sonic experiences. The key elements that enhance the sensation of place in a virtual environment (VE) are: Immersive audio: the computational aspects of the acoustical-space properties of Virutal Reality (VR) technologies Sonic interaction: the human-computer interplay through auditory feedback in VE VR systems: naturally support multimodal integration, impacting different application domains Sonic Interactions in Virtual Environments will feature state-of-the-art research on real-time auralization, sonic interaction design in VR, quality of the experience in multimodal scenarios, and applications. Contributors and editors include interdisciplinary experts from the fields of computer science, engineering, acoustics, psychology, design, humanities, and beyond. Their mission is to shape an emerging new field of study at the intersection of sonic interaction design and immersive media, embracing an archipelago of existing research spread in different audio communities and to increase among the VR communities, researchers, and practitioners, the awareness of the importance of sonic elements when designing immersive environments