The Sound of the Smell (and taste) of my Shoes too: Mapping the Senses using Emotion as a Medium

This work discusses basic human senses: sight; sound; touch; taste; and smell; and the way in which it may be possible to compensate for lack of one, or more, of these by explicitly representing stimuli using the remaining senses. There may be many situations or scenarios where not all five of these base senses are being stimulated, either because of an optional restriction or deficit or because of a physical or sensory impairment such as loss of sight or touch sensation. Related to this there are other scenarios where sensory matching problems may occur. For example: a user immersed in a virtual environment may have a sense of smell from the real world that is unconnected to the virtual world. In particular, this paper is concerned with how sound can be used to compensate for the lack of other sensory stimulation and vice-versa. As a link is well established already between the visual, touch, and auditory systems, more attention is given to taste and smell, and their relationship with sound. This work presents theoretical concepts, largely oriented around mapping other sensory qualities to sound, based upon existing work in the literature and emerging technologies, to discuss where particular gaps currently exist, how emotion could be a medium to cross-modal representations, and how these might be addressed in future research. It is postulated that descriptive qualities, such as timbre or emotion, are currently the most viable routes for further study and that this may be later integrated with the wider body of research into sensory augmentation

The Sound of the Smell (and Taste) of My Shoes Too: Mapping the Senses Using Emotion as a Medium

This work discusses basic human senses: sight; sound; touch; taste; and smell; and the way in which it may be possible to compensate for lack of one, or more, of these by explicitly representing stimuli using the remaining senses. There may be many situations or scenarios where not all five of these base senses are being stimulated, either because of an optional restriction or deficit or because of a physical or sensory impairment such as loss of sight or touch sensation. Related to this there are other scenarios where sensory matching problems may occur. For example: a user immersed in a virtual environment may have a sense of smell from the real world that is unconnected to the virtual world. In particular, this paper is concerned with how sound can be used to compensate for the lack of other sensory stimulation and vice-versa. As a link is well established already between the visual, touch, and auditory systems, more attention is given to taste and smell, and their relationship with sound. This work presents theoretical concepts, largely oriented around mapping other sensory qualities to sound, based upon existing work in the literature and emerging technologies, to discuss where particular gaps currently exist, how emotion could be a medium to cross-modal representations, and how these might be addressed in future research. It is postulated that descriptive qualities, such as timbre or emotion, are currently the most viable routes for further study and that this may be later integrated with the wider body of research into sensory augmentation

Olfaction-enhanced multimedia: Perspectives and challenges

This is the post-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2011 Springer VerlagOlfaction—or smell—is one of the last challenges which multimedia and multimodal applications have to conquer. Enhancing such applications with olfactory stimuli has the potential to create a more complex—and richer—user multimedia experience, by heightening the sense of reality and diversifying user interaction modalities. Nonetheless, olfaction-enhanced multimedia still remains a challenging research area. More recently, however, there have been initial signs of olfactory-enhanced applications in multimedia, with olfaction being used towards a variety of goals, including notification alerts, enhancing the sense of reality in immersive applications, and branding, to name but a few. However, as the goal of a multimedia application is to inform and/or entertain users, achieving quality olfaction-enhanced multimedia applications from the users’ perspective is vital to the success and continuity of these applications. Accordingly, in this paper we have focused on investigating the user perceived experience of olfaction-enhanced multimedia applications, with the aim of discovering the quality evaluation factors that are important from a user’s perspective of these applications, and consequently ensure the continued advancement and success of olfaction-enhanced multimedia applications

Branding sensorial : uma introdução

Monografia (graduação)—Universidade de Brasília, Faculdade de Comunicação, Departamento de Audiovisual e Publicidade, Memorial de Projeto Final em Publicidade, 2013.O trabalho tem a intenção de dar mais um passo na construção de novos caminhos do branding sensorial, sugerindo estratégias de construção de marca a partir de um uso mais holístico dos cinco sentidos. Em uma sociedade extremamente focada no visual e com uma grande saturação de informação, enxerga-se na audição, tato, paladar e olfato, uma possibilidade de uma marca se destacar em relação à concorrência, além de uma oportunidade de oferecer uma melhor experiência de marca ao consumidor. Em formato de livro introdutório, o projeto sugere sistemas de significação nos cinco sentidos, assumindo que estímulos de qualquer natureza podem ser significantes para a construção de um significado de marca. Fornecendo uma breve base teórica sobre cada um dos sentidos, o projeto busca, ainda, incentivar os leitores na construção de caminhos não expostos no livro

The Effects of Computerized Smell of Memory

It has long been recognized that there is a major correlation between smell and memory. Until recently, commercialized multi-sensory experiences involving olfaction were limited to non-computerized mediums. Companies that manufacture computerized scent technologies tout the educational benefits of their product, yet prior to this study, there appeared to be no scholarly research in regard to the efficacy of computerized scent producing peripherals in educational environments. The aim of this research was to determine the odor memory enhancement benefits of incorporating olfactory, computerized peripherals into computerized multimedia-learning environments, from both a context dependent and context independent stand point. Specifically, within a multimedia environment, the goal of this study was to ascertain whether or not there would be a significant memory performance difference between subjects who were exposed to scents at both encoding and recall, over subjects who were exposed to scents at encoding only. There were 61 subjects tested in a carefully designed and controlled experiment. Subjects were 6th, 7th, and 8th grade students from a local private school. Subjects were randomly assigned to one of three conditions: Administration of a multimedia presentation with computerized smell during the presentation but not during post-testing, administration of a multimedia presentation with computerized smell present during both the presentation and post-testing, or a control group that watched the multimedia presentation without smell and post-tested without smell. Subjects were pre-tested several weeks prior to commencement of the study and then given a post-test approximately 48 hours after viewing the presentation. It was hypothesized that subjects in both experimental conditions would demonstrate an improvement in memory over the control group based on previous studies regarding odor memory. Although there was significant improvement within groups from pre-test to post-test, there was no significant difference found between groups. Based on these results, it would appear that in regard to this study, adding computer-generated scents to multimedia environments provided no measurable value as far as memory is concerned There are a number of issues of which future studies in the area of computerized olfaction and memory should be mindful. These include the level of immersion, the duration of the presentation, the duration of aromas, the level of subject interactivity, the age of the subjects, the scent delivery method, the type of scent technology used, and the types of questions asked of subjects

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences