Audio-visual-olfactory resource allocation for tri-modal virtual environments

© 2019 IEEE. Virtual Environments (VEs) provide the opportunity to simulate a wide range of applications, from training to entertainment, in a safe and controlled manner. For applications which require realistic representations of real world environments, the VEs need to provide multiple, physically accurate sensory stimuli. However, simulating all the senses that comprise the human sensory system (HSS) is a task that requires significant computational resources. Since it is intractable to deliver all senses at the highest quality, we propose a resource distribution scheme in order to achieve an optimal perceptual experience within the given computational budgets. This paper investigates resource balancing for multi-modal scenarios composed of aural, visual and olfactory stimuli. Three experimental studies were conducted. The first experiment identified perceptual boundaries for olfactory computation. In the second experiment, participants (N=25) were asked, across a fixed number of budgets (M=5), to identify what they perceived to be the best visual, acoustic and olfactory stimulus quality for a given computational budget. Results demonstrate that participants tend to prioritize visual quality compared to other sensory stimuli. However, as the budget size is increased, users prefer a balanced distribution of resources with an increased preference for having smell impulses in the VE. Based on the collected data, a quality prediction model is proposed and its accuracy is validated against previously unused budgets and an untested scenario in a third and final experiment

Psychometric Mapping of Audio Features to Perceived Physical Characteristics of Virtual Objects

Physically-based sound synthesis can simulate virtual sound sources whose audio features reflect the physical characteristics of corresponding objects displayed in a virtual environment, allowing for real-time generation of content without relying on pre-existing audio samples. This, however, requires efficient control strategies for sound synthesis models that, depending on the nature of the sounding objects, require to be mapped to varying physical characteristics displayed through visual information. In this experiment, participants were asked to adjust a set of sound synthesis parameters based on varying physical characteristics of a virtual bouncing ball: distance, elasticity and radius. Statistical analysis of recorded subject responses shows that object radius influences evaluation of pitch and amplitude for the object's representation. Similarly, distance influences user evaluation of both reverb and amplitude whilst elasticity doesn't influence user evaluation of the feature distributions. This result is consistent across user groups evaluated: audio experts and na\"ive listeners. Models are produced that encode these observations using linear regression, enabling automatic parameterisation of this feature space for audio synthesis engines

Recent advances in smellscape research for the built environment.

The interrelationships between humans, smells and the built environment have been the focus of increasing numbers of research studies in the past ten years. This paper reviews these trends and identifies the challenges in smellscape research from three aspects: methodological approaches, artistic design interventions and museum practices, and odour policy making. In response to the gaps and challenges identified, three areas of future research have also been identified for this field: smell archives and databases, social justice within odour control and management, and research into advanced building materials. [Abstract copyright: Copyright © 2021 Xiao, Aletta, Radicchi, McLean, Shiner and Verbeek.

Recent Advances in Smellscape Research for the Built Environment

The interrelationships between humans, smells and the built environment have been the focus of increasing numbers of research studies in the past ten years. This paper reviews these trends and identifies the challenges in smellscape research from three aspects: methodological approaches, artistic design interventions and museum practices, and odour policy making. In response to the gaps and challenges identified, three areas of future research have also been identified for this field: smell archives and databases, social justice within odour control and management, and research into advanced building materials

Exploring crossmodal correspondences for future research in human movement augmentation

"Crossmodal correspondences" are the consistent mappings between perceptual dimensions or stimuli from different sensory domains, which have been widely observed in the general population and investigated by experimental psychologists in recent years. At the same time, the emerging field of human movement augmentation (i.e., the enhancement of an individual's motor abilities by means of artificial devices) has been struggling with the question of how to relay supplementary information concerning the state of the artificial device and its interaction with the environment to the user, which may help the latter to control the device more effectively. To date, this challenge has not been explicitly addressed by capitalizing on our emerging knowledge concerning crossmodal correspondences, despite these being tightly related to multisensory integration. In this perspective paper, we introduce some of the latest research findings on the crossmodal correspondences and their potential role in human augmentation. We then consider three ways in which the former might impact the latter, and the feasibility of this process. First, crossmodal correspondences, given the documented effect on attentional processing, might facilitate the integration of device status information (e.g., concerning position) coming from different sensory modalities (e.g., haptic and visual), thus increasing their usefulness for motor control and embodiment. Second, by capitalizing on their widespread and seemingly spontaneous nature, crossmodal correspondences might be exploited to reduce the cognitive burden caused by additional sensory inputs and the time required for the human brain to adapt the representation of the body to the presence of the artificial device. Third, to accomplish the first two points, the benefits of crossmodal correspondences should be maintained even after sensory substitution, a strategy commonly used when implementing supplementary feedback

That Scent Evokes an Image—On the Impact of Olfactory Cues on User Image Recall in Digital Multisensory Environments

Copyright © 2023 The Author(s). In traditional digital filing systems, people mostly use text as a key to categorise images, and retrieve them in the future. The use of other media as keys for image retrieval is rarely used, notwithstanding that multisensory digital media – mulsemedia – can be harnessed to improve users’ performance and help them to retrieve their images. In this respect, olfactory media (engaging the sense of smell) is an example, as people can categorise their images by using congruent olfactory media. Accordingly, we investigated the impact of employing olfactory media as a key for retrieving a set of images. Moreover, we also studied the impact of the usage of olfactory media in this context on a user’s performance and Quality of Experience (QoE). To this end, we developed an olfactory-enhanced application (SCENT2IMAGE) in which olfactory media was emitted alongside a 5X5 matrix of images, of which users had to recognize 4 images congruent with the emitted scents. Furthermore, we developed a word-only version of the application (WORD2IMAGE) in which words alone were used as an equivalent key instead of olfactory media. Forty-four participants were invited and took part in our experiment, evenly split into a control and experimental group. Results highlight that using olfactory media does have a significant impact on user performance by helping them find related images. Moreover, using olfactory effects in this context was also found to enhance user QoE. Lastly, our findings underscore that users were willing to use olfactory-enhanced applications for categorizing/retrieving their albums and images...

Presence 2005: the eighth annual international workshop on presence, 21-23 September, 2005 University College London (Conference proceedings)

OVERVIEW (taken from the CALL FOR PAPERS) Academics and practitioners with an interest in the concept of (tele)presence are invited to submit their work for presentation at PRESENCE 2005 at University College London in London, England, September 21-23, 2005. The eighth in a series of highly successful international workshops, PRESENCE 2005 will provide an open discussion forum to share ideas regarding concepts and theories, measurement techniques, technology, and applications related to presence, the psychological state or subjective perception in which a person fails to accurately and completely acknowledge the role of technology in an experience, including the sense of 'being there' experienced by users of advanced media such as virtual reality. The concept of presence in virtual environments has been around for at least 15 years, and the earlier idea of telepresence at least since Minsky's seminal paper in 1980. Recently there has been a burst of funded research activity in this area for the first time with the European FET Presence Research initiative. What do we really know about presence and its determinants? How can presence be successfully delivered with today's technology? This conference invites papers that are based on empirical results from studies of presence and related issues and/or which contribute to the technology for the delivery of presence. Papers that make substantial advances in theoretical understanding of presence are also welcome. The interest is not solely in virtual environments but in mixed reality environments. Submissions will be reviewed more rigorously than in previous conferences. High quality papers are therefore sought which make substantial contributions to the field. Approximately 20 papers will be selected for two successive special issues for the journal Presence: Teleoperators and Virtual Environments. PRESENCE 2005 takes place in London and is hosted by University College London. The conference is organized by ISPR, the International Society for Presence Research and is supported by the European Commission's FET Presence Research Initiative through the Presencia and IST OMNIPRES projects and by University College London

The application of multiple modalities to improve home care and reminder systems

Existing home care technology tends to be pre-programmed systems limited to one or two interaction modalities. This can make them inaccessible to people with sensory impairments and unable to cope with a dynamic and heterogeneous environment such as the home. This thesis presents research that considers how home care technology can be improved through employing multiple visual, aural, tactile and even olfactory interaction methods. A wide range of modalities were tested to gather a better insight into their properties and merits. That information was used to design and construct Dyna-Cue, a prototype multimodal reminder system. Dyna-Cue was designed to use multiple modalities and to switch between them in real time to maintain higher levels of effectiveness and acceptability. The Dyna-Cue prototype was evaluated against other models of reminder delivery and was shown to be an effective and appropriate tool that can help people to manage their time and activities

Smells, well-being and the built environment

In this Research Topic, we aimed to collect a range of contributions to understand the emotional and wellbeing responses resulting from smells in different public spaces (museums, highstreets, heritage buildings, food districts, neighborhoods, squares, etc.) to inform future spatial design and management. The articles in this Research Topic are presented according to three types of contributions: reviews and conceptual analyses, empirical research in fieldwork, in laboratory studies and technological applications. Reviews and Conceptual Analyses Xiao et al. reviewed smellscape research studies conducted in the past 10 years to identify the challenges and related areas of future research, namely smell archives and databases, social justice within odor control and management, and research into advanced building materials. Spence reviewed the changing role of smells in the built environment from negative associations with sanitation to meaningful personal and cultural associations with memories and experiences which led to an evaluation of different approaches in examining the impact of smells on people's mood or wellbeing and the challenges of researching smells in the multi-sensory environment. Moving from the sick building sydrome to sick transport sydrome, Spence further reviewed the smells in transport environments as aesthetic and functional, and suggests challenges for future transportation to produce a more tangible vision to integrate smells in the design process to achieve the right balance of olfactory stimulation. Looking backwards to scented past, Bembibre and Strlič make the case for the need of knowledge exchange and interdisciplinary interpretation of findings in the field of olfactory heritage, providing an overview of methodological and museal studies as well as challenges associated with historical scent reconstruction. Empirical Research - Fieldwork Pálsdóttir et al. carried out a field study with participants suffering from stress-related mental disorders and explored how they would describe their smellscape perception of a garden in the context of a nature-based rehabilitation intervention. In a different field study, de Groot investigated whether ambient scents could affect customers' subjective experience and spending behavior in an experiment with customers of a second-hand clothing store. The author concluded that for that to happen, the smellscape should have a meaningful link to the physical context. Masaoka et al. present the results of a study conducted to examine whether continuous odor stimuli associated with autobiographical memories could activate olfactory areas in the brain of older adults and assess whether this odor stimulation could have a protective effect against age-related cognitive decline. Empirical Research - Laboratory Studies and Technological Applications Masaoka et al. investigated the potential protective effect from age-related cognitive decline of continuous odor stimuli associated with autobiographical memories and whether those could activate the above olfactory areas in older adults. Jiang et al. used blood pressure, pulse rate, EEG, POMS, and SD data to examine the odor-visual effects of the Primula forbesii Franch compared with the non-fragrant Primula malacoides Franch on the physiological and psychological state of Chinese female college students in the indoor environment. Courrèges et al. examined the correlations between odor and texture in users' perceptions of cosmetic creams cross-culturally, in laboratory conditions, using questionnaires, minimizing the impacts of branded messages from the packing and retail spaces. Amores et al. discussed the design and technical implementation of Essence- a smartphone-controlled wearable device that monitors users' EEG and real-time sleep staging algorithm to release scents to interact with users- in home-based sleep environments. The articles included in this Research Topic represent a nice balance between the theoretical reviews, empirical studies and laboratory research, showing the vibrance and dynamic in this research field as well as new technological developments such as extended reality, emotional sensors (i.e. EEG, GSR) and odor monitoring devices. New insights are drawn into the theoretical frameworks to understand relationships between smells, wellbeing and emotions, behaviors and physiological aspects; methodological approaches to measure smell triggered emotions, experiences, and quality of life; practical explorations on the process and challenges of using smells to influence user experiences in the built environment

Soundscape in Urban Forests

This Special Issue of Forests explores the role of soundscapes in urban forested areas. It is comprised of 11 papers involving soundscape studies conducted in urban forests from Asia and Africa. This collection contains six research fields: (1) the ecological patterns and processes of forest soundscapes; (2) the boundary effects and perceptual topology; (3) natural soundscapes and human health; (4) the experience of multi-sensory interactions; (5) environmental behavior and cognitive disposition; and (6) soundscape resource management in forests