The use of Virtual Reality in Enhancing Interdisciplinary Research and Education

Virtual Reality (VR) is increasingly being recognized for its educational potential and as an effective way to convey new knowledge to people, it supports interactive and collaborative activities. Affordable VR powered by mobile technologies is opening a new world of opportunities that can transform the ways in which we learn and engage with others. This paper reports our study regarding the application of VR in stimulating interdisciplinary communication. It investigates the promises of VR in interdisciplinary education and research. The main contributions of this study are (i) literature review of theories of learning underlying the justification of the use of VR systems in education, (ii) taxonomy of the various types and implementations of VR systems and their application in supporting education and research (iii) evaluation of educational applications of VR from a broad range of disciplines, (iv) investigation of how the learning process and learning outcomes are affected by VR systems, and (v) comparative analysis of VR and traditional methods of teaching in terms of quality of learning. This study seeks to inspire and inform interdisciplinary researchers and learners about the ways in which VR might support them and also VR software developers to push the limits of their craft.Comment: 6 Page

Digitally enhancing tasting experiences

Our experience of food and drink depends on more than merely the ingredients that have been combined in a dish or drink and the way in which it has been prepared. There is growing interest in the question of whether digital technologies/solutions can be used to enhance consumers’ multisensory tasting experiences. Projection mapping and sonic seasoning have been incorporated onto the menu in a number of world-leading restaurants for several years now. However, there has, as yet, been relatively little uptake of such digital technologies in the home environment. And while a wide (and growing) range of food and beverage brands have come out with sensory apps and sonic seasoning playlists in recent years, they do not tend to be present in the marketplace for very long. In fact, it would appear that the majority of augmented and virtual reality solutions have more relevance to the world of consumer/food science research than necessarily to enhancing the tasting experience for those in the home environment. Looking to the future, the widespread societal uptake of digitally-enhanced tasting experiences will likely only occur when such solutions provide an obvious, and demonstrable, benefit to the intended users

Thermal Sweet Taste Machine for Multisensory Internet

This paper presents a new taste interface for multisensory communication called "Thermal Sweet Taste Machine". We developed this interface in order to create sweet sensations, by manipulating the temperature on the tongue, without using chemicals. This device device changes the temperature on the surface of the tongue (from 20°C to 40°C) within a short period of time using a computer controlled circuit. Our preliminary user studies suggested that this device would be effective in two ways; producing the sweet sensations without the aid of chemicals, and enhancing the sweetness of the food and drinks. Here we discuss our concept, development of the interface, and some preliminary studies that has been carried out. We believe our technology would enhance the experiences and capabilities in future multisensory communication in different disciplines such as Human-Computer Interaction, human robot interactions, gaming and interacting with artificial agents

Thermal and wind devices for multisensory human-computer interaction: an overview

In order to create immersive experiences in virtual worlds, we need to explore different human senses (sight, hearing, smell, taste, and touch). Many different devices have been developed by both industry and academia towards this aim. In this paper, we focus our attention on the researched area of thermal and wind devices to deliver the sensations of heat and cold against people’s skin and their application to human-computer interaction (HCI). First, we present a review of devices and their features that were identified as relevant. Then, we highlight the users’ experience with thermal and wind devices, highlighting limitations either found or inferred by the authors and studies selected for this survey. Accordingly, from the current literature, we can infer that, in wind and temperature-based haptic systems (i) users experience wind effects produced by fans that move air molecules at room temperature, and (ii) there is no integration of thermal components to devices intended for the production of both cold or hot airflows. Subsequently, an analysis of why thermal wind devices have not been devised yet is undertaken, highlighting the challenges of creating such devices.Espírito Santo Research and Innovation Foundation (FAPES, Brazil) - Finance Code 2021-GL60J), the Coordination for the Improvement of Higher Education Personnel (CAPES, Brazil) - Finance Code 88881.187844/2018-01 and 88887.570688/2020-00 and by the National Council for Scientific and Technological (CNPq, Brazil) - Finance Code 307718/2020-4. The work was also funded by the European Union’s Horizon 2020 Research and Innovation programme under Grant Agreement no. 688503. E. B. Saleme additionally acknowledges aid from the Federal Institute of Espírito Santo

Inputs and outputs: engagement in digital media from the maker's perspective

In the process of developing a technology assembly that can objectively measure engagement on a moment-by-moment basis, subjective responses to stimuli must be shown to correlate with the component technologies, such as motion capture or psychophysiology. Subjective scales for engagement are not all consistent in segregating the measurement of causes (inputs to the audience) and effects (outputs from the audience); this lack of separation can obscure appropriate inferences in the relationship between cause and effect. Inputs to the audience are scripted, and are controllable by the maker. An output is what the designed experience engenders in the end-user, and outputs can include both mental states (satisfaction) and physical activities (heart rate) during the stimulus and subsequently. Inputs can be maximised by design, whereas to optimise outputs from the end-user, one needs an empirical process because outputs are dependent upon an interpretive process or entry into a biological system. Outputs will be highly dependent on audience and context, and they will often be quite variable, even in individuals from a similar audience profile. It is critical that, in instruments assessing the relationship between inputs and outputs, controllable inputs to the end-user must not be conflated with outputs engendered in the end-user

Digitizing the chemical senses: possibilities & pitfalls

Many people are understandably excited by the suggestion that the chemical senses can be digitized; be it to deliver ambient fragrances (e.g., in virtual reality or health-related applications), or else to transmit flavour experiences via the internet. However, to date, progress in this area has been surprisingly slow. Furthermore, the majority of the attempts at successful commercialization have failed, often in the face of consumer ambivalence over the perceived benefits/utility. In this review, with the focus squarely on the domain of Human-Computer Interaction (HCI), we summarize the state-of-the-art in the area. We highlight the key possibilities and pitfalls as far as stimulating the so-called ‘lower’ senses of taste, smell, and the trigeminal system are concerned. Ultimately, we suggest that mixed reality solutions are currently the most plausible as far as delivering (or rather modulating) flavour experiences digitally is concerned. The key problems with digital fragrance delivery are related to attention and attribution. People often fail to detect fragrances when they are concentrating on something else; And even when they detect that their chemical senses have been stimulated, there is always a danger that they attribute their experience (e.g., pleasure) to one of the other senses – this is what we call ‘the fundamental attribution error’. We conclude with an outlook on digitizing the chemical senses and summarize a set of open-ended questions that the HCI community has to address in future explorations of smell and taste as interaction modalities

Exploration of mid-air haptics experience design

Ultrasonic Mid-air Haptics (UMH) is a novel technology that uses the mechanical properties of sound waves to create a pressure point in mid-air. This pressure point, called focal point, can slightly bend the skin and be felt in mid-air without any attachment to the body. This thesis focuses on both studying how to integrate this technology with other senses (i.e. vision and audition) and exploring the range of tactile sensations it can provide. The first two projects presented in this document present the integration of ultrasonic mid-air haptics with audio-visual content. The first project describes the process of creating a unique haptic experience that was part of a six-weeks multisensory exhibition in a museum. The second project moved from the museum to a controlled environment and explored the creation of haptic experiences based on physiologic measurements for six short films. Both studies showed the positive value of adding ultrasonic mid-air haptics to traditional media through higher reported arousal and participants’ high enthusiasm for multisensory content. In the two latter projects of this thesis, it was explored how we could extend the range of possible tactile sensations provided by UMHs. We introduced a new technique called Spatio-Temporal Modulation (STM). It enabled the creation of brand-new tactile experiences, including more salient shapes and wider range of textures. We also provided some guidelines on how to control some of the tactile properties of the sensation, including strength,roughness,or regularity. The findings of those four projects contribute to the growing body of knowledge of UMHs. A summary of the key contributions is provided at the end of the thesis as well as several leads for future works

Sound Aesthetic: A Form of Narrative

This research presents an exploration into a novel design methodology that incorporates architecture, multimedia, and interactive digital technologies to create an immersive experience that encourages a spatial and sensorial discourse between user and their built environment. This immersive design method creates a continuous narrative that allows a multi-directional interaction between the two. This interaction creates a “sound” architectural aesthetic that changes the experience of space. The target of the interaction between user and space is the five human senses resulting in an immersive aesthetic. In order to illustrate this immersive aesthetic, five architectural prototypes were created using an assorted design workflow of parametric programming environment and interactive prototyping platform. This workflow is employed for the creation of five prototypes used for the simulation that has user interaction as an input and formal geometries as an output. These five prototypes target various human senses in order to enhance the immersive aesthetic. Each protoype is evaluated according to individual prototype’s ability to stimulate user’s senses. Finally, future research based on the outcomes of this research is suggested

Multimodality in VR: A survey

Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR, and its role and benefits in user experience, together with different applications that leverage multimodality in many disciplines. These works thus encompass several fields of research, and demonstrate that multimodality plays a fundamental role in VR; enhancing the experience, improving overall performance, and yielding unprecedented abilities in skill and knowledge transfer

Telepresence and the Role of the Senses

The telepresence experience can be evoked in a number of ways. A well-known example is a player of videogames who reports about a telepresence experience, a subjective experience of being in one place or environment, even when physically situated in another place. In this paper we set the phenomenon of telepresence into a theoretical framework. As people react subjectively to stimuli from telepresence, empirical studies can give more evidence about the phenomenon. Thus, our contribution is to bridge the theoretical with the empirical. We discuss theories of perception with an emphasis on Heidegger, Merleau-Ponty and Gibson, the role of the senses and the Spinozian belief procedure. The aim is to contribute to our understanding of this phenomenon. A telepresence-study that included the affordance concept is used to empirically study how players report sense-reactions to virtual sightseeing in two cities. We investigate and explore the interplay of the philosophical and the empirical. The findings indicate that it is not only the visual sense that plays a role in this experience, but all senses