Digitizing the chemical senses: possibilities & pitfalls

Many people are understandably excited by the suggestion that the chemical senses can be digitized; be it to deliver ambient fragrances (e.g., in virtual reality or health-related applications), or else to transmit flavour experiences via the internet. However, to date, progress in this area has been surprisingly slow. Furthermore, the majority of the attempts at successful commercialization have failed, often in the face of consumer ambivalence over the perceived benefits/utility. In this review, with the focus squarely on the domain of Human-Computer Interaction (HCI), we summarize the state-of-the-art in the area. We highlight the key possibilities and pitfalls as far as stimulating the so-called ‘lower’ senses of taste, smell, and the trigeminal system are concerned. Ultimately, we suggest that mixed reality solutions are currently the most plausible as far as delivering (or rather modulating) flavour experiences digitally is concerned. The key problems with digital fragrance delivery are related to attention and attribution. People often fail to detect fragrances when they are concentrating on something else; And even when they detect that their chemical senses have been stimulated, there is always a danger that they attribute their experience (e.g., pleasure) to one of the other senses – this is what we call ‘the fundamental attribution error’. We conclude with an outlook on digitizing the chemical senses and summarize a set of open-ended questions that the HCI community has to address in future explorations of smell and taste as interaction modalities

Smell, Taste, and Temperature Interfaces

Everyday life hinges on smell, taste, and temperature-based experiences, from eating to detecting potential hazards (e.g., smell of rotten food, microbial threats, and non-microbial threats such as from hazardous gases) to responding to thermal behavioral changes. These experiences are formative as visceral, vital signals of information, and contribute directly to our safety, well-being, and enjoyment. Despite this, contemporary technology mostly stimulates vision, audition, and - more recently - touch, unfortunately leaving out the senses of smell taste and temperature. In the last decade, smell, taste, and temperature interfaces have gained a renewed attention in the field of Human Computer Interaction, fueled by the growth of virtual reality and wearable devices. As these modalities are further explored, it is imperative to discuss underlying cultural contexts of these experiences, how researchers can robustly stimulate and sense these modalities, and in what contexts such multisensory technologies are meaningful. This workshop addresses these topics and seeks to provoke critical discussions around chemo- and thermo-sensory HCI

Intranasal Chemosensory Lateralization Through the Multi-electrode Transcutaneous Electrical Nasal Bridge Stimulation

Numerous studies have been conducted on display techniques for intranasal chemosensory perception. However, a limited number of studies have focused on the presentation of sensory spatial information. To artificially produce intranasal chemosensory spatial perception, we focused on a technique to induce intranasal chemosensation by transcutaneous electrical stimulation between the nasal bridge and the back of the neck. Whether this technique stimulates the trigeminal nerve or the olfactory nerve remains debatable; if this method stimulates the trigeminal nerve, the differences in the amount of stimulation to the left and right trigeminal branches would evoke lateralization of intranasal chemosensory perception. Therefore, we propose a novel method to lateralize intranasal chemosensation by selectively stimulating the left or right trigeminal nerve branches through the shifting of an electrode on the nasal bridge to the left or right. Finite element simulations reveal that electrical stimulation applied between the electrodes on the left/right nasal bridge and the back of the neck results in the construction of a high current density area on the left/right branch of the trigeminal nerve. The results of two psychophysical experiments reveal that intranasal chemosensation can be lateralized by using the proposed method. The results of our experiment also suggest that lateralization is not the result of electrically induced tactile sensation of the skin surface but rather due to the distribution of stimuli to the trigeminal nerves. To the best of our knowledge, this study is the first successful lateralization of intranasal chemosensation that utilizes an easy-to-apply method without involving nostril blocking

Odor-Based Molecular Communications: State-of-the-Art, Vision, Challenges, and Frontier Directions

Humankind mimics the processes and strategies that nature has perfected and uses them as a model to address its problems. That has recently found a new direction, i.e., a novel communication technology called molecular communication (MC), using molecules to encode, transmit, and receive information. Despite extensive research, an innate MC method with plenty of natural instances, i.e., olfactory or odor communication, has not yet been studied with the tools of information and communication technologies (ICT). Existing studies focus on digitizing this sense and developing actuators without inspecting the principles of odor-based information coding and MC, which significantly limits its application potential. Hence, there is a need to focus cross-disciplinary research efforts to reveal the fundamentals of this unconventional communication modality from an ICT perspective. The ways of natural odor MC in nature need to be anatomized and engineered for end-to-end communication among humans and human-made things to enable several multi-sense augmented reality technologies reinforced with olfactory senses for novel applications and solutions in the Internet of Everything (IoE). This paper introduces the concept of odor-based molecular communication (OMC) and provides a comprehensive examination of olfactory systems. It explores odor communication in nature, including aspects of odor information, channels, reception, spatial perception, and cognitive functions. Additionally, a comprehensive comparison of various communication systems sets the foundation for further investigation. By highlighting the unique characteristics, advantages, and potential applications of OMC through this comparative analysis, the paper lays the groundwork for exploring the modeling of an end-to-end OMC channel, considering the design of OMC transmitters and receivers, and developing innovative OMC techniques

Information Olfactation: Theory, Design, and Evaluation

Olfactory feedback for analytical tasks is a virtually unexplored area in spite of the advantages it offers for information recall, feature identification, and location detection. Here we introduce the concept of ‘Information Olfactation’ as the fragrant sibling of information visualization, and discuss how scent can be used to convey data. Building on a review of the human olfactory system and mirroring common visualization practice, we propose olfactory marks, the substrate in which they exist, and their olfactory channels that are available to designers. To exemplify this idea, we present ‘viScent(1.0)’: a six-scent stereo olfactory display capable of conveying olfactory glyphs of varying temperature and direction, as well as a corresponding software system that integrates the display with a traditional visualization display. We also conduct a comprehensive perceptual experiment on Information Olfactation: the use of olfactory marks and channels to convey data. More specifically, following the example from graphical perception studies, we design an experiment that studies the perceptual accuracy of four ``olfactory channels''---scent type, scent intensity, airflow, and temperature---for conveying three different types of data---nominal, ordinal, and quantitative. We also present details of an advanced 24-scent olfactory display: ‘viScent(2.0)’ and its software framework that we designed in order to run this experiment. Our results yield a ranking of olfactory channels for each data type that follows similar principles as rankings for visual channels, such as those derived by Mackinlay, Cleveland & McGill, and Bertin

Earth as Interface: Exploring chemical senses with Multisensory HCI Design for Environmental Health Communication

As environmental problems intensify, the chemical senses -that is smell and taste, are the most relevantsenses to evidence them.As such, environmental exposure vectors that can reach human beings comprise air,food, soil and water[1].Within this context, understanding the link between environmental exposures andhealth[2]is crucial to make informed choices, protect the environment and adapt to new environmentalconditions[3].Smell and taste lead therefore to multi-sensorial experiences which convey multi-layered information aboutlocal and global events[4]. However, these senses are usually absent when those problems are represented indigital systems. The multisensory HCIdesign framework investigateschemical sense inclusion withdigital systems[5]. Ongoing efforts tackledigitalization of smell and taste for digital delivery, transmission or substitution [6]. Despite experimentsproved technological feasibility, its dissemination depends on relevant applicationdevelopment[7].This thesis aims to fillthose gaps by demonstratinghow chemical senses provide the means to link environment and health based on scientific andgeolocation narratives [8], [9],[10]. We present a Multisensory HCI design process which accomplished symbolicdisplaying smell and taste and led us to a new multi-sensorial interaction system presented herein. We describe the conceptualization, design and evaluation of Earthsensum, an exploratory case study project.Earthsensumoffered to 16 participants in the study, environmental smell and taste experiences about real geolocations to participants of the study. These experiences were represented digitally using mobilevirtual reality (MVR) and mobile augmented reality (MAR). Its technologies bridge the real and digital Worlds through digital representations where we can reproduce the multi-sensorial experiences. Our study findings showed that the purposed interaction system is intuitive and can lead not only to a betterunderstanding of smell and taste perception as also of environmental problems. Participants comprehensionabout the link between environmental exposures and health was successful and they would recommend thissystem as education tools. Our conceptual design approach was validated and further developments wereencouraged.In this thesis,we demonstratehow to applyMultisensory HCI methodology to design with chemical senses. Weconclude that the presented symbolic representation model of smell and taste allows communicatingtheseexperiences on digital platforms. Due to its context-dependency, MVR and MAR platforms are adequatetechnologies to be applied for this purpose.Future developments intend to explore further the conceptual approach. These developments are centredon the use of the system to induce hopefully behaviourchange. Thisthesisopens up new application possibilities of digital chemical sense communication,Multisensory HCI Design and environmental health communication.À medida que os problemas ambientais se intensificam, os sentidos químicos -isto é, o cheiroe sabor, são os sentidos mais relevantes para evidenciá-los. Como tais, os vetores de exposição ambiental que podem atingir os seres humanos compreendem o ar, alimentos, solo e água [1]. Neste contexto, compreender a ligação entre as exposições ambientais e a saúde [2] é crucial para exercerescolhas informadas, proteger o meio ambiente e adaptar a novas condições ambientais [3]. O cheiroe o saborconduzemassima experiências multissensoriais que transmitem informações de múltiplas camadas sobre eventos locais e globais [4]. No entanto, esses sentidos geralmente estão ausentes quando esses problemas são representados em sistemas digitais. A disciplina do design de Interação Humano-Computador(HCI)multissensorial investiga a inclusão dossentidos químicos em sistemas digitais [9]. O seu foco atual residena digitalização de cheirose sabores para o envio, transmissão ou substituiçãode sentidos[10]. Apesar dasexperimentaçõescomprovarem a viabilidade tecnológica, a sua disseminação está dependentedo desenvolvimento de aplicações relevantes [11]. Estatese pretendepreencher estas lacunas ao demonstrar como os sentidos químicos explicitama interconexãoentre o meio ambiente e a saúde, recorrendo a narrativas científicas econtextualizadasgeograficamente[12], [13], [14]. Apresentamos uma metodologiade design HCImultissensorial que concretizouum sistema de representação simbólica de cheiro e sabor e nos conduziu a um novo sistema de interação multissensorial, que aqui apresentamos. Descrevemos o nosso estudo exploratório Earthsensum, que integra aconceptualização, design e avaliação. Earthsensumofereceu a 16participantes do estudo experiências ambientais de cheiro e sabor relacionadas com localizações geográficasreais. Essas experiências foram representadas digitalmente através derealidade virtual(VR)e realidade aumentada(AR).Estas tecnologias conectamo mundo real e digital através de representações digitais onde podemos reproduzir as experiências multissensoriais. Os resultados do nosso estudo provaramque o sistema interativo proposto é intuitivo e pode levar não apenas a uma melhor compreensão da perceção do cheiroe sabor, como também dos problemas ambientais. O entendimentosobre a interdependência entre exposições ambientais e saúde teve êxitoe os participantes recomendariam este sistema como ferramenta para aeducação. A nossa abordagem conceptual foi positivamentevalidadae novos desenvolvimentos foram incentivados. Nesta tese, demonstramos como aplicar metodologiasde design HCImultissensorialpara projetar com ossentidos químicos. Comprovamosque o modelo apresentado de representação simbólica do cheiroe do saborpermite comunicar essas experiênciasem plataformas digitais. Por serem dependentesdocontexto, as plataformas de aplicações emVR e AR são tecnologias adequadaspara este fim.Desenvolvimentos futuros pretendem aprofundar a nossa abordagemconceptual. Em particular, aspiramos desenvolvera aplicaçãodo sistema para promover mudanças de comportamento. Esta tese propõenovas possibilidades de aplicação da comunicação dos sentidos químicos em plataformas digitais, dedesign multissensorial HCI e de comunicação de saúde ambiental

New Thermal Taste Actuation Technology for Future Multisensory Virtual Reality and Internet

Today’s virtual reality (VR) applications are mainly based on audio, visual, and haptic interactions between human and virtual world. Integrating the sense of taste into VR is difficult since we are dependent on chemical-based taste delivery systems. Therefore, developing a proper non-chemical digital taste actuation technology can unlock taste experiences in VR applications such as gaming, multisensory entertainment, remote dining, and online shopping. This paper presents the ‘Thermal Taste Machine’, a new digital taste actuation technology that can effectively produce and modify thermal taste sensations on the tongue. This device changes the temperature of the surface of the tongue within a short period of time (from 25 ◦ C to 40 ◦ C while heating and from 25 ◦ C to 10 ◦ C while cooling). We tested this device on human subjects and described the experience of thermal taste using 20 known (taste and non-taste) sensations. Our results suggested that rapidly heating the tongue produce sweetness, fatty/oiliness, electric taste, warmness, and reduced the sensibility for metallic taste. Similarly, participants reported that the cooling the tongue produced mint taste, pleasantness, and coldness. By conducting an another user study on the perceived sweetness of sucrose solutions after the thermal stimulation, we found that heating the tongue significantly enhanced the intensity of sweetness for both thermal tasters and non-thermal tasters. Also, we found that faster temperature rise on the tongue produce more intense sweet sensations for thermal tasters. We believe that this technology will be useful in two ways: First, it can produce taste sensations without using chemicals for the individuals who are sensitive to thermal taste. Second, the temperature rise of the device can be used as a way to enhance the intensity of sweetness. We believe that this technology can be used to digitally produce and enhance taste sensations in future virtual reality applications. The key novelties of this paper are as follows: 1. Development of a thermal taste actuation technology for stimulating the human taste receptors, 2. Characterization of the thermal taste produced by the device based on a set of taste related sensations and non-taste related sensations, 3. Research on enhancing the intensity for sucrose using thermal stimulation, 4. Research on how different speeds of heating affect the intensity of sweetness produced by thermal stimulation

Virtual flavor: High-fidelity simulation of real flavor experiences

Food and drink are key parts of our lives. While virtual reality has the potential to provide a high-fidelity simulation of real experiences in virtual worlds, the incorporation of flavor appreciation within these virtual experiences has largely been ignored. This article introduces a virtual flavor device to simulate real flavor experiences. The goal is to provide virtual flavor experiences, using food-safe chemicals for the three components of a flavor (taste, aroma, mouthfeel), which are perceived as “indistinguishable” from the equivalent real experience. Furthermore, because we are delivering a simulation, the same device can be used to take a user on a “flavor discovery journey” from a start flavor to a new, preferred flavor by adding or removing any amount of the components. In the first experiment, participants (N = 28) were exposed to real and virtual samples of orange juice, and the health product, rooibos tea, and asked to rate their similarity. The second experiment investigated how participants (N = 6) could move within “flavor space” from one flavor to another. The results show that it is possible to simulate, with a high degree of precision, a real flavor experience, and precisely controlled “flavor discovery journeys” can be undertaken using virtual flavors