Digitizing the chemical senses: possibilities & pitfalls

Many people are understandably excited by the suggestion that the chemical senses can be digitized; be it to deliver ambient fragrances (e.g., in virtual reality or health-related applications), or else to transmit flavour experiences via the internet. However, to date, progress in this area has been surprisingly slow. Furthermore, the majority of the attempts at successful commercialization have failed, often in the face of consumer ambivalence over the perceived benefits/utility. In this review, with the focus squarely on the domain of Human-Computer Interaction (HCI), we summarize the state-of-the-art in the area. We highlight the key possibilities and pitfalls as far as stimulating the so-called ‘lower’ senses of taste, smell, and the trigeminal system are concerned. Ultimately, we suggest that mixed reality solutions are currently the most plausible as far as delivering (or rather modulating) flavour experiences digitally is concerned. The key problems with digital fragrance delivery are related to attention and attribution. People often fail to detect fragrances when they are concentrating on something else; And even when they detect that their chemical senses have been stimulated, there is always a danger that they attribute their experience (e.g., pleasure) to one of the other senses – this is what we call ‘the fundamental attribution error’. We conclude with an outlook on digitizing the chemical senses and summarize a set of open-ended questions that the HCI community has to address in future explorations of smell and taste as interaction modalities

Development of an Intervention for Improving Food Acceptance of People with Hearing Loss

Hearing loss, defined as the partial or total inability to hear sound in one or both ears, is the most common sensory deficit in adults to date. Approximately 15% of American adults aged 18 and over report some trouble hearing. The impact of hearing loss may be profound, with consequences for the social, functional, and psychological well-being of the person. Surprisingly, very little attention has been paid on whether auditory loss can significantly impact consumers’ sensory perception and overall enjoyment of food. There were four objectives of this dissertation study. Chapter 1 aimed to determine the impacts of hearing loss on the sensory perception and acceptance of solid, and liquid food matrices with various intensities of textural attributes. Chapter 2 was designed to understand the relationships between hearing loss and aroma, flavor and taste perception and acceptance. Chapter 3 aimed to determine the impacts of environmental cues on consumers’ with hearing loss perception of their eating environments and food liking and perception in a social dining context. Finally, Chapter 4 aimed to develop an appropriate intervention that improves consumers’ with hearing loss overall food acceptance. Results showed that auditory loss impacted the overall acceptance and loudness perception of solid food samples. Pitch intensity was found as a significant negative contributor to the overall liking of solid food samples in individuals with hearing loss. In addition, subjects with hearing loss were not able to discriminate solid food samples with smaller differences in crispness. Loudness perception of liquid foods was also impacted by hearing loss. The group with hearing loss rated liquid samples as less loud compared to the group with normal hearing. No impact of hearing loss was observed on the overall enjoyment of liquid samples. Hearing loss decreased the aroma, flavor perception, and flavor acceptance of applesauce, and orange juice, but little effects were observed on taste perception. Loud external auditory cues negatively impacted the texture liking and flavor perception of food, as well as the general comfort and engagement of subjects with hearing loss during social dining. Finally, a flavor-enhanced food product proved to be an appropriate intervention plan to improve individuals with hearing loss overall food acceptance. The outcomes of this dissertation study may offer new strategies for the improvement of the enjoyment of food for consumers with auditory loss. Additionally, this research may motivate the food industry to develop new products for the growing consumer segment that are people with hearing loss

Development of an Intervention for Improving Food Acceptance of People with Hearing Loss

Hearing loss, defined as the partial or total inability to hear sound in one or both ears, is the most common sensory deficit in adults to date. Approximately 15% of American adults aged 18 and over report some trouble hearing. The impact of hearing loss may be profound, with consequences for the social, functional, and psychological well-being of the person. Surprisingly, very little attention has been paid on whether auditory loss can significantly impact consumers’ sensory perception and overall enjoyment of food. There were four objectives of this dissertation study. Chapter 1 aimed to determine the impacts of hearing loss on the sensory perception and acceptance of solid, and liquid food matrices with various intensities of textural attributes. Chapter 2 was designed to understand the relationships between hearing loss and aroma, flavor and taste perception and acceptance. Chapter 3 aimed to determine the impacts of environmental cues on consumers’ with hearing loss perception of their eating environments and food liking and perception in a social dining context. Finally, Chapter 4 aimed to develop an appropriate intervention that improves consumers’ with hearing loss overall food acceptance. Results showed that auditory loss impacted the overall acceptance and loudness perception of solid food samples. Pitch intensity was found as a significant negative contributor to the overall liking of solid food samples in individuals with hearing loss. In addition, subjects with hearing loss were not able to discriminate solid food samples with smaller differences in crispness. Loudness perception of liquid foods was also impacted by hearing loss. The group with hearing loss rated liquid samples as less loud compared to the group with normal hearing. No impact of hearing loss was observed on the overall enjoyment of liquid samples. Hearing loss decreased the aroma, flavor perception, and flavor acceptance of applesauce, and orange juice, but little effects were observed on taste perception. Loud external auditory cues negatively impacted the texture liking and flavor perception of food, as well as the general comfort and engagement of subjects with hearing loss during social dining. Finally, a flavor-enhanced food product proved to be an appropriate intervention plan to improve individuals with hearing loss overall food acceptance. The outcomes of this dissertation study may offer new strategies for the improvement of the enjoyment of food for consumers with auditory loss. Additionally, this research may motivate the food industry to develop new products for the growing consumer segment that are people with hearing loss

Dining with the Cyborgs: Disembodied Consumption and the Rhetoric of Food Media in the Digital Age

This project explores digital media productions based specifically on food and cooking in order to demonstrate that new communication technologies are increasingly incorporating all five of the bodily senses. In doing so, they contribute significantly to the emergence of new ideological apparatuses appropriate for a global community. These apparatuses – including the formation of a posthumanist subject, the use of technology to support embodied cognition, and the establishment of entertainment as an ideological institution – have become the harbingers of a rhetorical evolution. Based on the work of Gregory Ulmer, along with Jacques Derrida, N. Katherine Hayles, Donna Haraway, and Cary Wolfe, this evolution expands the work of Plato and Aristotle by overcoming the privileging of mind over body and abstract reasoning over concrete physical experience. As such hierarchies become turned on their heads, a renewed emphasis on materiality and embodiment demands virtual products that stimulate the body. As such, a phenomenon I have named disembodied consumption takes place whereby users\u27 chemical senses can be incited through participation with digital technologies. Through the stimulation of these physical senses, and in turn the connected emotions, today\u27s digital citizens are practicing the rhetorical method referred to by Ulmer as conduction. By examining sites, blogs, and postings that include references to food and flavor, I reveal examples of conduction and show how this method is necessary for the development of well-being, and the defeat of compassion fatigue in digital society

Earth as Interface: Exploring chemical senses with Multisensory HCI Design for Environmental Health Communication

As environmental problems intensify, the chemical senses -that is smell and taste, are the most relevantsenses to evidence them.As such, environmental exposure vectors that can reach human beings comprise air,food, soil and water[1].Within this context, understanding the link between environmental exposures andhealth[2]is crucial to make informed choices, protect the environment and adapt to new environmentalconditions[3].Smell and taste lead therefore to multi-sensorial experiences which convey multi-layered information aboutlocal and global events[4]. However, these senses are usually absent when those problems are represented indigital systems. The multisensory HCIdesign framework investigateschemical sense inclusion withdigital systems[5]. Ongoing efforts tackledigitalization of smell and taste for digital delivery, transmission or substitution [6]. Despite experimentsproved technological feasibility, its dissemination depends on relevant applicationdevelopment[7].This thesis aims to fillthose gaps by demonstratinghow chemical senses provide the means to link environment and health based on scientific andgeolocation narratives [8], [9],[10]. We present a Multisensory HCI design process which accomplished symbolicdisplaying smell and taste and led us to a new multi-sensorial interaction system presented herein. We describe the conceptualization, design and evaluation of Earthsensum, an exploratory case study project.Earthsensumoffered to 16 participants in the study, environmental smell and taste experiences about real geolocations to participants of the study. These experiences were represented digitally using mobilevirtual reality (MVR) and mobile augmented reality (MAR). Its technologies bridge the real and digital Worlds through digital representations where we can reproduce the multi-sensorial experiences. Our study findings showed that the purposed interaction system is intuitive and can lead not only to a betterunderstanding of smell and taste perception as also of environmental problems. Participants comprehensionabout the link between environmental exposures and health was successful and they would recommend thissystem as education tools. Our conceptual design approach was validated and further developments wereencouraged.In this thesis,we demonstratehow to applyMultisensory HCI methodology to design with chemical senses. Weconclude that the presented symbolic representation model of smell and taste allows communicatingtheseexperiences on digital platforms. Due to its context-dependency, MVR and MAR platforms are adequatetechnologies to be applied for this purpose.Future developments intend to explore further the conceptual approach. These developments are centredon the use of the system to induce hopefully behaviourchange. Thisthesisopens up new application possibilities of digital chemical sense communication,Multisensory HCI Design and environmental health communication.À medida que os problemas ambientais se intensificam, os sentidos químicos -isto é, o cheiroe sabor, são os sentidos mais relevantes para evidenciá-los. Como tais, os vetores de exposição ambiental que podem atingir os seres humanos compreendem o ar, alimentos, solo e água [1]. Neste contexto, compreender a ligação entre as exposições ambientais e a saúde [2] é crucial para exercerescolhas informadas, proteger o meio ambiente e adaptar a novas condições ambientais [3]. O cheiroe o saborconduzemassima experiências multissensoriais que transmitem informações de múltiplas camadas sobre eventos locais e globais [4]. No entanto, esses sentidos geralmente estão ausentes quando esses problemas são representados em sistemas digitais. A disciplina do design de Interação Humano-Computador(HCI)multissensorial investiga a inclusão dossentidos químicos em sistemas digitais [9]. O seu foco atual residena digitalização de cheirose sabores para o envio, transmissão ou substituiçãode sentidos[10]. Apesar dasexperimentaçõescomprovarem a viabilidade tecnológica, a sua disseminação está dependentedo desenvolvimento de aplicações relevantes [11]. Estatese pretendepreencher estas lacunas ao demonstrar como os sentidos químicos explicitama interconexãoentre o meio ambiente e a saúde, recorrendo a narrativas científicas econtextualizadasgeograficamente[12], [13], [14]. Apresentamos uma metodologiade design HCImultissensorial que concretizouum sistema de representação simbólica de cheiro e sabor e nos conduziu a um novo sistema de interação multissensorial, que aqui apresentamos. Descrevemos o nosso estudo exploratório Earthsensum, que integra aconceptualização, design e avaliação. Earthsensumofereceu a 16participantes do estudo experiências ambientais de cheiro e sabor relacionadas com localizações geográficasreais. Essas experiências foram representadas digitalmente através derealidade virtual(VR)e realidade aumentada(AR).Estas tecnologias conectamo mundo real e digital através de representações digitais onde podemos reproduzir as experiências multissensoriais. Os resultados do nosso estudo provaramque o sistema interativo proposto é intuitivo e pode levar não apenas a uma melhor compreensão da perceção do cheiroe sabor, como também dos problemas ambientais. O entendimentosobre a interdependência entre exposições ambientais e saúde teve êxitoe os participantes recomendariam este sistema como ferramenta para aeducação. A nossa abordagem conceptual foi positivamentevalidadae novos desenvolvimentos foram incentivados. Nesta tese, demonstramos como aplicar metodologiasde design HCImultissensorialpara projetar com ossentidos químicos. Comprovamosque o modelo apresentado de representação simbólica do cheiroe do saborpermite comunicar essas experiênciasem plataformas digitais. Por serem dependentesdocontexto, as plataformas de aplicações emVR e AR são tecnologias adequadaspara este fim.Desenvolvimentos futuros pretendem aprofundar a nossa abordagemconceptual. Em particular, aspiramos desenvolvera aplicaçãodo sistema para promover mudanças de comportamento. Esta tese propõenovas possibilidades de aplicação da comunicação dos sentidos químicos em plataformas digitais, dedesign multissensorial HCI e de comunicação de saúde ambiental

Interruption of an immersive experience in an ethnic restaurant

The service industry has evolved towards experience immersion, representing a viable strategy for long-term sustainability of food service – one of the top five industries providing the highest employment across Canada. This topic is explored from two perspectives, (1) the context of ethnically-themed restaurant operations, and (2) cognition as related to the attention of diners to aspects of their restaurant experience and effects of distraction on these attentional processes. This study investigates whether distraction by personal electronic devices (PEDs) reduces diners’ attention to their experience. The hypothesis was tested across two interrelated studies. Study One is comprised of qualitative work to develop a bank of questionnaire items that assess the range of content and a scaling model for the dependent variables assessing diners’ attention. Study Two investigates the dimensionality of the items comprising the dependent measures. Results suggest that, rather than a distraction, PED usage might enhance diners’ sensory immersive experience.ethnically-themed restaurant operation

Exploring and designing for multisensory interactions with 3D printed food

Experience of food is as varied as it is widespread, part of mundane activities but also embedded in rituals and celebrations. Despite its pervasive richness it has yet to be fully exploited to support embodied and multisensory experiences within Human-Computer Interaction. This thesis addresses this shortcoming, drawing on the unique qualities of food experience in combination with novel technology to design rich, affective, and embodied interactions through food. This work approaches 3D printed food as a material to design emotion- and memory-based experiences with food, and 3D printing of food as a technology for crafting multisensory user experiences in everyday contexts. These perspectives are integrated through the design and evaluation of novel interactions with 3D printed food, following a Research through Design approach combined with material approaches. Through this enquiry, novel research tools for HCI were also created for working with food, flavour, and taste. The thesis comprises seven studies that advance knowledge, based on gaps identified, and novel theoretical framings in a systematic literature review. Through a survey of user perceptions of 3D printed food, opportunities for user experience-based applications were highlighted. An identified opportunity for affective interactions through taste was considered through lab-based studies and interviews with chefs and food designers on using 3D printed food. This was extended through a co-design study with couples in romantic relationships to create flavours of 3D printed food to support emotional expression and coregulation. The use of flavours to cue experience was then explored in relation to self-defining memories with older adults. Through both co-design studies, a multisensory probe kit was built and evaluated to support designing with the senses in HCI and to further explore ideas from the study into food and memory and an app prototype designed for creating personalised flavour-based memory cues. Collectively, these studies support applications of the 3D printing of food for emotional and memory-based applications in HCI, as well as theoretical and methodological contributions to multisensory design and design with food and the body in HCI

Digitally Stimulating the Sensation of Taste Through Electrical and Thermal Stimulation

Ph.DDOCTOR OF PHILOSOPH