Towards multisensory storytelling with taste and flavor

Film makers, producers, and theaters have continuously looked at ways to embody and/or integrate multiple sensory cues in the experiences they deliver. Here, we present a reflection on past attempts, lessons learnt, and future directions for the community around multisensory TV, film, and multimedia as a historical, though renewed, space of content creation. In particular, we present an overview of what we call "tasty film", that is, film involving taste, flavor, and more broadly food and drink inputs, to influence the audience experience. We suggest that such elements should be considered beyond "add-ons" in film experiences. We advocate for experimentation with new kinds of storytelling taking inspiration from multisensory design research and work on sensory substitution. We position this article as a starting point for anyone interested in multisensory film involving taste, flavor, and foods

3rd International Workshop on Multisensory Approaches to Human-Food Interaction

This is the introduction paper to the third version of the workshop on 'Multisensory Approaches to Human-Food Interaction' organized at the 20th ACM International Conference on Multimodal Interaction in Boulder, Colorado, on October 16th, 2018. This workshop is a space where the fast growing research on Multisensory Human-Food Interaction is presented. Here we summarize the workshop's key objectives and contributions

Reflection on the design of food systems and experiences for sustainable transformations

The importance of food and technology in modern society is undeniable. Technological advances have revolutionized how we produce, distribute and prepare food beyond local boundaries, and even how we eat. Eating is one of the most multisensory experiences in everyday life. All of our five senses (i.e. taste, smell, vision, hearing and touch) are involved. We first eat with our eyes, we can smell the food before we taste it, and then experience its textures and flavours in our mouth. However, the experience does not stop there. The sounds that come both from the environment in which we are immersed in while eating and our interactions with the food (e.g. chewing) and utensils we use to eat further influence our eating experiences. In all that, digital technology plays an increasingly important role, especially using emerging immersive technologies such as virtual and augmented reality (VR/AR). Designing at the intersection between technology and food requires multi-stakeholder commitments and a human experience-centred approach. Furthermore, it is essential to look beyond disciplinary boundaries and account for insights on various levels including the perceptual effects, experiential layers and technological advancements

Multisensory experiences in HCI

The use of vision and audition for interaction dominated the field of human-computer interaction (HCI) for decades, despite the fact that nature has provided us with many more senses for perceiving and interacting with the world around us. Recently, HCI researchers have started trying to capitalize on touch, taste, and smell when designing interactive tasks, especially in gaming, multimedia, and art environments. Here we provide a snapshot of our research into touch, taste, and smell, which we’re carrying out at the Sussex Computer Human Interaction (SCHI—pronounced “sky”) Lab at the University of Sussex in Brighton, UK

Multisensory Experiences: A Primer

We present a primer on multisensory experiences, the different components of this concept, as well as a reflection of its implications for individuals and society. We define multisensory experiences, illustrate how to understand them, elaborate on the role of technology in such experiences, and present the three laws of multisensory experiences, which can guide discussion on their implications. Further, we introduce the case of multisensory experiences in the context of eating and human-food interaction to illustrate how its components operationalize. We expect that this article provides a first point of contact for those interested in multisensory experiences, as well as multisensory experiences in the context of human-food interaction.</jats:p

Virtually tasty: An investigation of the effect of ambient lightning and 3D-shaped taste stimuli on taste perception in virtual reality

Taste perception is influenced by sensory information not only about the food itself but also about the external environment where the food is tasted. Prior studies have shown that both visual attributes of the environment (e.g., light colour, location) and the shape associated to food (e.g., plates, cutlery) can influence people's taste perception and expectations. However, previous studies are typically based on non-edible shapes usually shown as 2D images or presented as 3D tangible objects aimed to be perceived by subjects' hand. Therefore, the effect of mouthfeel of differently shaped foods on taste perception remains unclear. Capitalising on the advantages of virtual reality (VR) to manipulate multisensory features, we explore the effects of coloured (red, blue, neutral) virtual environments on the taste (sweet, neutral) perception of differently shaped taste samples (rounded/spiky shapes according to the Kiki-Bouba paradigm). Overall, our results showed increased ratings of sweetness when participants tasted Bouba-shaped samples (rounded) relative to Kiki-shaped samples (spiky) suggesting that tactile attributes perceived inside the mouth can influence sweetness perception. Furthermore, we concluded that lighting colour in a virtual setting might dampen experiences of sweetness. However, this effect may only be present when there is a cross-modal correspondence with taste. Based on our findings, we conclude by describing considerations for designing eating experiences in VR

Is Food a (cross-cultural/interpersonal) communication medium?

The central question of this Master thesis is whether food is a (cross-cultural or interpersonal) communication medium or not. And if it is, how does it work and how does it relate to visual communication? This question is looked at through a multidisciplinary theoretical lens and accompanied by case studies. The newly emerged discipline of food design is the correct field to carry out such research. But what is food design? Is it even a true design discipline? This thesis argues that food design is a design discipline, although its territory within design has not been clearly defined yet. This thesis is an attempt to decode food design. It is believed that food is a form of nonverbal communication, which is used in a very similar manner to a language. The literature review includes a comprehensive study of different communication models and theories and considers food’s communicational aspects from both macro and micro levels. Food is also very visual. Therefore, from a visual communication point of view, it deserves to be studied properly. However, food goes beyond just sight. It stimulates all the five senses. Reviewing the comprehensive research on the multi-sensory dimensions of food carried out by neuro-gastronomists, reveals more about the communicational potential of food. Food is, naturally, the territory of chefs. That is why another part of this thesis is dedicated to the developments of the world of gastronomy, in an attempt to find out important clues, which link gastronomy to design. In the recent decades the chefs, designers and communication scientists have met each other in order to elevate food to a totally new level. Through studying the elements of this paradigm shift this thesis is finalised with two case studies, which examine food as a communication medium in practice. Through this multidisciplinary theoretical study and the case studies this Master thesis provides a framework for food design as a discipline and shows that food is, indeed, a communication medium, which can connect people on a visceral level, beyond cultural and linguistic barriers

Investigating the impact of food tourism vlogger entrepreneurs' language characteristics on audiences' attitude and behaviours

Purpose Despite entrepreneurial linguistic style gaining increased attention in entrepreneurship studies, the field for digital vlogger entrepreneurs still lacks a comprehensive understanding of how linguistic patterns enhance audiences attitude and behaviour. This study aims to propose a conceptual model of “language-mental imagery-attitude-behaviour model” that leads to the examination of rich sensory language style of food travel vlogger entrepreneurs and its persuasive effect on audiences' attitude and behavioural intention. Design/methodology/approach The present study utilises a stimulus-based survey method that involves a sensory-rich vlog script extracted from a high social media engagement authentic vlog. Data are collected through an online questionnaire distributed to a sample of 355 participants via the Amazon Turk mechanism. The study employs confirmatory factor analysis and structural equation modelling to test the proposed hypotheses, with the aim of contributing to the advancement of theories of embodied cognition in entrepreneurial language by examining the attitudes and behaviours of audiences exposed to sensory-rich language. The findings of this research provide valuable insights into the effects of sensory-rich language on audience responses and can inform future research on the role of embodied cognition in entrepreneurial communication. Findings The findings demonstrate that vlogger entrepreneurial sensory-rich linguistic communication style positively influence audiences' attitude, behavioural involvement with food and intention to taste. Visit intention will be enhanced via the mediating effects of attitude, behavioural involvement with food and intention to taste. Practical implications This research highlights the significance of sensory-rich language for vlogger entrepreneurs in entrepreneurial communication, digital storytelling and for destination marketing enterprises in creating a digital sensory engagement marketing strategy. Originality/value The study contributes to the literature by elucidating the theories of embodied cognition in entrepreneurial communication. By examining the relationships between vlogger communication evoked mental imagery, audiences attitude and behaviours, this study provides novel insights into the effectiveness of sensory-rich language in vlogger entrepreneurial communication and its impact on audience engagement. These findings have important implications for communication scholars and practitioners alike, shedding light on the role of embodied cognition in entrepreneurial language and the potential of sensory-rich language to enhance audience engagement

Food Reality Experience

How can virtual reality be a medium to enhance inclusion and interaction among tablemates? How is it possible to humanize technological divide, leveraging accessibility and literacy of people of different ages, towards the correct use of technological devices? This paper discuss how the use of ICT at Social Tables has created a new language and has fostered connections and interactions among participants; how it has spread the knowledge of food, places, and traditions from different countries; how it has created and activated a fluid experience of all the senses with which we are endowed. This has been made possible, through the experiments conducted during the Food Reality Experience: performative dinners where ingredients and food are mixed with 3D videos

Earth as Interface: Exploring chemical senses with Multisensory HCI Design for Environmental Health Communication

As environmental problems intensify, the chemical senses -that is smell and taste, are the most relevantsenses to evidence them.As such, environmental exposure vectors that can reach human beings comprise air,food, soil and water[1].Within this context, understanding the link between environmental exposures andhealth[2]is crucial to make informed choices, protect the environment and adapt to new environmentalconditions[3].Smell and taste lead therefore to multi-sensorial experiences which convey multi-layered information aboutlocal and global events[4]. However, these senses are usually absent when those problems are represented indigital systems. The multisensory HCIdesign framework investigateschemical sense inclusion withdigital systems[5]. Ongoing efforts tackledigitalization of smell and taste for digital delivery, transmission or substitution [6]. Despite experimentsproved technological feasibility, its dissemination depends on relevant applicationdevelopment[7].This thesis aims to fillthose gaps by demonstratinghow chemical senses provide the means to link environment and health based on scientific andgeolocation narratives [8], [9],[10]. We present a Multisensory HCI design process which accomplished symbolicdisplaying smell and taste and led us to a new multi-sensorial interaction system presented herein. We describe the conceptualization, design and evaluation of Earthsensum, an exploratory case study project.Earthsensumoffered to 16 participants in the study, environmental smell and taste experiences about real geolocations to participants of the study. These experiences were represented digitally using mobilevirtual reality (MVR) and mobile augmented reality (MAR). Its technologies bridge the real and digital Worlds through digital representations where we can reproduce the multi-sensorial experiences. Our study findings showed that the purposed interaction system is intuitive and can lead not only to a betterunderstanding of smell and taste perception as also of environmental problems. Participants comprehensionabout the link between environmental exposures and health was successful and they would recommend thissystem as education tools. Our conceptual design approach was validated and further developments wereencouraged.In this thesis,we demonstratehow to applyMultisensory HCI methodology to design with chemical senses. Weconclude that the presented symbolic representation model of smell and taste allows communicatingtheseexperiences on digital platforms. Due to its context-dependency, MVR and MAR platforms are adequatetechnologies to be applied for this purpose.Future developments intend to explore further the conceptual approach. These developments are centredon the use of the system to induce hopefully behaviourchange. Thisthesisopens up new application possibilities of digital chemical sense communication,Multisensory HCI Design and environmental health communication.À medida que os problemas ambientais se intensificam, os sentidos químicos -isto é, o cheiroe sabor, são os sentidos mais relevantes para evidenciá-los. Como tais, os vetores de exposição ambiental que podem atingir os seres humanos compreendem o ar, alimentos, solo e água [1]. Neste contexto, compreender a ligação entre as exposições ambientais e a saúde [2] é crucial para exercerescolhas informadas, proteger o meio ambiente e adaptar a novas condições ambientais [3]. O cheiroe o saborconduzemassima experiências multissensoriais que transmitem informações de múltiplas camadas sobre eventos locais e globais [4]. No entanto, esses sentidos geralmente estão ausentes quando esses problemas são representados em sistemas digitais. A disciplina do design de Interação Humano-Computador(HCI)multissensorial investiga a inclusão dossentidos químicos em sistemas digitais [9]. O seu foco atual residena digitalização de cheirose sabores para o envio, transmissão ou substituiçãode sentidos[10]. Apesar dasexperimentaçõescomprovarem a viabilidade tecnológica, a sua disseminação está dependentedo desenvolvimento de aplicações relevantes [11]. Estatese pretendepreencher estas lacunas ao demonstrar como os sentidos químicos explicitama interconexãoentre o meio ambiente e a saúde, recorrendo a narrativas científicas econtextualizadasgeograficamente[12], [13], [14]. Apresentamos uma metodologiade design HCImultissensorial que concretizouum sistema de representação simbólica de cheiro e sabor e nos conduziu a um novo sistema de interação multissensorial, que aqui apresentamos. Descrevemos o nosso estudo exploratório Earthsensum, que integra aconceptualização, design e avaliação. Earthsensumofereceu a 16participantes do estudo experiências ambientais de cheiro e sabor relacionadas com localizações geográficasreais. Essas experiências foram representadas digitalmente através derealidade virtual(VR)e realidade aumentada(AR).Estas tecnologias conectamo mundo real e digital através de representações digitais onde podemos reproduzir as experiências multissensoriais. Os resultados do nosso estudo provaramque o sistema interativo proposto é intuitivo e pode levar não apenas a uma melhor compreensão da perceção do cheiroe sabor, como também dos problemas ambientais. O entendimentosobre a interdependência entre exposições ambientais e saúde teve êxitoe os participantes recomendariam este sistema como ferramenta para aeducação. A nossa abordagem conceptual foi positivamentevalidadae novos desenvolvimentos foram incentivados. Nesta tese, demonstramos como aplicar metodologiasde design HCImultissensorialpara projetar com ossentidos químicos. Comprovamosque o modelo apresentado de representação simbólica do cheiroe do saborpermite comunicar essas experiênciasem plataformas digitais. Por serem dependentesdocontexto, as plataformas de aplicações emVR e AR são tecnologias adequadaspara este fim.Desenvolvimentos futuros pretendem aprofundar a nossa abordagemconceptual. Em particular, aspiramos desenvolvera aplicaçãodo sistema para promover mudanças de comportamento. Esta tese propõenovas possibilidades de aplicação da comunicação dos sentidos químicos em plataformas digitais, dedesign multissensorial HCI e de comunicação de saúde ambiental