Olfaction-enhanced multimedia: Perspectives and challenges

This is the post-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2011 Springer VerlagOlfaction—or smell—is one of the last challenges which multimedia and multimodal applications have to conquer. Enhancing such applications with olfactory stimuli has the potential to create a more complex—and richer—user multimedia experience, by heightening the sense of reality and diversifying user interaction modalities. Nonetheless, olfaction-enhanced multimedia still remains a challenging research area. More recently, however, there have been initial signs of olfactory-enhanced applications in multimedia, with olfaction being used towards a variety of goals, including notification alerts, enhancing the sense of reality in immersive applications, and branding, to name but a few. However, as the goal of a multimedia application is to inform and/or entertain users, achieving quality olfaction-enhanced multimedia applications from the users’ perspective is vital to the success and continuity of these applications. Accordingly, in this paper we have focused on investigating the user perceived experience of olfaction-enhanced multimedia applications, with the aim of discovering the quality evaluation factors that are important from a user’s perspective of these applications, and consequently ensure the continued advancement and success of olfaction-enhanced multimedia applications

Materializing interaction

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, February 2013.Cataloged from PDF version of thesis. "September 2012."Includes bibliographical references (p. 141-148).At the boundary between people, objects and spaces, we encounter a broad range of surfaces. Their properties perform functional roles such as permeability, comfort or illumination, while conveying information such as an object's affordances, composition, or history of use. However, today surfaces are static and can neither adapt to our changing needs, nor communicate dynamic information and sense user input. As technology advances and we progress towards a world imbued with programmable materials, how will designers create physical surfaces that are adaptive and can take full advantage of our sensory apparatus? This dissertation looks at this question through the lens of a three-tier methodology consisting of the development of programmable composites; their application in design and architecture; and contextualization through a broader material and surface taxonomy. The focus is placed primarily on how materials and their aggregate surface properties can be used to engage our senses. A series of design probes and four final implementations are presented, each addressing specific programmable material and surface properties. Surflex, Sprout 1/O, and Shutters are continuous surfaces which can change shape to modify their topology, texture and permeability, and Six-Forty by Four-Eighty is a light-emitting display surface composed of autonomous and reconfigurable physical pixels. The technical and conceptual objectives of these designs are evaluated through exhibitions in a variety of public spaces, such as museums, galleries, fairs, as well as art and design festivals. This dissertation seeks to provide contributions on multiple levels, including: the development of techniques for the creation and control of programmable surfaces; the definition of a vocabulary and taxonomy to describe and compare previous work in this area; and finally, uncovering design principles for the underlying development of future programmable surface aesthetics.by Marcelo Coelho.Ph.D

Marketing the city of smells

This article explores how smell might contribute to urban identity, building on the strong links between smell, limbic processing and emotion. It critically examines existing scent marketing, psychology, and urban olfaction literatures, exploring the potential for the marketing of urban places through smell and capitalizing in particular on ambient smells that already exist within a locale. The article makes an initial threefold contribution to theory and practice: (i) demonstrating the current use of smell in city marketing, and the inherent challenges arising; (ii) identifying ways in which smell might be used in future urban place marketing activities, and in particular to more explicitly communicate the experiential attributes of being in a particular city; and (iii) proposing that olfaction may, in certain circumstances, be an effective way of incorporating a more participatory modus operandi within urban place marketing effort. The article concludes with a further overarching theoretical contribution, involving a consideration of place marketing that incorporates non-representational perspectives

Development and Testing of a Small-Size Olfactometer for the Perception of Food and Beverages in Humans

Studies of olfactory perception and about the way humans interact with, and perceive food and beverages require appropriate olfactory devices. Moreover, small size, and portable interfaces are needed within the context of Human Computer Interaction (HCI), to enrich and complete the design of different mediated experiences. In this paper, the authors tested a new portable olfactory device for the orthonasal administration of smells. The main aim was to verify if the experience generated by the odors delivered through such device can affect people's taste perception. Once established that people could perceive odors using the olfactory device, a group of participants was asked to taste two different types of food (Experiment 1) and three types of beverages (Experiment 2) and to evaluate them on a number of perceptual-dimensions (such as pleasantness, freshness, sweetness, saltiness, and bitterness). The participants could taste the food and the beverage without the presence of additional olfactory stimuli, or under conditions where olfactory stimuli (the smell of chocolate or citrus) were also presented using the device. The results showed that the participants' evaluation of food and beverages was significantly modulated by the concurrently presented odors. The experimental results suggest that: (1) the device is effective in controlling the delivery of odors to human participants without the complexity of management that often affect larger odors delivery systems; (2) odors administered by means of such device can have an effects on food and beverage perception, without the need to change their chemical properties

Estudo de viabilidade de desenvolvimento de um demonstrador de Realidade Virtual para as operações de servicing sobre a aeronave A-29B Super Tucano

Since the birth of aviation, the industry has experienced changes on how an airline operator must perform maintenance operations and how new technicians are trained to perform them to ensure the airworthiness of the aircraft, where the reduction of human errors resulting from non-compliance to the stipulated regulations are one of the main concerns. With the surge of new Virtual Reality technologies, EASA Part-147 approved organizations are enabled to incorporate and direct this new knowledge to the training of new EASA Part-66 approved technicians. However, Virtual Reality technologies must only be used and thought of as a complementary training tool, as it does not provide competencies to perform manual actions as welding, drilling, screwing, among others. Virtual Reality allows the trainee to test his theoretical knowledge at any given time without being limited by the availability of real equipment/components necessary to continue his training, without creating the risk of causing any material damage. All the training performed while resorting to the use of Virtual Reality technologies aims to improve knowledge retention, resulting in better approval rates. In this dissertation, a feasibility study and development of a Virtual Reality demonstrator directed to maintenance training for fuel servicing operations over an A-29B Super Tucano aeroplane must be performed, with the reduction of incidents/accidents reports resulting from wrongly performed procedures set as the main goal. The Virtual Reality application was built on a game engine called Unity, where the user can choose from VR hardware currently available or a mouse and keyboard to perform a fueling procedure. For each procedure, there are three modes available, guided, unguided and VR versions. The guided procedure will provide the user with an oriented experience, where no deviation from the maintenance manual for the fuel servicing action chosen can be performed. Each task will be highlighted in the correct order, providing a visual indication of the task to be executed. An exact copy of the Embraer maintenance manual for the designated procedure will also be provided to be used as a checklist. The unguided procedure allows deviation from the maintenance manual and wrong actions are allowed to be performed. However, a score will be given at the end of the procedure to evaluate how accurately the manual was followed. Embraer maintenance manual will be also provided, but with no checklist. Lastly, the VR version will follow the same fundamentals as the guided version. However, the input and output devices used on the previous 2 versions are replaced by an HMD and two handheld controllers.Desde o seu nascimento, a indústria aeronáutica tem vindo a vivenciar mudanças relativamente a como um operador aéreo realiza as suas atividades de manutenção e como treina os seus técnicos de modo a manter a aeronavegabilidade da sua frota, sendo que a ocorrência de incidentes/acidentes que resultem do incumprimento das regulações/procedimentos impostos pela operadora aérea sejam um foco de melhoria. Com a evolução e surgimento de tecnologias de Realidade Virtual, foi possível a entidades formadoras aprovadas segundo a regulação EASA Part-147, incorporar e direcionar este conhecimento para o treino de técnicos de manutenção. No entanto as mesmas apenas devem ser usadas como uma ferramenta complementar ao treino prático, pois não permitem que se adquira aptidões práticas para realizar tarefas manuais como soldadura, furação, aparafusamento, entre outros. Estas tecnologias permitem que o instruendo tenha a possibilidade de testar os seus conhecimentos teóricos a qualquer momento sem estar limitado pela disponibilidade de componentes reais para dar continuidade à sua formação. Todo este processo de treino com auxílio a aplicações de Realidade Virtual pretende melhorar a retenção de conhecimento, que necessariamente levará a um aumento nas taxas de aprovação dos instruendos. Neste trabalho pretende-se realizar o estudo de viabilidade e desenvolvimento de um demonstrador de Realidade Virtual direcionado ao treino de manutenção, para operações de fuel servicing para a aeronave A29B Super Tucano, com o objetivo de reduzir o número de relatos de incidentes/acidentes resultantes de um procedimento mal-executado. A aplicação de Realidade Virtual desenvolvida no decorrer de este trabalho de investigação, teve por base o motor de jogo Unity, em que o utilizador pode recorrer ao hardware VR atualmente disponível, ou a rato e teclado para realizar o procedimento de fuel servicing. A utilização em simultâneo das opções anteriormente referidas não é possível. Para cada procedimento, três modos de realização da tarefa são possíveis, a versão guiada, não-guiada e RV. A versão guiada fornece ao utilizador uma experiência orientada, onde qualquer desvio do manual de manutenção para o procedimento de reabastecimento escolhido não é permitido. Cada tarefa, segundo a ordem tal como surgem no manual estará destacada, de modo a fornecer uma indicação visual da tarefa a ser executada. Uma cópia exata do manual da Embraer para o procedimento escolhido é também fornecida para ser usado como um checklist. A versão não-guiada permite desvios aos procedimentos do manual de manutenção, assim como a realização de ações erradas do procedimento. No entanto, no final do treino são fornecidos elementos para quantificar os eventuais desvios do manual que tenham ocorrido durante o procedimento. Neste caso, o manual é igualmente fornecido, mas sem checklist. Por fim, na versão de Realidade Virtual, tal como acontece na versão guiada, os fundamentos são iguais, no entanto, a diferença que merece maior destaque é o facto de os aparelhos de input e output deixem de ser o rato, teclado e monitor e passamos a usar um HMD e handheld controllers

Multisensory Integration as per Technological Advances: A Review

Multisensory integration research has allowed us to better understand how humans integrate sensory information to produce a unitary experience of the external world. However, this field is often challenged by the limited ability to deliver and control sensory stimuli, especially when going beyond audio–visual events and outside laboratory settings. In this review, we examine the scope and challenges of new technology in the study of multisensory integration in a world that is increasingly characterized as a fusion of physical and digital/virtual events. We discuss multisensory integration research through the lens of novel multisensory technologies and, thus, bring research in human–computer interaction, experimental psychology, and neuroscience closer together. Today, for instance, displays have become volumetric so that visual content is no longer limited to 2D screens, new haptic devices enable tactile stimulation without physical contact, olfactory interfaces provide users with smells precisely synchronized with events in virtual environments, and novel gustatory interfaces enable taste perception through levitating stimuli. These technological advances offer new ways to control and deliver sensory stimulation for multisensory integration research beyond traditional laboratory settings and open up new experimentations in naturally occurring events in everyday life experiences. Our review then summarizes these multisensory technologies and discusses initial insights to introduce a bridge between the disciplines in order to advance the study of multisensory integration

Earth as Interface: Exploring chemical senses with Multisensory HCI Design for Environmental Health Communication

As environmental problems intensify, the chemical senses -that is smell and taste, are the most relevantsenses to evidence them.As such, environmental exposure vectors that can reach human beings comprise air,food, soil and water[1].Within this context, understanding the link between environmental exposures andhealth[2]is crucial to make informed choices, protect the environment and adapt to new environmentalconditions[3].Smell and taste lead therefore to multi-sensorial experiences which convey multi-layered information aboutlocal and global events[4]. However, these senses are usually absent when those problems are represented indigital systems. The multisensory HCIdesign framework investigateschemical sense inclusion withdigital systems[5]. Ongoing efforts tackledigitalization of smell and taste for digital delivery, transmission or substitution [6]. Despite experimentsproved technological feasibility, its dissemination depends on relevant applicationdevelopment[7].This thesis aims to fillthose gaps by demonstratinghow chemical senses provide the means to link environment and health based on scientific andgeolocation narratives [8], [9],[10]. We present a Multisensory HCI design process which accomplished symbolicdisplaying smell and taste and led us to a new multi-sensorial interaction system presented herein. We describe the conceptualization, design and evaluation of Earthsensum, an exploratory case study project.Earthsensumoffered to 16 participants in the study, environmental smell and taste experiences about real geolocations to participants of the study. These experiences were represented digitally using mobilevirtual reality (MVR) and mobile augmented reality (MAR). Its technologies bridge the real and digital Worlds through digital representations where we can reproduce the multi-sensorial experiences. Our study findings showed that the purposed interaction system is intuitive and can lead not only to a betterunderstanding of smell and taste perception as also of environmental problems. Participants comprehensionabout the link between environmental exposures and health was successful and they would recommend thissystem as education tools. Our conceptual design approach was validated and further developments wereencouraged.In this thesis,we demonstratehow to applyMultisensory HCI methodology to design with chemical senses. Weconclude that the presented symbolic representation model of smell and taste allows communicatingtheseexperiences on digital platforms. Due to its context-dependency, MVR and MAR platforms are adequatetechnologies to be applied for this purpose.Future developments intend to explore further the conceptual approach. These developments are centredon the use of the system to induce hopefully behaviourchange. Thisthesisopens up new application possibilities of digital chemical sense communication,Multisensory HCI Design and environmental health communication.À medida que os problemas ambientais se intensificam, os sentidos químicos -isto é, o cheiroe sabor, são os sentidos mais relevantes para evidenciá-los. Como tais, os vetores de exposição ambiental que podem atingir os seres humanos compreendem o ar, alimentos, solo e água [1]. Neste contexto, compreender a ligação entre as exposições ambientais e a saúde [2] é crucial para exercerescolhas informadas, proteger o meio ambiente e adaptar a novas condições ambientais [3]. O cheiroe o saborconduzemassima experiências multissensoriais que transmitem informações de múltiplas camadas sobre eventos locais e globais [4]. No entanto, esses sentidos geralmente estão ausentes quando esses problemas são representados em sistemas digitais. A disciplina do design de Interação Humano-Computador(HCI)multissensorial investiga a inclusão dossentidos químicos em sistemas digitais [9]. O seu foco atual residena digitalização de cheirose sabores para o envio, transmissão ou substituiçãode sentidos[10]. Apesar dasexperimentaçõescomprovarem a viabilidade tecnológica, a sua disseminação está dependentedo desenvolvimento de aplicações relevantes [11]. Estatese pretendepreencher estas lacunas ao demonstrar como os sentidos químicos explicitama interconexãoentre o meio ambiente e a saúde, recorrendo a narrativas científicas econtextualizadasgeograficamente[12], [13], [14]. Apresentamos uma metodologiade design HCImultissensorial que concretizouum sistema de representação simbólica de cheiro e sabor e nos conduziu a um novo sistema de interação multissensorial, que aqui apresentamos. Descrevemos o nosso estudo exploratório Earthsensum, que integra aconceptualização, design e avaliação. Earthsensumofereceu a 16participantes do estudo experiências ambientais de cheiro e sabor relacionadas com localizações geográficasreais. Essas experiências foram representadas digitalmente através derealidade virtual(VR)e realidade aumentada(AR).Estas tecnologias conectamo mundo real e digital através de representações digitais onde podemos reproduzir as experiências multissensoriais. Os resultados do nosso estudo provaramque o sistema interativo proposto é intuitivo e pode levar não apenas a uma melhor compreensão da perceção do cheiroe sabor, como também dos problemas ambientais. O entendimentosobre a interdependência entre exposições ambientais e saúde teve êxitoe os participantes recomendariam este sistema como ferramenta para aeducação. A nossa abordagem conceptual foi positivamentevalidadae novos desenvolvimentos foram incentivados. Nesta tese, demonstramos como aplicar metodologiasde design HCImultissensorialpara projetar com ossentidos químicos. Comprovamosque o modelo apresentado de representação simbólica do cheiroe do saborpermite comunicar essas experiênciasem plataformas digitais. Por serem dependentesdocontexto, as plataformas de aplicações emVR e AR são tecnologias adequadaspara este fim.Desenvolvimentos futuros pretendem aprofundar a nossa abordagemconceptual. Em particular, aspiramos desenvolvera aplicaçãodo sistema para promover mudanças de comportamento. Esta tese propõenovas possibilidades de aplicação da comunicação dos sentidos químicos em plataformas digitais, dedesign multissensorial HCI e de comunicação de saúde ambiental

Examining the sense of agency in human-computer interaction

Humans are agents, we feel that we control the course of events on our everyday life. This refers to the Sense of Agency (SoA). This experience is not only crucial in our daily life, but also in our interaction with technology. When we manipulate a user interface (e.g., computer, smartphone, etc.), we expect that the system responds to our input commands with feedback, as we desire to feel that we are in charge of the interaction. If this interplay elicits a SoA, then the user will perceive an instinctive feeling of “I am controlling this”. Although research in Human-Computer Interaction (HCI) pursuits the design of intuitive and responsive systems, most of the current studies have been focussed mainly on interaction techniques (e.g., software-hardware) and User Experience (UX) (e.g., comfort, usability, etc.), and very little has been investigated in terms of the SoA i.e., the conscious experience of being in control regarding the interaction. In this thesis, we present an experimental exploration of the role of the SoA in interaction paradigms typical of HCI. After two chapters of introduction and related work, we describe a series of studies that explore agency implication in interaction with systems through human senses such as vision, audio, touch and smell. Chapter 3 explores the SoA in mid-air haptic interaction through touchless actions. Then, Chapter 4 examines agency modulation through smell and its application for olfactory interfaces. Chapter 5 describes two novel timing techniques based on auditory and haptic cues that provide alternative timing methods to the traditional Libet clock. Finally, we conclude with a discussion chapter that highlights the importance of our SoA during interactions with technology as well as the implications of the results found, in the design of user interfaces